diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 00000000..e195335a --- /dev/null +++ b/.gitattributes @@ -0,0 +1,2 @@ +# Tree-sitter C sources are generated from ext/tree-sitter-andy-cpp/grammar.js. +ext/tree-sitter-andy-cpp/src/*.c linguist-generated=true diff --git a/CLAUDE.md b/CLAUDE.md index 8ba8deed..e802518a 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -6,7 +6,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co - Run `cargo fmt` - Run `cargo clippy` and fix all warnings - Ensure all tests pass (`cargo test`) -- Do not leave `TODO` comments in code — either fix the issue immediately or open a GitHub issue and record it in `TODO.md` +- Do not leave `TODO` comments in code — either fix the issue immediately or open a GitHub issue ## Common Commands diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 75ab2188..5a4d35c2 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -107,7 +107,7 @@ cargo test ``` If you find yourself writing a `TODO` comment, please open a GitHub -issue instead and record it in [`TODO.md`](TODO.md). +issue instead. ## Git conventions diff --git a/README.md b/README.md index 4144612e..77c6eae0 100644 --- a/README.md +++ b/README.md @@ -1,20 +1,23 @@ # Andy C++ -Andy C++ is a small scripting language built for personal use — primarily for solving [Advent of Code](https://adventofcode.com/) puzzles and quick one-off scripts. It is not intended for production use. +Andy C++ is a programming language built primarily for solving [Advent of Code](https://adventofcode.com/) puzzles and quick one-off scripts. It's syntax and semantics are designed to feel familiar to those who know Rust, while offering some of the high-level flexibility from Python (without all of its pitfalls). -## Clanker Disclosure +Since version 0.3.0 the language uses a custom byte-code VM based on the Crafting Interpreters book. Code is: lexed, parsed, analysed, compiled and executed in separate steps. The project ships as a single binary that contains everything including a REPL, LSP and basic profiler. -After the 0.2.0 release I've been using this project as a playground to experiment and learn more about AI tools such as -Claude Code. I still spend countless hours pondering every little decision, but in the end most of the new code is -written by AI. I understand a lot of people are uncomfortable with that, and if that means you'd rather stay away from -this project then that's totally fine. If you would like to contribute to this project using AI tools, all I ask you is -that you (the human) carefully review your submissions and that you include in your PR description which parts were -generated by AI and which models you've used. - -_This section was written by a human._ +**Features:** +* Arbitrary-precision arithmetic (including rational numbers) +* A dynamic type system that tries to rescue you with static type checks ahead of compilation +* Higher order functions and closures +* `a.map(b)` and `map(a,b)` are [exactly the same](https://timfennis.github.io/andy-cpp/features/method-call-syntax.html) +* Functions that take two arguments can be used in [augmented assignment](https://timfennis.github.io/andy-cpp/features/augmented-assignment.html): `l map= fn(x) => x + 3` +* Marking a function `pure` enables [memoization](https://timfennis.github.io/andy-cpp/features/memoization.html) (but only through hashing, no equality checks; use at your own risk) +* Built in support for default dictionaries, MinHeap, MaxHeap and Deque +* A pretty rich but work in progress standard library ## Getting Started +The best way to try this project is to build it from source using the rust toolchain. There are binary releases but those are symbollic milestones, and contain bugs. + ### Prerequisites You need a working [Rust toolchain](https://rustup.rs/). @@ -22,7 +25,7 @@ You need a working [Rust toolchain](https://rustup.rs/). ### Install ```bash -cargo install --path ndc_bin +cargo install --git https://github.com/timfennis/andy-cpp ``` This installs the `ndc` binary. You can then run a script: @@ -41,170 +44,21 @@ To browse the built-in function documentation: ```bash ndc docs -ndc docs sort # filter by keyword +ndc docs map # filter by the keyword 'map' ``` For the language manual, see . -### VS Code Extension - -A VS Code extension is available on the -[Open VSX Registry](https://open-vsx.org/extension/TimFennis/andy-cpp). It provides: - -- Syntax highlighting for `.ndc` files -- Language server (LSP) with diagnostics, inlay type hints, and completions -- A "Run Script" command that executes the current file in the integrated terminal +### Editor support -The extension launches the LSP automatically using the `ndc` binary. If `ndc` is not on the PATH -that VS Code uses (common when installed via a shell like fish), set the `andy-cpp.ndcPath` setting -to the full path of the binary. - -### JetBrains IDEs (RustRover, IntelliJ, …) - -JetBrains IDEs are supported by importing `ext/andy-cpp` as a TextMate bundle (syntax -highlighting) and connecting the LSP4IJ plugin to `ndc lsp` (diagnostics, hover, -completion, inlay type hints, go to definition). See the +Installation and configuration instructions for VS Code, JetBrains IDEs, Neovim, +Helix, and other LSP-capable editors are available on the [editor support page](https://timfennis.github.io/andy-cpp/tooling/editor-support.html) -in the manual for setup instructions. - -### Other editors - -If you prefer a different editor, you can start the language server manually over stdio: - -```bash -ndc lsp --stdio -``` - -Point your editor's LSP client at this command for `.ndc` files. - -## Example - -Currently, the language has quite a lot of features allowing you to write some pretty neat programs. - -### Factorial - -You can produce very large numbers quite quickly because we use the [num](https://docs.rs/num/latest/num/) crate under -the hood. - -```ndc -fn factorial(n) { - if n == 1 { - return 1 - } - - n * factorial(n - 1) -} - -print(factorial(100)); - -// You can call all functions as if they are methods on an object -print(100.factorial()); -``` - -### Overloading - -Functions are matched by name and arity, so you can define multiple versions of the same function. - -```ndc -fn add(n) { n + 1 } -fn add(a, b) { a + b } - -// add(5) = 6, add(4) = 5, add(6, 5) = 11 -print(add(add(5), add(4))); // prints 11 -// 5.add() = 6, 4.add() = 5, 6.add(5) = 11 -print(add(5).add(4.add())); // prints 11 as well -``` - -### Use functions as augmented assignment operators - -Many functions can be used to -create [augmented assignment operators](https://blog.vero.site/post/noulith#augmented-assignment). - -```ndc -let r = 0; +in the manual. -for i in 0..100 { - // roughly translates to r = max(r, i * 8333446703 % 94608103) - r max= i * 8333446703 % 94608103; -} +## Examples -print(r); -``` - -### Higher-order functions - -Anonymous functions can be passed to built-ins like `map`, `filter`, and `sorted`. - -```ndc -let numbers = [5, 3, 8, 1, 9, 2, 7]; - -let evens = numbers.filter(fn(x) => x % 2 == 0); -let squares = numbers.map(fn(x) => x * x); -let top3 = numbers.sorted().reversed()[0..3]; - -print(evens); // [8, 2] -print(squares); // [25, 9, 64, 1, 81, 4, 49] -print(top3); // [9, 8, 7] -``` - -### Tuple vectorization - -Arithmetic operators work element-wise on tuples of numbers. - -```ndc -let a = (1, 2, 3); -let b = (4, 5, 6); - -print(a + b); // (5, 7, 9) -print(a * b); // (4, 10, 18) -print(a * 2); // (2, 4, 6) -print(10 - a); // (9, 8, 7) -``` - -### Maps and Sets - -Maps and sets share the same `%{}` syntax. - -```ndc -let map = %{"foo": "bar", "baz": 42}; -print(map["foo"]); // bar - -let set = %{1, 2, 3, 4}; -print(3 in set); // true -``` - -A default value for missing keys can be specified, similar to Python's `defaultdict`. - -```ndc -let counts = %{: 0}; -for word in ["apple", "banana", "apple", "cherry", "banana", "apple"] { - counts[word] += 1; -} -print(counts["apple"]); // 3 -print(counts["banana"]); // 2 -``` - -### List comprehensions - -The language supports list comprehensions with the same semantics as Haskell but a syntax slightly more similar to -Python. - -```ndc -fn pythagorean_triples(n) { - return [(a, b, c) for a in 1..=n, - b in a..=n, - c in b..=n, - if a ^ 2 + b ^ 2 == c ^ 2] -} -``` - -The same features are also available in regular for iterations. - -```ndc -for a in 1..=25, b in a..=25, c in b..=25, if a ^ 2 + b ^ 2 == c ^ 2 { - print(a, b, c); -} -``` +Many examples of the language can be found in [this](https://github.com/timfennis/advent-of-code-ndc) repository. ## Thanks @@ -212,3 +66,7 @@ This language and implementation was inspired by Robert Nystrom's book [Crafting Interpreters](https://craftinginterpreters.com/). I've also taken plagiaristic levels of inspiration from [Noulith](https://github.com/betaveros/noulith) which is the language that inspired me to read the book in the first place. + +## LLM Disclosure + +This project has had various levels of LLM involvement during its lifetime. The codebase is designed by humans and is meant to be read and maintained primarily by humans. Large language models are tools and, like all other tools, have strengths and limitations. They are allowed in this project when used responsibly. All contributions will be judged on their merits. diff --git a/TODO.md b/TODO.md deleted file mode 100644 index 186dbaa7..00000000 --- a/TODO.md +++ /dev/null @@ -1,96 +0,0 @@ -# TODO - -Open tasks collected from in-code comments. Resolve by implementing or opening a GitHub issue. - ---- - -## Medium: Type-checking warnings and errors - -These can be implemented incrementally once the basic type system is stable. - -- **Logical-operator operand types** (`ndc_analyser/src/analyser.rs` ~line 58) - `and` / `or` operands are not checked to be `Bool`. Should emit a warning or error when the - operand type is known and incompatible. - -- **Mismatched `if` branch types** (`ndc_analyser/src/analyser.rs` ~line 178) - When `true`-branch and `false`-branch types differ (and neither is `Any`), a warning could be - emitted. - -- **Missing semicolon warning in `if`** (`ndc_analyser/src/analyser.rs` ~line 174) - When the `true`-branch of an `if` produces a non-unit value but no `else` is present, a warning - for the missing semicolon would be helpful. - -- **`never` type for variable declarations** (`ndc_analyser/src/analyser.rs` ~line 66) - `let x = …` currently resolves to `unit`. Introducing a `never` / `!` type might be more - accurate, once the type lattice is fleshed out. - ---- - -## Medium: Semantic analysis correctness - -- **Conflicting binding on re-declaration** (`ndc_analyser/src/analyser.rs` ~line 141) - When a function name is declared a second time in the same scope, the analyser silently creates - a new binding instead of either updating the old one or raising an error. The right policy needs - to be decided and implemented. - -- **`debug_assert` → `unreachable!` in `find_function_candidates`** (`ndc_analyser/src/scope.rs` ~line 99) - A variadic function match should be impossible at this call-site. The `debug_assert!(false, …)` - should be replaced with `unreachable!` once we are confident the invariant holds. - ---- - -## Medium: Number / arithmetic semantics - -- **Bitwise NOT on non-integer numbers** (`ndc_core/src/num.rs` ~line 181) - Currently `!float` and `!rational` return `NaN` (matching Noulith behaviour). Decide whether this - is intentional for this language or whether it should be an error. - -- **`bigint → int` rounding in floor/ceil/round** (`ndc_core/src/num.rs` ~line 584) - After rounding a `Rational`, the result is converted to `BigInt` rather than trying to fit it - back into a machine `i64`. Should attempt the smaller representation first. - -- **Division performance** (`ndc_core/src/num.rs` ~line 323) - `Div` always promotes both operands to `Rational`. In the common `Int / Int` case this is - unnecessary. A fast path for integer operands would avoid the allocation. - ---- - -## Small: Lexer improvements - -- **Unicode escape sequences** (`ndc_lexer/src/string.rs` ~line 72) - String literals do not support `\uXXXX` escape sequences. Add support. - -- **`_` separator after decimal point** (`ndc_lexer/src/number.rs` ~line 130) - `1_000.0` is valid, but `1.0_0` is probably not intended. Consider rejecting `_` after `.`. - -- **Number suffix error interception** (`ndc_lexer/src/number.rs` ~line 48) - The suffix-error checks inside `lex_number` may be redundant since no numeric suffixes are - supported. Consider moving the check to after the lexer returns so it applies uniformly. - -- **`validator_for_radix` performance** (`ndc_lexer/src/number.rs` ~line 231) - The string-slice approach for validating digits by radix is O(radix) per character. A lookup - table or `char::to_digit` would be faster. - -- **`consume()` internal error handling** (`ndc_lexer/src/lib.rs` ~line 202) - `consume()` panics with `expect` on underflow. Document the invariant or add a proper internal - error type. - ---- - -## Small: Parser error messages - -- **Better error for boolean-returning `if` without semicolon** (`ndc_parser/src/parser.rs` ~line 738) - The pattern `if x == y { true } else { false }` triggers a generic parse error. A targeted - diagnostic would be more helpful. - -- **"Expected expression" error quality** (`ndc_parser/src/parser.rs` ~line 1001) - The fallback "Expected an expression but got '…'" message may not always accurately describe - the failure. Audit and improve. - ---- - -## Small: Test / debug - -- **Error rendering in block-scope test** (`tests/programs/004_basic/005_block_scope_destroys_local_variables.ndc` line 6) - The error is reported correctly but rendered weirdly in the test output. Investigate why and fix - the display. diff --git a/docs/analyser-readability-review.md b/docs/analyser-readability-review.md deleted file mode 100644 index 3557e4f0..00000000 --- a/docs/analyser-readability-review.md +++ /dev/null @@ -1,136 +0,0 @@ -# Analyser readability review (`ndc_analyser`) - -> Point-in-time review captured alongside the LSP work (PR #164). File/line -> references are a snapshot and will drift as the code changes; treat them as -> starting points, not exact coordinates. - -## Context - -The semantic analyser is the hardest part of the project to hold in one's head. -This review answers two questions, with the upcoming DefId/LSP-resolution work in -mind: (1) are there any *big* issues that only a major refactor can fix, and (2) -what *incremental* readability improvements are worth making? It proposes no -behaviour changes — it is an assessment plus a backlog. Sizes at time of writing: -`analyser.rs` ~930 lines, `scope.rs` ~1432 (~1038 production + ~394 tests), -`lib.rs` 5. - -## Headline verdict - -**No mandatory major refactor.** The architecture is sound for the language's feature -set: `Analyser` is a thin client over `ScopeTree`; types flow through side tables -(`expr_types`, `inferred_return_types`) plus in-place AST annotation (`resolved`, -`captures`, `inferred_type`); the compiler consumes the annotated AST. Nothing is -boxed-in. The difficulty is **accumulated local complexity**, which is entirely -addressable incrementally. - ---- - -## Part 1 — Big issues - -### The one architectural theme: `ScopeTree` conflates resolution + slot allocation - -`ScopeTree` (`scope.rs`) does two jobs at once: **lexical name resolution** (which -declaration a name means) and **VM stack-slot allocation** (the concrete `usize` -`ResolvedVar::{Local,Upvalue,Global}{slot}`). This entanglement is the root of both -the intricacy (the `base_offset` / `function_scope_idx` / upvalue-hoisting math -threaded through every lookup) and the "slot isn't a stable identity" limitation that -bit the LSP. - -**Feasibility (investigated):** -- The compiler is already **semi-independent** of analyser slots: it keeps its own - `num_locals` and only `max`es it against declared slots (`compiler.rs:513,758,792,811`), - and allocates its own temporaries (`compiler.rs:280-282,483-484`). So *local* slot - numbering could plausibly move to the compiler. -- **Globals** are purely positional in the `FunctionRegistry` iteration order - (`interpreter/src/lib.rs:221-229`, `scope.rs:370-384`, `vm.rs:204`) — moving their - assignment needs a stable name→slot map handed compiler-side. Contained, not hard. -- **Upvalues/captures are the hard, tightly-coupled part** and *not cleanly - separable*: the analyser computes `CaptureSource::{Local,Upvalue}(index)` - (`scope.rs:474-480,988-1025`), the compiler embeds it verbatim into `OpCode::Closure` - (`compiler.rs:736-742`), and the VM indexes `upvalues[slot]` directly (`vm.rs:491-532`). - Critically, **discovering captures *is* a name-resolution activity** (you must - resolve names across function boundaries to know what escapes), and the index *is* - the layout — so "separating resolution from layout" buys little here. -- Other consumers: REPL resume leans on `Compiler::num_locals` (`interpreter/src/lib.rs` - ~251/257/276); the LSP reads `ResolvedVar` but not slot numbers. Coupling surface is - small (~6 files, ~70 lines). - -**Verdict: feasible but low-ROI — recommend shelving.** It would mostly relocate -local-slot bookkeeping; it would *not* simplify the genuinely hard code (upvalue -hoisting, overload resolution — both intrinsic). It also touches the runtime hot path -(closure creation, REPL resume) for moderate risk. The planned **DefId side-table** -gives the LSP the stable identity it needs *without* this refactor, and would be the -natural seam if this is ever revisited. **Do DefId first; reconsider this only if a -concrete need appears.** - -### Not-big, but worth knowing -- `resolve_call` / `scalar_walk` (the 5-case overload + tuple-broadcast cascade, - `scope.rs:531-715`) is the most complex algorithm, but the complexity is *intrinsic* - (overloading × vectorization × closures). It is well-documented; it can be made more - readable (Part 2) but not fundamentally simpler without dropping features. - ---- - -## Part 2 — Incremental readability backlog (prioritized) - -All behaviour-preserving. Ordered by (value ÷ risk). Each is independently shippable. - -### Batch 1 — High value, near-zero risk (pure moves/renames/docs) -1. **Extract big `analyse_inner` arms into methods.** `analyse_inner` is ~367 lines - (`analyser.rs:104-470`). Move `FunctionDeclaration` (`282-371`, ~90 lines) → - `analyse_function_declaration`, `OpAssignment` (`194-281`, ~88 lines) → - `analyse_op_assignment`, `Assignment` (`169-193`) → `analyse_assignment`. Leaves the - dispatcher a scannable table of one-liners. **Biggest single win.** -2. **Fix `span` shadowing** in `resolve_lvalue_declarative` (`analyser.rs:721-757`): the - `Lvalue::Identifier { span, .. }` destructure shadows the method's `span` param — - rename one. Genuine footgun. -3. **Module-level orientation docs.** Add a short "how analysis works" header to - `analyser.rs` and `scope.rs` (two-phase function pre-registration; slot numbering & - `base_offset`; upvalue hoisting; the 5-case resolution). Document the `base_offset` / - `function_scope_idx` / `env_scopes` invariants once at their definitions - (`scope.rs:128-135`). Highest orientation ROI. -4. **Fix the `TOOD` typo** (`analyser.rs:538`) and capture the "get this from the AST - when the parser adds it" note as a real TODO.md entry / issue. - -### Batch 2 — Dedupe tricky logic (low risk, removes copy-paste) -5. **Unify "widen binding or error".** The same widen-then-check-annotation block - appears 3×: `analyser.rs:178-189` (Assignment), `244-255` (OpAssignment ident), - `265-272` (OpAssignment index). Extract one helper - `widen_binding(target, widened, value_type, span)`. -6. **Extract the upvalue-chain follower** in `scope.rs`. The `CaptureSource::Local | - Upvalue` walk is duplicated in `get_type` (`387-413`) and `get_binding_mut` - (`898-932`), and echoed in `hoist_upvalue` (`988-1025`). A `follow_upvalue_chain` - helper removes the worst `scope.rs` duplication. -7. **Naming pass.** `sig`/`type_sig` → consistent `arg_types`; `loose` → - `compatible_candidates`; `scope_ptr` → `scope_idx`; `env_scopes` → - `crossed_fn_boundaries` (+ doc). Cheap, high comprehension value. - -### Batch 3 — Structural tidy (small risk, needs tests first) -8. **Collapse dual error storage.** `Analyser.errors` (`analyser.rs:37`) duplicates - `AnalysisResult.errors`, reconciled in `take_result` (`60-64`). Emit straight into - `result.errors` and drop the field (check `emit`/`emit_external`/`has_errors`). -9. **Split `scalar_walk`** (`scope.rs:639-715`): factor the per-scope body - (find-exact / collect-loose / collect-all-by-name) into a `scan_scope` helper used by - both the loop and the global-scope fallback, removing the duplicated fallback block - (`647-651` vs `683-688`). -10. **Extract `resolve_lvalue_declarative`'s Sequence arm** (`analyser.rs:~762-819`) - into `resolve_sequence_lvalue`; it's long, nested, and has a shadow of `found_type`. -11. *(Optional)* **Error-constructor boilerplate** (`analyser.rs:~853-929`): 11 - `Self { text: format!(...), span }` constructors — a tiny macro or `new(span, msg)` - helper trims repetition. Low priority (currently readable). - -### Supporting: characterization tests (do before Batch 3) -`scope.rs` tests (~`1039-1432`, 20 cases) cover scope/upvalue mechanics well but **omit -`resolve_call`'s vec/dynamic paths** (`resolve_vec`, `VecResolution`, `Binding::Dynamic`, -`dynamic_return_type`, `extend_dedup`). Add characterization tests for the 5 resolution -cases first — they document behaviour *and* de-risk items 8–10. - ---- - -## Recommended sequence & verification - -If/when executed: Batch 1 → Batch 2 → (add resolution tests) → Batch 3, one small PR -per item, each gated on `cargo test` (workspace), `cargo clippy` clean, `cargo fmt`. The -functional suite (`tests/functional`) plus the `scope.rs` unit tests are the safety net; -the new characterization tests harden the riskiest area before it's touched. No item -changes runtime behaviour, so a green suite is sufficient verification. diff --git a/docs/design/vectorization.md b/docs/design/vectorization.md deleted file mode 100644 index ec6002a0..00000000 --- a/docs/design/vectorization.md +++ /dev/null @@ -1,104 +0,0 @@ -# Vectorization - -Operator syntax broadcasts element-wise over tuples. `a + b` where both -arguments are `Tuple` resolves to two `+(Int, Int)` calls and a -tuple build. The mechanism is gated to operator syntax so regular function -calls never accidentally broadcast. - -## Background - -PR [#140] widened `Binding::Dynamic` return types to `Any` to fix issue -[#139]: the analyser had been LUB-ing declared overload returns, but the -value-level dispatcher could fall through to vec dispatch and produce a -value no declared overload returned. The widening pessimised every dynamic -caller — including ones with no vec path at all. The current design -restores that precision by tracking vec-ness on the binding rather than -on a separate fallback path, and broadens vec to cover n-ary operators -and non-numeric overloads. - -[#139]: https://github.com/timfennis/andy-cpp/issues/139 -[#140]: https://github.com/timfennis/andy-cpp/pull/140 - -## Three pieces - -### 1. `Expression::OperatorCall` distinguishes operator desugars - -The parser emits `Expression::OperatorCall { function, arguments }` for -`a + b`, `-x`, `op=`, and `not x` — same shape as `Call` but a distinct -variant. Downstream layers pattern-match exhaustively: the analyser opts -into vec dispatch on `OperatorCall` only, while `Call` keeps regular -semantics. No flag, no curated list of operator names anywhere outside -the parser. - -### 2. `Candidate` distinguishes scalar from vec overloads - -```rust -pub enum Candidate { - Scalar(ResolvedVar), - /// Element-wise tuple broadcast over the scalar that `var()` returns. - Vec(ResolvedVar), -} -``` - -`Binding::{Resolved,Dynamic}` carry `Candidate`/`Vec`. The -analyser pins `Resolved(Candidate::Vec(scalar))` when per-position -resolution unanimously picks one scalar; it carries a mixed list as -`Dynamic` when types aren't precise enough. - -### 3. Per-position vec resolution - -For an operator-form call `op(a₁, …, aₙ)` where at least one `aᵢ` is -statically a non-empty tuple of length `k`, the analyser: - -1. Builds a per-position signature for each `i ∈ 0..k`: tuple args - contribute `arg[i]`, scalar args broadcast unchanged. -2. Looks up scalar overloads for each position signature. -3. **All positions pick the same scalar**: emit - `Binding::Resolved(Candidate::Vec(scalar))`, result type - `Tuple`. -4. **Mixed positions**: emit `Binding::Dynamic(merged_candidates)`, - result type = per-position LUB wrapped as `Tuple<…>`. -5. **Any position has zero candidates**: emit `Binding::None`. The call - can't succeed at runtime either, so we error at compile time with - `function_not_found`. - -## Runtime dispatch - -Two opcodes carry vec work: - -* `CallVec(args)` — the compiler emits this for `Resolved(Vec)`. The - scalar is loaded directly (no `OverloadSet` wrapper); the VM reads the - broadcast axis from the tuple args at runtime and calls the known - scalar `axis_len` times. This is the fast path that recovers the perf - the per-element re-probe would cost. - -* `Call(args)` with an `OverloadSet` callee — used for `Dynamic`. The - dispatcher walks candidates in priority order: scalars first - (first-match-wins), then vec candidates produce a `Callable::Vec` - carrying the list of scalars that the broadcast loop narrows per - element pair. The pinned-single-scalar case (one vec candidate) skips - the per-element probe via the same fast path `CallVec` uses. - -Element-call failures surface with `while vectorising '' at index N` -prefixed to the inner message, so the outer call and failing position -appear in the error. - -## What changed vs the old design - -| Old | New | -|---|---| -| Binary numeric vec only | n-ary, any scalar overload | -| `Binding::Dynamic` widened all returns to `Any` | LUB-d for pure scalar; precise `Tuple<…>` for vec | -| Runtime `try_vectorized_call` post-check | First-class candidate in `OverloadSet` + `CallVec` opcode | -| Mixed-element tuples crashed mid-iteration | Compile-time `function_not_found` | -| Unary `-(1, 2, 3)` errored | Broadcasts to `(-1, -2, -3)` | - -## Notes - -* **Per-position LUB collapse**: `(Int, Float) + (Float, Int)` infers - `Tuple` rather than the per-element-precise - `Tuple`. The simpler uniform return type keeps the - candidate list small; the cost is rare in practice. -* **Empty tuples** decline vec resolution — they have no broadcast axis. -* **Indexing** (`a[i]`) parses as `Call`, not `OperatorCall`: there's no - natural broadcast story for `(list_a, list_b)[i]`. diff --git a/ext/tree-sitter-andy-cpp/README.md b/ext/tree-sitter-andy-cpp/README.md deleted file mode 100644 index 47a0b3a0..00000000 --- a/ext/tree-sitter-andy-cpp/README.md +++ /dev/null @@ -1,187 +0,0 @@ -# tree-sitter-andy-cpp - -A [tree-sitter](https://tree-sitter.github.io/tree-sitter/) grammar for the -**Andy C++** (`.ndc`) language. - -Tree-sitter powers incremental, error-tolerant syntax trees used by editors such -as **Neovim**, **Helix**, **Zed**, and **Emacs** for highlighting, structural -selection, folding, and code navigation. (VS Code does not use tree-sitter for -highlighting — it uses the TextMate grammar in `../andy-cpp/syntaxes/`.) - -The grammar mirrors the precedence ladder and constructs implemented in -`ndc_lexer` / `ndc_parser`. It is validated against the interpreter's full -functional-test corpus: every valid `.ndc` program under -`tests/functional/programs/` parses without errors. - -## Layout - -``` -grammar.js # the grammar definition -tree-sitter.json # package metadata (generated/maintained by the CLI) -queries/ - highlights.scm # syntax highlighting - locals.scm # scopes & definitions (variables, params, functions) - injections.scm # `#!` shebang line highlighted as bash -test/corpus/ # tree-sitter test cases -src/ # generated parser (run `tree-sitter generate`) - scanner.c # external scanner: named op-assign + raw strings -``` - -## Developing - -Requires Node.js. The tree-sitter CLI is a dev dependency. - -```bash -cd ext/tree-sitter-andy-cpp -npm install # installs tree-sitter-cli -npx tree-sitter generate # regenerate src/parser.c from grammar.js -npx tree-sitter test # run test/corpus -npx tree-sitter parse path.ndc # dump the parse tree for a file -``` - -Re-run `generate` after every edit to `grammar.js`. Commit the regenerated -`src/` so consumers can build without the CLI. - -### Re-validating against the interpreter corpus - -```bash -cd ext/tree-sitter-andy-cpp -fail=0 -for f in $(find ../../tests/functional/programs -name '*.ndc'); do - npx tree-sitter parse -q "$f" >/dev/null 2>&1 || { echo "ERR $f"; fail=1; } -done -[ $fail -eq 0 ] && echo "all valid programs parse" -``` - -The only files that report errors are the interpreter's deliberate -`// expect-error:` cases (malformed input) — that is the expected outcome. - -## Editor integration - -The parser's language name is **`andy_cpp`** (the symbol exported by the -generated parser is `tree_sitter_andy_cpp`). - -The instructions below drive each editor's **built-in** tree-sitter runtime, so -they don't depend on a plugin manager or a specific nvim-treesitter version. -Building the parser needs Node.js and a C compiler. - -### Optional: a language server - -The interpreter ships an LSP server, started with `ndc lsp` over stdio. It -provides hover (inferred types), completion, go-to-definition, document symbols -and inlay hints. Install the `ndc` binary so it's on your `PATH`: - -```bash -cargo install --git https://github.com/timfennis/andy-cpp -``` - -The editor sections below wire this up alongside highlighting. - -### Neovim - -Neovim has a built-in tree-sitter runtime, so nvim-treesitter is not required to -load this grammar. - -1. Build the parser and install it with the queries where Neovim's runtimepath - can find them (the output file must be named `andy_cpp.so`): - - ```bash - cd ext/tree-sitter-andy-cpp - npm install - mkdir -p ~/.config/nvim/parser ~/.config/nvim/queries/andy_cpp - npx tree-sitter build -o ~/.config/nvim/parser/andy_cpp.so - cp queries/*.scm ~/.config/nvim/queries/andy_cpp/ - ``` - -2. Add to your config (`init.lua`): - - ```lua - -- Treat .ndc files as the `andy_cpp` filetype. - vim.filetype.add({ extension = { ndc = "andy_cpp" } }) - - -- Start tree-sitter highlighting for those buffers. - vim.api.nvim_create_autocmd("FileType", { - pattern = "andy_cpp", - callback = function(args) - pcall(vim.treesitter.start, args.buf, "andy_cpp") - end, - }) - - -- Language server (Neovim 0.11+). Skip if you didn't install `ndc`. - vim.lsp.config("ndc_lsp", { - cmd = { "ndc", "lsp" }, - filetypes = { "andy_cpp" }, - root_markers = { ".git" }, -- falls back to the file's directory - }) - vim.lsp.enable("ndc_lsp") - - -- Optional: show inlay hints once the server attaches. - vim.api.nvim_create_autocmd("LspAttach", { - callback = function(args) - local client = vim.lsp.get_client_by_id(args.data.client_id) - if client and client.name == "ndc_lsp" then - pcall(vim.lsp.inlay_hint.enable, true, { bufnr = args.buf }) - end - end, - }) - ``` - -Rebuild (step 1) after each `tree-sitter generate`, re-copy the queries after -editing them, then restart Neovim. After rebuilding the `ndc` binary, reload the -server with `:LspRestart`. - -> **Already map `.ndc` to a different filetype?** (for example, via an existing -> `ftdetect` rule.) Keep that filetype, drop the `vim.filetype.add` call, and -> point the parser at it with -> `vim.treesitter.language.register("andy_cpp", "")`. Use -> `` as the autocmd `pattern` and in the LSP `filetypes` list. - -### Helix - -Helix has built-in tree-sitter and LSP support. Add to -`~/.config/helix/languages.toml`: - -```toml -[[language]] -name = "andy-cpp" -scope = "source.andy-cpp" -file-types = ["ndc"] -comment-tokens = ["//"] -indent = { tab-width = 4, unit = " " } -language-servers = ["ndc-lsp"] # omit if you didn't install `ndc` - -[language-server.ndc-lsp] -command = "ndc" -args = ["lsp"] - -[[grammar]] -name = "andy-cpp" -# A git source must include `rev`. If you have the repo checked out, a local -# path is simpler: source = { path = "/abs/path/to/ext/tree-sitter-andy-cpp" } -source = { git = "https://github.com/timfennis/andy-cpp", rev = "master", subpath = "ext/tree-sitter-andy-cpp" } -``` - -Build the grammar into Helix's runtime and install the queries. Rather than -`hx --grammar build` (which rebuilds every grammar and needs the output -directory to already exist), build just this one with the tree-sitter CLI — the -output file must be named `andy-cpp.so` to match the grammar name: - -```bash -cd ext/tree-sitter-andy-cpp -npm install -mkdir -p ~/.config/helix/runtime/grammars ~/.config/helix/runtime/queries/andy-cpp -npx tree-sitter build -o ~/.config/helix/runtime/grammars/andy-cpp.so -cp queries/*.scm ~/.config/helix/runtime/queries/andy-cpp/ -``` - -Check it with `hx --health andy-cpp` (Tree-sitter parser, Highlight queries, and -the `ndc-lsp` server should all be ✓). - -> The binary is `hx` in most installs but may be `helix` (e.g. some distro -> packages); use whichever your install provides. - -## Known limitations - -- **Doubly-nested generics** in type annotations (`List>`) can - mis-tokenize the closing `>>`. Single-level generics (`Map`, - `Option`) are fine. diff --git a/ext/tree-sitter-andy-cpp/install.sh b/ext/tree-sitter-andy-cpp/install.sh new file mode 100755 index 00000000..8c6e40cf --- /dev/null +++ b/ext/tree-sitter-andy-cpp/install.sh @@ -0,0 +1,78 @@ +#!/bin/sh + +set -eu + +usage() { + echo "Usage: $0 " >&2 +} + +editor=${1:-} +case "$editor" in + neovim|nvim) + editor=neovim + ;; + helix) + ;; + -h|--help) + usage + exit 0 + ;; + *) + usage + exit 2 + ;; +esac + +source_dir=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd) +config_home=${XDG_CONFIG_HOME:-${HOME:?HOME must be set when XDG_CONFIG_HOME is unset}/.config} +compiler=${CC:-cc} + +if ! command -v "$compiler" >/dev/null 2>&1; then + echo "C compiler not found: $compiler" >&2 + echo "Install a C compiler or set CC to its executable path." >&2 + exit 1 +fi + +case $(uname -s) in + Darwin) + shared_flag=-dynamiclib + ;; + Linux|FreeBSD|OpenBSD|NetBSD) + shared_flag=-shared + ;; + *) + echo "Unsupported operating system: $(uname -s)" >&2 + exit 1 + ;; +esac + +case "$editor" in + neovim) + parser_dir=$config_home/nvim/parser + query_dir=$config_home/nvim/queries/andy_cpp + parser_name=andy_cpp.so + ;; + helix) + parser_dir=$config_home/helix/runtime/grammars + query_dir=$config_home/helix/runtime/queries/andy-cpp + parser_name=andy-cpp.so + ;; +esac + +build_dir=$(mktemp -d "${TMPDIR:-/tmp}/tree-sitter-andy-cpp.XXXXXX") +cleanup() { + rm -rf "$build_dir" +} +trap cleanup 0 1 2 15 + +"$compiler" -O2 -fPIC "$shared_flag" -I"$source_dir/src" \ + "$source_dir/src/parser.c" "$source_dir/src/scanner.c" \ + -o "$build_dir/$parser_name" + +mkdir -p "$parser_dir" "$query_dir" +cp "$build_dir/$parser_name" "$parser_dir/$parser_name" +cp "$source_dir"/queries/*.scm "$query_dir/" + +echo "Installed the Andy C++ Tree-sitter parser for $editor:" +echo " parser: $parser_dir/$parser_name" +echo " queries: $query_dir" diff --git a/manual/src/tooling/editor-support.md b/manual/src/tooling/editor-support.md index fd2eff6a..6b8ec8d1 100644 --- a/manual/src/tooling/editor-support.md +++ b/manual/src/tooling/editor-support.md @@ -1,15 +1,16 @@ # Editor support Andy C++ ships a language server (LSP) so editors can offer rich feedback as you -write `.ndc` files. The server is built into the `ndc` binary and is started with: +write `.ndc` files. If you have Andy C++ isntalled you automatically also have the language server. + +You can start the language server like this: ```bash ndc lsp --stdio ``` -Most users don't run this by hand — the [VS Code extension](https://open-vsx.org/) -launches it automatically. Any LSP-capable editor can use it by pointing at the -`ndc lsp --stdio` command for the `andy-cpp` language and the `.ndc` file extension. +Most users do not run this command by hand. Editor integrations either launch it +automatically or are configured to run it for `.ndc` files. ## What the language server provides @@ -27,6 +28,19 @@ launches it automatically. Any LSP-capable editor can use it by pointing at the variable declarations in the file. - **Go-to-definition** — jump from a variable or function usage to its declaration. +## VS Code and compatible editors + +The [Andy C++ extension on Open VSX](https://open-vsx.org/extension/TimFennis/andy-cpp) +provides syntax highlighting, all of the language-server features listed above, and +a **Run Script** command that executes the current file in the integrated terminal. + +Install it from the Extensions view in editors that use the Open VSX registry. For +Microsoft VS Code, download the VSIX file from the Open VSX page and install it with +**Extensions: Install from VSIX...** in the command palette. + +The extension launches `ndc lsp` automatically. If `ndc` is not on the `PATH` seen +by the editor, set `andy-cpp.ndcPath` to the full path of the binary. + ## JetBrains IDEs (RustRover, IntelliJ, …) JetBrains IDEs are supported without a dedicated plugin, in two independent parts. @@ -53,8 +67,116 @@ plugin connects the IDE to the language server: above. `ext/lsp4ij-ndc/template.json` in the repository contains the same configuration as a reference. +## Neovim + +Neovim has a built-in Tree-sitter runtime, so `nvim-treesitter` is not required. +The generated parser is committed to the Andy C++ repository and can be compiled +with a C compiler; installing it does not require Node.js or npm. + +Clone the repository and run the installer: + +```bash +git clone --depth 1 https://github.com/timfennis/andy-cpp.git +cd andy-cpp/ext/tree-sitter-andy-cpp +./install.sh neovim +``` + +The script supports Linux, the BSDs, and macOS. It installs the parser and queries +under `${XDG_CONFIG_HOME:-$HOME/.config}/nvim`. Set `CC` to select a different C +compiler. + +Add the following to `init.lua`: + +```lua +-- Treat .ndc files as the `andy_cpp` filetype. +vim.filetype.add({ extension = { ndc = "andy_cpp" } }) + +-- Start Tree-sitter highlighting for those buffers. +vim.api.nvim_create_autocmd("FileType", { + pattern = "andy_cpp", + callback = function(args) + pcall(vim.treesitter.start, args.buf, "andy_cpp") + end, +}) + +-- Language server (Neovim 0.11+). +vim.lsp.config("ndc_lsp", { + cmd = { "ndc", "lsp", "--stdio" }, + filetypes = { "andy_cpp" }, + root_markers = { ".git" }, +}) +vim.lsp.enable("ndc_lsp") + +-- Optional: show inlay hints once the server attaches. +vim.api.nvim_create_autocmd("LspAttach", { + callback = function(args) + local client = vim.lsp.get_client_by_id(args.data.client_id) + if client and client.name == "ndc_lsp" then + pcall(vim.lsp.inlay_hint.enable, true, { bufnr = args.buf }) + end + end, +}) +``` + +Run `./install.sh neovim` again after updating the grammar or its queries, then +restart Neovim. After rebuilding `ndc`, reload the language server with +`:LspRestart`. + +If `.ndc` is already mapped to a different filetype, omit `vim.filetype.add` and +register the parser for that filetype instead: + +```lua +vim.treesitter.language.register("andy_cpp", "your_filetype") +``` + +Use the same filetype in the autocmd pattern and language-server configuration. + +## Helix + +Helix also has built-in Tree-sitter and LSP support. Add the following to +`~/.config/helix/languages.toml` (or the equivalent path below +`XDG_CONFIG_HOME`): + +```toml +[[language]] +name = "andy-cpp" +scope = "source.andy-cpp" +file-types = ["ndc"] +comment-tokens = ["//"] +indent = { tab-width = 4, unit = " " } +language-servers = ["ndc-lsp"] + +[language-server.ndc-lsp] +command = "ndc" +args = ["lsp", "--stdio"] + +[[grammar]] +name = "andy-cpp" +source = { git = "https://github.com/timfennis/andy-cpp", rev = "master", subpath = "ext/tree-sitter-andy-cpp" } +``` + +From a checkout of the Andy C++ repository, install the parser and queries without +Node.js or npm: + +```bash +cd ext/tree-sitter-andy-cpp +./install.sh helix +``` + +The files are installed under +`${XDG_CONFIG_HOME:-$HOME/.config}/helix/runtime`. Check the setup with +`hx --health andy-cpp` (or `helix --health andy-cpp` on systems where the binary +uses that name). + +## Other editors + +Any editor with an LSP client can use the Andy C++ language server. Configure it +to run `ndc lsp --stdio` for `.ndc` files with the language id `andy-cpp`. + ## Notes - The server uses full-document synchronisation and re-analyses on each edit. - While the buffer is mid-edit and doesn't parse, the last successful analysis is retained so hints and dot-completion keep working. +- The Tree-sitter grammar currently has trouble with doubly nested generic type + annotations such as `List>`. Single-level generics work as expected.