Skip to content

Reader recovery: partial trees and every error (ADR 0009 phase 4) - #91

Merged
ademar merged 1 commit into
masterfrom
parser-recovery
Aug 22, 2026
Merged

Reader recovery: partial trees and every error (ADR 0009 phase 4)#91
ademar merged 1 commit into
masterfrom
parser-recovery

Conversation

@ademar

@ademar ademar commented Aug 22, 2026

Copy link
Copy Markdown
Member

What

Phase 4 of ADR 0009 — the last of the runtime-side phases.

Parser.readLocatedExprListRecovering parses as much of a broken input as possible: every form that parses, plus one error per broken region, all with exact spans. The design keeps recovery outside the grammar — broken regions are re-windowed and re-run through the same strict parsers, with a small exact position-remap rule (a window position on line 1 shifts by the base column; every later line already has real columns) — so the strict readers compile depends on are untouched.

Semantics:

  • Recovery resumes at the next line that opens a form at column 1 — the convention that makes top-level forms recovery points.
  • The final broken region is re-parsed with its unclosed brackets closed (string- and comment-aware, [/( tracked as a stack), so the form being typed still yields a tree.
  • Deliberately not done, and recorded in the ADR: mid-file broken regions are not bracket-completed, a broken region's own content is dropped up to the resync point, and error positions stay the strict reader's.

Payoffs, exactly where phase 3 predicted

  • The language server's last-good-parse defines cache is deleted: completion and semantic tokens now run against the actual buffer mid-edit. A buffer like (define twice (vau …))␊)␊(twice still classifies the trailing twice as an operative and publishes both errors.
  • ik check reports every broken region in a file (--json emits them all), instead of stopping at the first.
  • The server publishes every diagnostic per change, not just the first.

The ADR also corrects its own estimate in place: "the largest single piece" landed smaller than the language server, because composing recovery from the strict parsers avoided the rewrite the estimate priced in.

Verification

  • Clean --no-incremental rebuild: zero warnings. Full suite: 446/446 (10 new: 8 recovering-reader tests — clean-input parity with the strict reader, resync, stray closers, per-region errors with exact line assertions, trailing completion, exact span remap, unclosed-string guard, nesting-bomb control — plus multi-region check and a broken-buffer LSP test asserting the full semantic-token encoding and both diagnostics).
  • Real-binary smoke: ik check --json on a two-region file emits two diagnostics at lines 1 and 3, exit 1.
  • All examples run (yingyang skipped); CLR fault sweep: faulted (process aborted): 0.
  • Benchmarks deliberately skipped: the recovering reader is cold code reached only by check and the language server; the strict parse path is byte-identical.

View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.

Phase 4 of ADR 0009. readLocatedExprListRecovering re-windows broken
input and re-runs the same strict parsers, remapping positions exactly
(a window position on line 1 shifts by the base column; later lines
already have real columns). Recovery is composition around the strict
readers, not error-recovery alternatives woven through the grammar --
the strict path compile depends on is untouched.

Semantics: one error per broken region, worded by the strict parser;
parsing resumes at the next line that opens a form at column 1; and the
final broken region is re-parsed with its unclosed brackets closed
(respecting strings and comments), so the form being typed still yields
a tree. Deliberately not done: mid-file broken regions are not
bracket-completed, and error positions stay the strict reader's.

The payoffs land where phase 3 said they would: the language server's
last-good-parse defines cache is deleted, completion and semantic
tokens run against the actual buffer mid-edit, ik check reports every
broken region in a file, and the server publishes every error instead
of the first. The ADR corrects "the largest single piece" in place: it
landed smaller than the language server, because composing recovery
from the strict parsers avoided the rewrite the estimate priced in.
@ademar
ademar merged commit a68a8a7 into master Aug 22, 2026
3 checks passed
@ademar
ademar deleted the parser-recovery branch August 22, 2026 18:40
@ademar ademar mentioned this pull request Aug 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant