Line-aligned emission: source-true backtraces and debuggers by construction - #12
Merged
Conversation
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
3 tasks
Every user statement must be emitted at its original source line so backtraces, breakpoints and debugger display are correct by construction, with no SourceMap or backtrace filtering. Four angles: - emitted text places each user statement at its source line - synthetic (loc-less) injected code never displaces user statements - raw backtrace_locations cite source lines with no filtering - the compiled iseq line table contains user statement source lines (the "can break file:N bind" proxy) All four are red today: Unparser.unparse regenerates layout from scratch. Co-authored-by: Cursor <cursoragent@cursor.com>
Replaces bare Unparser.unparse in Transformer#transform_file_source with LineAlignedEmitter: every loc-carrying statement is emitted at its original source line (padding with blank lines), synthetic loc-less code packs onto the current line, and multi-line normalizations (e.g. modifier-if) compress back to one line when re-parse-verified safe. Backtraces, breakpoints and debugger display become correct by construction — so SourceMap and its registration are deleted. Authoring layer (TransformationHelper): s routes registered custom types (ASTTransform::Node.register) to their classes; s_at anchors fresh nodes to a source location; defer returns a Deferral placement/execution marker pair (token-linked, ControlFlowGuard-validated); run_after is the paved road for sequence-level reordering. DeferralLowering turns marker pairs into hidden-lvar lambdas and calls, reconciling tokens statically (one placement, one-or-more calls) with UnmatchedDeferralError otherwise. The emitter's postcondition rejects unlowered custom types. ast_transform/test_helpers ships assert_line_aligned and assert_backtrace_lines for transform authors' own suites. Stage 0 acceptance tests are green; full suite 70 tests, 0 failures. Co-authored-by: Cursor <cursoragent@cursor.com>
transform(source) now emits line-aligned output like transform_file_source, so in-memory callers (and rspock's transformation tests) observe the exact text the loader would compile. Expectations in TransformationTest update from Unparser's normalized re-indentation to source-anchored layout: statements keep their source lines, consumed transform! annotations leave blank lines, and nested bodies are emitted flush-left (indentation is not semantic; lines are). Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Container openers (e.g. a test("name #{row} line #{line}") block header)
can contain dstr interpolations referencing scope locals; without the
local-variable context Unparser's round-trip verification fails. Route
container_delimiters through the same scoped unparse as statements.
Co-authored-by: Cursor <cursoragent@cursor.com>
README gains the authoring contract section (loc'd nodes emit at their source line; synthetic nodes pack; textual order is source order), the TransformationHelper toolkit (s/s_at/defer/run_after), custom IR node registration, and the test helpers. CHANGELOG for 3.0.0. Co-authored-by: Cursor <cursoragent@cursor.com>
Ruby 3.2 is EOL (March 2026) — required_ruby_version and the CI matrix move to 3.3+. unparser floor raised to >= 0.8: the emitter passes static_local_variables: (an 0.7 interface — 0.6 would ArgumentError) and relies on 0.8's prism-based round-trip verification for Ruby >= 3.4 syntax. parser floor raised to >= 3.3 to match unparser's own floor (the declared >= 3.0 could never actually resolve lower). Lockfile now on unparser 0.9.0. Co-authored-by: Cursor <cursoragent@cursor.com>
…ches Codecov flagged the patch gaps. Real fixtures now exercise rescue headers (exception list + capture), else, ensure, and standalone begin/end containers at their source lines; assert_line_aligned's failure path is pinned by a statement-swapping transform (the contract violation it exists to catch — note AST::Node#updated compares children by ==, which ignores locations, so a loc-only rewrite is invisible); DeferralToken#inspect and compress_to_single_line's parse-failure fallback (a totality guard with no natural trigger while Unparser normalizes heredocs to inline strings) get direct tests. Two branches were genuinely unreachable and are deleted rather than covered: statements_of only ever receives nodes or nil, and pack never sees a blank last line because place overwrites padded lines immediately. Co-authored-by: Cursor <cursoragent@cursor.com>
dev.yml pins Ruby 4.0.6 (latest; converges with the org toolchain) and declares up/test; dev up provisions the per-machine .shadowenv.d (now gitignored) via rbenv, so the right Ruby activates without manual PATH surgery. .ruby-version gives plain rbenv users the same pin. Inert for external contributors — plain Bundler still works, README says so. Co-authored-by: Cursor <cursoragent@cursor.com>
Parser::AST::Node#updated returns self when children compare == — and AST::Node#== ignores source locations, so a Processor pass replacing a node with an equal-valued one (different loc) silently no-ops through every updated() up the tree. Surfaced while writing the misalignment test for assert_line_aligned. Co-authored-by: Cursor <cursoragent@cursor.com>
dev now single-sources the project Ruby from the dependencies.rb manifest; dev.yml's ruby: key is removed. Toolchain-only manifest — gems stay bundler-managed via the hand-written gemspec/Gemfile. Co-authored-by: Cursor <cursoragent@cursor.com>
Deferral now has near-transparent semantics instead of transform-time policing: - The hidden closure is a non-lambda proc, so a deferred return still returns from the enclosing method (placement and execution always share one method activation). - Locals assigned by deferred statements are pre-declared (x = x) before the proc, keeping them method-scope so statements after the execution point can read them. - ControlFlowGuard and NonDeferrableError are gone. Jumps severed from their owner keep Ruby's native behavior (break/retry fail loudly, next/redo silently alter flow) — what a surface allows users to defer is the transform author's call, documented on defer. Also: the emitter recurses into :itblock containers (parity with :block/:numblock), flattens loc-less :begin statements (the lowered placement carries its pre-declarations in one), and CI adds Ruby 3.4. Co-authored-by: Cursor <cursoragent@cursor.com>
The two free-floating markers let authors build programs that only failed two stages later (run_after's pure-sink arrangement pinned behavior the lowering rejected). Thunk makes those states unrepresentable: - Thunk < ASTTransform::Node, children [token, *body], invariants enforced in initialize — every construction path shares it, including Processor rebuilds (MalformedThunkError). Built via thunk(*statements); the token is internal. - One node, placed where the body must EXECUTE; ThunkLowering derives the proc's textual placement from the body's source locations and inserts it into the enclosing statement sequence by line order. Placements never cross a def boundary (the hidden lvar must share the call's activation) but do escape block literals (closures). - Multiplexing is reusing the node: one proc, N calls. Diverging bodies on one token and bodies whose lines fall after the execution point raise ThunkPlacementError; the three UnmatchedDeferralError modes (orphan/premature/duplicate) no longer exist. - run_after keeps its surface, now removing the run and inserting a thunk; its impossible pure-sink support is gone. - AbstractTransformation#on_ast_thunk descends thunk bodies by default, so later passes process wrapped statements instead of skipping them via handler_missing. - Hidden lvars renamed __ast_deferred_N__ -> __ast_thunk_N__. Co-authored-by: Cursor <cursoragent@cursor.com>
Line-aligned emission previously emitted everything flush-left, which kept line fidelity but made emitted artifacts and test expectations hard to read against the source. Statements opening a fresh line now carry their original loc column as leading whitespace — cosmetic only, packed statements and Unparser continuation lines are unaffected. Co-authored-by: Cursor <cursoragent@cursor.com>
Style the files this branch adds or rewrites, bump rubocop-shopify to ~> 3.0 now that the Ruby floor is 3.3 (dropping the parallel pin), and move development dependencies to the Gemfile per Gemspec/DevelopmentDependencies, which 3.0 enables. Co-authored-by: Cursor <cursoragent@cursor.com>
The narrative comments were wrapped at ~80 columns, wasting vertical space now that the style guide's line length is 120. Code examples and diagrams keep their layout. Co-authored-by: Cursor <cursoragent@cursor.com>
JPDuchesne
force-pushed
the
jpd/line-aligned-emission
branch
from
July 24, 2026 04:17
8567998 to
e587836
Compare
…less collaborators LineAlignedEmitter.new(ast, source_path).emit was the narrow-responsibility smell: operation parameters in the constructor, a sole no-param public method, single-use instances. Gathering what changes together uncovers two concepts that were smeared across the emitter's ivars: - Layout: line-addressed output (pad-or-pack cursor mechanics), AST-free. - StatementRenderer: the Unparser adapter, built once per tree with the collected locals to work around statement-in-isolation unparsing. The emitter keeps only the Ruby knowledge (structure walking, line targeting, keyword non-packing) and goes stateless; ThunkLowering gets the same treatment with its token hashes extracted into a private Registry and run renamed to the domain verb lower. Stateless services are constructor-time collaborators (Transformer wires the emitter, the emitter wires the lowering, kwarg defaults as the DI seam); operation-scoped objects (Layout, StatementRenderer, Registry) are created at entry and die with the call. Co-authored-by: Cursor <cursoragent@cursor.com>
initialize already destructures children for invariant checks; capture the pieces there (before super freezes the node) rather than allocating a fresh body array on every accessor call. The body array is frozen so the shared reference cannot be mutated out from under children, and updated re-runs initialize so the capture cannot go stale. Co-authored-by: Cursor <cursoragent@cursor.com>
An agglomerated errors.rb obscures ownership. Each error is defined by the class that raises it, following the Thunk::Id nesting pattern (de-stuttered where the namespace already says thunk): - TransformationHelper::MissingLocationError (raised by s_at) - Thunk::MalformedError (raised by Thunk#initialize) - ThunkLowering::PlacementError (raised during lowering) - LineAlignedEmitter::UnloweredNodeTypeError (the emitter postcondition) Co-authored-by: Cursor <cursoragent@cursor.com>
Constructing a Thunk with wrong arguments is exactly what ArgumentError is for; a custom MalformedError class added ceremony without meaning. Matches run_after, which already raises ArgumentError for bad arguments. Co-authored-by: Cursor <cursoragent@cursor.com>
TestHelpers becomes ASTTransform::Testing::Assertions (ast_transform/testing/assertions), following the active_support/testing shape: the testing namespace marks the consumer-facing test-only surface and leaves room for future framework-specific entries. Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Replaces bare
Unparser.unparsewithLineAlignedEmitter: every statement carrying a source location is emitted at its original source line (padding with blank lines;;-packing synthetic or displaced code). Backtrace and debugger line numbers become the source line numbers by construction, soSourceMapand its registration are deleted.Authoring layer
TransformationHelpergains the transform-author toolkit:s_at(anchor, type, *children)— loc-anchored node construction (MissingLocationErrorfor loc-less anchors).defer(*statements)— returns aDeferralplacement/execution marker pair, token-linked and validated by a control-flow guard (NonDeferrableErrorforreturn/break/… that would re-bind to the deferral lambda).run_after(statements, run:, after:)— the paved road: execution reordering that preserves textual/source order (identity-matched).ASTTransform::Node.register— type-routed custom IR node classes throughs; the emitter's postcondition (UnloweredNodeTypeError) rejects custom types that were not lowered before emission.DeferralLoweringlowers marker pairs into hidden-lvar lambdas + calls, with static reconciliation (UnmatchedDeferralErroron orphan/premature/duplicate markers; multiplexed calls allowed).ast_transform/test_helpers(test-only) shipsassert_line_alignedandassert_backtrace_linesfor transform authors' own suites.Acceptance
The red tests landed first (
test/ast_transform/line_alignment_test.rb) and now pass: emission positions, rawbacktrace_locationsline numbers, and the compiled iseq line table all cite source lines with no filtering. Suite: 75 tests, 0 failures.Breaking (3.0.0)
required_ruby_version >= 3.3, CI matrix now 3.3/3.4/4.0.unparser >= 0.8(the emitter usesstatic_local_variables:and 0.8's prism-based round-trip verification for Ruby >= 3.4 syntax) andparser >= 3.3(unparser's own floor).ASTTransform::SourceMapremoved.Transformer#transform/#transform_file_sourceemit line-aligned output (source-anchored layout, newline-terminated) instead of Unparser's re-normalized formatting.rspockframework/rspock#13 adopts this and deletes its whole mapping apparatus.