From 4d38dd8b4a468933e9f4b7d93b6d1cb1469791bd Mon Sep 17 00:00:00 2001 From: Jean-Philippe Date: Wed, 29 Jul 2026 09:52:00 -0400 Subject: [PATCH 1/4] Add analysis passes and a public parse entry Extract the traversal core (sealed process, process_node hook, thunk descent, TransformationHelper) from AbstractTransformation into AbstractProcessor, and add AbstractAnalysis as the read-only leaf: same engine, but run discards the rebuilt tree and returns the analysis instance. Move parsing into SourceParser and expose ASTTransform.parse / parse_file for analysis-only consumers that never emit. Document the pass taxonomy (structural / positional builders / sibling-annotation) on AbstractTransformation, Transformation, and the README. Co-authored-by: Cursor --- CHANGELOG.md | 6 ++ Gemfile.lock | 2 +- README.md | 50 ++++++++++++++++ lib/ast_transform.rb | 27 +++++++++ lib/ast_transform/abstract_analysis.rb | 38 ++++++++++++ lib/ast_transform/abstract_processor.rb | 39 +++++++++++++ lib/ast_transform/abstract_transformation.rb | 49 ++++++---------- lib/ast_transform/source_parser.rb | 56 ++++++++++++++++++ lib/ast_transform/transformation.rb | 5 ++ lib/ast_transform/transformer.rb | 26 ++------- lib/ast_transform/version.rb | 2 +- test/ast_transform/abstract_analysis_test.rb | 61 ++++++++++++++++++++ test/ast_transform/source_parser_test.rb | 57 ++++++++++++++++++ 13 files changed, 362 insertions(+), 56 deletions(-) create mode 100644 lib/ast_transform/abstract_analysis.rb create mode 100644 lib/ast_transform/abstract_processor.rb create mode 100644 lib/ast_transform/source_parser.rb create mode 100644 test/ast_transform/abstract_analysis_test.rb create mode 100644 test/ast_transform/source_parser_test.rb diff --git a/CHANGELOG.md b/CHANGELOG.md index a8e6ef5..5622fb2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,12 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). +## [3.1.0] - 2026-07-29 +### Added +- Analysis passes: `ASTTransform::AbstractAnalysis`, the read-only counterpart of `AbstractTransformation`. Both leaves share the extracted traversal core `ASTTransform::AbstractProcessor` (sealed `process`, `process_node` hook, thunk descent, `TransformationHelper`) and differ only in what `run` returns — the rebuilt tree for transformations, the analysis instance (result readers) for analyses. +- Public parse entry: `ASTTransform.parse(source, file_path:)` / `ASTTransform.parse_file(path)` expose the framework's Prism-backed parsing to analysis-only consumers that never emit. `ASTTransform::SourceParser` owns the parsing seam; `Transformer#build_ast` / `#build_ast_from_file` delegate to it (unchanged public behavior). +- Pass-taxonomy documentation on `AbstractTransformation`, `Transformation`, and the README: structural (`on_*` handlers, pattern matched anywhere), positional (node builders with the bare `run(node) → node` duck type, caller owns traversal), and sibling-annotation (marker statement + next sibling, matched in `process_node` where the child list is visible — `transform!` itself). + ## [3.0.0] - 2026-07-24 ### Added - Line-aligned emission: transformed code is emitted with every loc-carrying statement on its original source line, making backtraces, breakpoints, and debugger display correct by construction (`LineAlignedEmitter`). diff --git a/Gemfile.lock b/Gemfile.lock index 6a40419..327d529 100644 --- a/Gemfile.lock +++ b/Gemfile.lock @@ -1,7 +1,7 @@ PATH remote: . specs: - ast_transform (3.0.0) + ast_transform (3.1.0) parser (>= 3.3) prism (>= 1.5) unparser (>= 0.8) diff --git a/README.md b/README.md index a505849..c7d5f9c 100644 --- a/README.md +++ b/README.md @@ -130,6 +130,16 @@ class MyTransformation < ASTTransformation::AbstractTransformation end ``` +#### Choosing a pass shape + +Passes come in three shapes, distinguished by how the rewrite site is found: + +* **Structural** — the pattern alone identifies the site. Subclass `AbstractTransformation` and write `on_*` handlers; the pass rewrites the pattern wherever it occurs in the tree. +* **Positional (node builders)** — the *caller* owns traversal and has already located the site. Implement a plain `run(node) → node` object (include `TransformationHelper` for the `s(...)` vocabulary); the builder maps a single node to its replacement subtree and never walks. Anything with that duck type composes with `Transformer` and other passes. +* **Sibling-annotation** — the pattern is a marker statement plus its NEXT sibling. Match in `process_node`, where a node's child list is visible: an `on_*` handler sees one node, never its siblings, and cannot delete itself from its parent. `ASTTransform::Transformation` (the `transform!` detector) is the canonical example. + +For read-only passes that harvest information instead of rewriting, see [Analysis passes](#analysis-passes). + #### Transformation discoverability ASTTransform automatically loads your transformations at compile time. As such, we expect your files to be located at a known path. @@ -162,6 +172,46 @@ In the above, `node#updated` allows updating the node, either its type or its ch The [ast gem](https://github.com/whitequark/ast) uses a pattern in which a Transformation may implement a method matching a node type, i.e. `on_class`, `on_send`, `on_lvar`, etc... This is very useful when transformations should process all nodes of this type. +### Analysis passes + +Not every pass rewrites. An analysis walks the tree and harvests information — `Parser::AST::Processor` has no read-only mode (every walk functionally rebuilds the tree), so an analysis is simply a walk whose rebuilt tree is discarded. Derive from `ASTTransform::AbstractAnalysis`: it shares the traversal engine with `AbstractTransformation` (including thunk descent and the `s(...)` matching vocabulary) and differs only in what `run` returns — the analysis instance, so callers chain result readers off the run. + +```ruby +require 'ast_transform/abstract_analysis' + +class TypeAliasRanges < ASTTransform::AbstractAnalysis + attr_reader :ranges + + def initialize + @ranges = [] + super + end + + def on_block(node) + send_node = node.children.first + @ranges << (node.loc.expression.first_line..node.loc.expression.last_line) if type_alias?(send_node) + super + end + + private + + def type_alias?(send_node) + send_node == s(:send, s(:const, nil, :T), :type_alias) + end +end +``` + +Harvest state in `on_*` handlers and always call `super` so traversal continues. Node equality ignores source locations, so `s(...)` patterns match structurally. + +Analysis-only consumers don't need a `Transformer`; parse directly through the shared seam: + +```ruby +ASTTransform.parse(source) # => Parser::AST::Node +ASTTransform.parse_file(file_path) # => Parser::AST::Node + +TypeAliasRanges.new.run(ASTTransform.parse(source)).ranges +``` + ### Line-aligned emission and the authoring contract ASTTransform owns text and lines; transform authors own semantics and execution order. The contract: diff --git a/lib/ast_transform.rb b/lib/ast_transform.rb index e07cf69..2f23a90 100644 --- a/lib/ast_transform.rb +++ b/lib/ast_transform.rb @@ -4,11 +4,32 @@ require "ast_transform/instruction_sequence" require "ast_transform/instruction_sequence/mixin" require "ast_transform/instruction_sequence/bootsnap_mixin" +require "ast_transform/source_parser" module ASTTransform DEFAULT_OUTPUT_PATH = Pathname.new("").join("tmp", "ast_transform").to_s class << self + # Parses the given +source+ into the framework's AST vocabulary. The entry point for analysis-only consumers + # that never emit (see ASTTransform::AbstractAnalysis); transformation pipelines go through Transformer. + # + # @param source [String] The input source code. + # @param file_path [String] The file path recorded on source locations. + # + # @return [Parser::AST::Node] The AST. + def parse(source, file_path: "tmp") + source_parser.parse(source, file_path: file_path) + end + + # Parses the source in the given +file_path+ (see .parse). + # + # @param file_path [String] The input file path. + # + # @return [Parser::AST::Node] The AST. + def parse_file(file_path) + source_parser.parse_file(file_path) + end + def acronyms @acronyms ||= [] end @@ -35,5 +56,11 @@ class << RubyVM::InstructionSequence def output_path @output_path || DEFAULT_OUTPUT_PATH end + + private + + def source_parser + @source_parser ||= SourceParser.new + end end end diff --git a/lib/ast_transform/abstract_analysis.rb b/lib/ast_transform/abstract_analysis.rb new file mode 100644 index 0000000..d0821dd --- /dev/null +++ b/lib/ast_transform/abstract_analysis.rb @@ -0,0 +1,38 @@ +# frozen_string_literal: true + +require "ast_transform/abstract_processor" + +module ASTTransform + # Base class for read-only analysis passes: subclass, harvest state in +on_*+ handlers (always call +super+ so + # traversal continues), and expose results through readers. The walk still functionally rebuilds the tree — + # Parser::AST::Processor has no read-only mode — but +run+ discards the rebuilt tree, so handlers never need to + # care what they return. + # + # class SendCounter < ASTTransform::AbstractAnalysis + # attr_reader :count + # + # def initialize + # @count = 0 + # super + # end + # + # def on_send(node) + # @count += 1 + # super + # end + # end + # + # SendCounter.new.run(ASTTransform.parse(source)).count + class AbstractAnalysis < AbstractProcessor + # Runs this analysis on +node+, discarding the rebuilt tree. + # Note: If you want to add one-time setup or result finalization, override this, then call super. + # + # @param node [Parser::AST::Node] The node to be analyzed. + # + # @return [ASTTransform::AbstractAnalysis] self, so callers can chain result readers off the run. + def run(node) + process(node) + self + end + end +end diff --git a/lib/ast_transform/abstract_processor.rb b/lib/ast_transform/abstract_processor.rb new file mode 100644 index 0000000..bea4c6d --- /dev/null +++ b/lib/ast_transform/abstract_processor.rb @@ -0,0 +1,39 @@ +# frozen_string_literal: true + +require "ast_transform/transformation_helper" + +module ASTTransform + # Shared traversal core for tree passes. Parser::AST::Processor is a rewriting walker — every visit functionally + # rebuilds the tree — so transformation and analysis share one engine and differ only in what they keep: + # AbstractTransformation's +run+ returns the rebuilt tree, AbstractAnalysis's +run+ discards it and returns the + # harvested results. Subclass one of those leaves rather than this class. + class AbstractProcessor < Parser::AST::Processor + include TransformationHelper + + # Used internally by Parser::AST::Processor to process each node. DO NOT OVERRIDE. + def process(node) + return node unless node.is_a?(Parser::AST::Node) + + process_node(node) + end + + # Thunks are framework-owned IR: descend into the body so passes that don't know about thunks still process the + # wrapped statements. Without this, Processor's handler_missing default would pass the node through opaquely, + # hiding the body from every later pass. The token (first child) is not a node and passes through untouched. + def on_ast_thunk(node) + node.updated(nil, [node.children[0], *process_all(node.children.drop(1))]) + end + + private + + # Processes the given +node+. + # Note: If you want to do processing on each node, override this. + # + # @param node [Parser::AST::Node] The node being visited. + # + # @return [Parser::AST::Node] The rebuilt node. + def process_node(node) + method(:process).super_method.call(node) + end + end +end diff --git a/lib/ast_transform/abstract_transformation.rb b/lib/ast_transform/abstract_transformation.rb index 56fd0c7..faf7b12 100644 --- a/lib/ast_transform/abstract_transformation.rb +++ b/lib/ast_transform/abstract_transformation.rb @@ -1,12 +1,24 @@ # frozen_string_literal: true -require "ast_transform/transformation_helper" +require "ast_transform/abstract_processor" module ASTTransform - class AbstractTransformation < Parser::AST::Processor - include TransformationHelper - - # Runs this transformation on +node+. + # Base class for structural transformations: subclass and write +on_*+ handlers to rewrite a pattern wherever it + # occurs in the tree. This is one of three authoring shapes — pick by how the rewrite site is found: + # + # - Structural (this class): the pattern alone identifies the site, so +on_*+ handlers rewrite it anywhere in + # the tree. + # - Positional (node builders): a plain object with +run(node) → node+, including only TransformationHelper. The + # caller owns traversal and applies the builder at a site it already located; the builder never walks. Anything + # with that duck type composes with Transformer and other passes — deriving from this class is just one way to + # implement it. + # - Sibling-annotation: the pattern is a marker statement plus its NEXT sibling, matched in +process_node+ where + # the child list is visible — an +on_*+ handler sees one node, never its siblings, and cannot delete itself + # from its parent. See ASTTransform::Transformation (the +transform!+ detector) for the canonical example. + # + # For read-only passes that harvest information instead of rewriting, see ASTTransform::AbstractAnalysis. + class AbstractTransformation < AbstractProcessor + # Runs this transformation on +node+ and returns the rebuilt tree. # Note: If you want to add one-time checks to the transformation, override this, then call super. # # @param node [Parser::AST::Node] The node to be transformed. @@ -15,32 +27,5 @@ class AbstractTransformation < Parser::AST::Processor def run(node) process(node) end - - # Used internally by Parser::AST::Processor to process each node. DO NOT OVERRIDE. - def process(node) - return node unless node.is_a?(Parser::AST::Node) - - process_node(node) - end - - # Thunks are framework-owned IR: descend into the body so passes that don't know about thunks still process the - # wrapped statements. Without this, Processor's handler_missing default would pass the node through opaquely, - # hiding the body from every later transformation. The token (first child) is not a node and passes through - # untouched. - def on_ast_thunk(node) - node.updated(nil, [node.children[0], *process_all(node.children.drop(1))]) - end - - private - - # Processes the given +node+. - # Note: If you want to do processing on each node, override this. - # - # @param node [Parser::AST::Node] The node to be transformed. - # - # @return [Parser::AST::Node] The transformed node. - def process_node(node) - method(:process).super_method.call(node) - end end end diff --git a/lib/ast_transform/source_parser.rb b/lib/ast_transform/source_parser.rb new file mode 100644 index 0000000..f9e3709 --- /dev/null +++ b/lib/ast_transform/source_parser.rb @@ -0,0 +1,56 @@ +# frozen_string_literal: true + +require "prism" +require "prism/translation/parser" +require "ast_transform/kwargs_builder" + +module ASTTransform + # Owns source → AST parsing: Prism's C parser through its whitequark translation layer, so every consumer gets the + # node vocabulary Parser::AST::Processor understands. Transformer parses through this seam, and ASTTransform.parse + # exposes it to analysis-only consumers that never emit. + class SourceParser + # Parses the given +source+. + # + # @param source [String] The input source code. + # @param file_path [String] The file path recorded on source locations. This is important for source mapping + # in backtraces. + # + # @return [Parser::AST::Node] The AST. + def parse(source, file_path: "tmp") + buffer = create_buffer(source, file_path) + parser.parse(buffer) + end + + # Parses the source in the given +file_path+. + # + # @param file_path [String] The input file path. + # + # @return [Parser::AST::Node] The AST. + def parse_file(file_path) + parse(File.read(file_path), file_path: file_path) + end + + private + + # Builds a source buffer over +source+ in the parser's default encoding. + # + # @param source [String] The input source code. + # @param file_path [String] The file path recorded on the buffer. + # + # @return [Parser::Source::Buffer] The buffer. + def create_buffer(source, file_path) + buffer = Parser::Source::Buffer.new(file_path) + buffer.source = source.dup.force_encoding(parser.default_encoding) + + buffer + end + + # The memoized parser, reset between parses. + # + # @return [Prism::Translation::Parser] The parser. + def parser + @parser&.reset + @parser ||= Prism::Translation::Parser.new(ASTTransform::KwargsBuilder.new) + end + end +end diff --git a/lib/ast_transform/transformation.rb b/lib/ast_transform/transformation.rb index 1fe4caa..90d4239 100644 --- a/lib/ast_transform/transformation.rb +++ b/lib/ast_transform/transformation.rb @@ -6,6 +6,11 @@ require "unparser" module ASTTransform + # The +transform!+ detector — the canonical sibling-annotation pass (see AbstractTransformation for the taxonomy). + # A +transform!(...)+ statement is a pragma on the NEXT sibling: the transformations it names are applied to the + # following class definition or constant assignment, and the marker itself is deleted from the child list. Both + # effects need the parent's child list in view, which is why matching happens in +process_node+ — an +on_send+ + # handler would see the marker node alone, with no access to its next sibling and no way to remove itself. class Transformation < ASTTransform::AbstractTransformation TRANSFORM_AST = s(:send, nil, :transform!) diff --git a/lib/ast_transform/transformer.rb b/lib/ast_transform/transformer.rb index 567aaf4..16d9248 100644 --- a/lib/ast_transform/transformer.rb +++ b/lib/ast_transform/transformer.rb @@ -1,10 +1,7 @@ # frozen_string_literal: true -require 'prism' -require 'prism/translation/parser' -require 'unparser' -require 'ast_transform/kwargs_builder' require 'ast_transform/line_aligned_emitter' +require 'ast_transform/source_parser' module ASTTransform class Transformer @@ -15,6 +12,7 @@ class Transformer def initialize(*transformations, emitter: LineAlignedEmitter.new) @transformations = transformations @emitter = emitter + @source_parser = SourceParser.new end # Builds the AST for the given +source+. @@ -24,8 +22,7 @@ def initialize(*transformations, emitter: LineAlignedEmitter.new) # # @return [Parser::AST::Node] The AST. def build_ast(source, file_path: "tmp") - buffer = create_buffer(source, file_path) - parser.parse(buffer) + @source_parser.parse(source, file_path: file_path) end # Builds the AST for the given +file_path+. @@ -34,8 +31,7 @@ def build_ast(source, file_path: "tmp") # # @return [Parser::AST::Node] The AST. def build_ast_from_file(file_path) - source = File.read(file_path) - build_ast(source, file_path: file_path) + @source_parser.parse_file(file_path) end # Transforms the given +source+. @@ -89,19 +85,5 @@ def transform_ast(ast) transformation.run(ast) end end - - private - - def create_buffer(source, file_path) - buffer = Parser::Source::Buffer.new(file_path) - buffer.source = source.dup.force_encoding(parser.default_encoding) - - buffer - end - - def parser - @parser&.reset - @parser ||= Prism::Translation::Parser.new(ASTTransform::KwargsBuilder.new) - end end end diff --git a/lib/ast_transform/version.rb b/lib/ast_transform/version.rb index 2cfdbce..3370827 100644 --- a/lib/ast_transform/version.rb +++ b/lib/ast_transform/version.rb @@ -1,5 +1,5 @@ # frozen_string_literal: true module ASTTransform - VERSION = "3.0.0" + VERSION = "3.1.0" end diff --git a/test/ast_transform/abstract_analysis_test.rb b/test/ast_transform/abstract_analysis_test.rb new file mode 100644 index 0000000..a7cab8d --- /dev/null +++ b/test/ast_transform/abstract_analysis_test.rb @@ -0,0 +1,61 @@ +# frozen_string_literal: true + +require "test_helper" +require "transformation_helper" +require "ast_transform/abstract_analysis" + +module ASTTransform + class AbstractAnalysisTest < Minitest::Test + extend ASTTransform::Declarative + include ASTTransform::Helpers::TransformationHelper + + class SendNameCollector < ASTTransform::AbstractAnalysis + attr_reader :names + + def initialize + @names = [] + super + end + + def on_send(node) + @names << node.children[1] + super + end + end + + test "#run returns the analysis instance" do + analysis = SendNameCollector.new + + assert_same analysis, analysis.run(s(:send, nil, :a)) + end + + test "#run harvests from every matching node in the tree" do + ast = ASTTransform.parse(<<~CODE) + foo + bar(baz) + CODE + + assert_equal [:foo, :bar, :baz], SendNameCollector.new.run(ast).names + end + + test "#run returns the analysis instance for non-node input without harvesting" do + analysis = SendNameCollector.new + + assert_same analysis, analysis.run(Object.new) + assert_empty analysis.names + end + + test "#run descends into thunk bodies" do + ast = s(:begin, thunk(s(:send, nil, :hidden))) + + assert_equal [:hidden], SendNameCollector.new.run(ast).names + end + + test "#run leaves the analyzed tree equal to a fresh parse" do + ast = ASTTransform.parse("foo(bar)") + SendNameCollector.new.run(ast) + + assert_equal ASTTransform.parse("foo(bar)"), ast + end + end +end diff --git a/test/ast_transform/source_parser_test.rb b/test/ast_transform/source_parser_test.rb new file mode 100644 index 0000000..f37d2e6 --- /dev/null +++ b/test/ast_transform/source_parser_test.rb @@ -0,0 +1,57 @@ +# frozen_string_literal: true + +require "tmpdir" +require "test_helper" +require "transformation_helper" +require "ast_transform/source_parser" + +module ASTTransform + class SourceParserTest < Minitest::Test + extend ASTTransform::Declarative + include ASTTransform::Helpers::TransformationHelper + + def setup + @source_parser = ASTTransform::SourceParser.new + end + + test "#parse returns the expected AST" do + assert_equal s(:send, nil, :method_call), @source_parser.parse("method_call") + end + + test "#parse records the file path on source locations" do + ast = @source_parser.parse("method_call", file_path: "some/file.rb") + + assert_equal "some/file.rb", ast.loc.expression.source_buffer.name + end + + test "#parse can be called repeatedly on one instance" do + assert_equal s(:send, nil, :first), @source_parser.parse("first") + assert_equal s(:send, nil, :second), @source_parser.parse("second") + end + + test "#parse_file parses the file's source and records its path" do + Dir.mktmpdir do |dir| + file_path = File.join(dir, "input.rb") + File.write(file_path, "method_call\n") + + ast = @source_parser.parse_file(file_path) + + assert_equal s(:send, nil, :method_call), ast + assert_equal file_path, ast.loc.expression.source_buffer.name + end + end + + test "ASTTransform.parse parses through the shared seam" do + assert_equal s(:send, nil, :method_call), ASTTransform.parse("method_call") + end + + test "ASTTransform.parse_file parses the file's source" do + Dir.mktmpdir do |dir| + file_path = File.join(dir, "input.rb") + File.write(file_path, "method_call\n") + + assert_equal s(:send, nil, :method_call), ASTTransform.parse_file(file_path) + end + end + end +end From 814a69329d4f6fb24d4b03e7babd6bf7f56ca673 Mon Sep 17 00:00:00 2001 From: Jean-Philippe Date: Wed, 29 Jul 2026 10:11:33 -0400 Subject: [PATCH 2/4] Drop module-level parse pass-throughs; clients instantiate SourceParser MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Class-method delegation to a hidden memoized instance is the shape the org's constructor-injection rule forbids: it hides the collaborator and locks the seam shut. SourceParser is the public parse entry — callers construct it and keep the instance for as many parses as they need. Co-authored-by: Cursor --- CHANGELOG.md | 2 +- README.md | 10 +++++--- lib/ast_transform.rb | 26 -------------------- lib/ast_transform/abstract_analysis.rb | 2 +- lib/ast_transform/source_parser.rb | 4 +-- test/ast_transform/abstract_analysis_test.rb | 7 +++--- test/ast_transform/source_parser_test.rb | 13 ---------- 7 files changed, 14 insertions(+), 50 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 5622fb2..3dd6e63 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,7 +7,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [3.1.0] - 2026-07-29 ### Added - Analysis passes: `ASTTransform::AbstractAnalysis`, the read-only counterpart of `AbstractTransformation`. Both leaves share the extracted traversal core `ASTTransform::AbstractProcessor` (sealed `process`, `process_node` hook, thunk descent, `TransformationHelper`) and differ only in what `run` returns — the rebuilt tree for transformations, the analysis instance (result readers) for analyses. -- Public parse entry: `ASTTransform.parse(source, file_path:)` / `ASTTransform.parse_file(path)` expose the framework's Prism-backed parsing to analysis-only consumers that never emit. `ASTTransform::SourceParser` owns the parsing seam; `Transformer#build_ast` / `#build_ast_from_file` delegate to it (unchanged public behavior). +- Public parse seam: `ASTTransform::SourceParser` (`#parse(source, file_path:)` / `#parse_file(path)`) extracts the framework's Prism-backed parsing from `Transformer` so analysis-only consumers that never emit can instantiate it directly. `Transformer#build_ast` / `#build_ast_from_file` delegate to it (unchanged public behavior). - Pass-taxonomy documentation on `AbstractTransformation`, `Transformation`, and the README: structural (`on_*` handlers, pattern matched anywhere), positional (node builders with the bare `run(node) → node` duck type, caller owns traversal), and sibling-annotation (marker statement + next sibling, matched in `process_node` where the child list is visible — `transform!` itself). ## [3.0.0] - 2026-07-24 diff --git a/README.md b/README.md index c7d5f9c..e170df6 100644 --- a/README.md +++ b/README.md @@ -203,13 +203,15 @@ end Harvest state in `on_*` handlers and always call `super` so traversal continues. Node equality ignores source locations, so `s(...)` patterns match structurally. -Analysis-only consumers don't need a `Transformer`; parse directly through the shared seam: +Analysis-only consumers don't need a `Transformer`; instantiate the parsing seam directly (`Transformer` uses the same class internally) and keep the instance for as many parses as you need: ```ruby -ASTTransform.parse(source) # => Parser::AST::Node -ASTTransform.parse_file(file_path) # => Parser::AST::Node +parser = ASTTransform::SourceParser.new -TypeAliasRanges.new.run(ASTTransform.parse(source)).ranges +parser.parse(source) # => Parser::AST::Node +parser.parse_file(file_path) # => Parser::AST::Node + +TypeAliasRanges.new.run(parser.parse(source)).ranges ``` ### Line-aligned emission and the authoring contract diff --git a/lib/ast_transform.rb b/lib/ast_transform.rb index 2f23a90..643030d 100644 --- a/lib/ast_transform.rb +++ b/lib/ast_transform.rb @@ -10,26 +10,6 @@ module ASTTransform DEFAULT_OUTPUT_PATH = Pathname.new("").join("tmp", "ast_transform").to_s class << self - # Parses the given +source+ into the framework's AST vocabulary. The entry point for analysis-only consumers - # that never emit (see ASTTransform::AbstractAnalysis); transformation pipelines go through Transformer. - # - # @param source [String] The input source code. - # @param file_path [String] The file path recorded on source locations. - # - # @return [Parser::AST::Node] The AST. - def parse(source, file_path: "tmp") - source_parser.parse(source, file_path: file_path) - end - - # Parses the source in the given +file_path+ (see .parse). - # - # @param file_path [String] The input file path. - # - # @return [Parser::AST::Node] The AST. - def parse_file(file_path) - source_parser.parse_file(file_path) - end - def acronyms @acronyms ||= [] end @@ -56,11 +36,5 @@ class << RubyVM::InstructionSequence def output_path @output_path || DEFAULT_OUTPUT_PATH end - - private - - def source_parser - @source_parser ||= SourceParser.new - end end end diff --git a/lib/ast_transform/abstract_analysis.rb b/lib/ast_transform/abstract_analysis.rb index d0821dd..63d7086 100644 --- a/lib/ast_transform/abstract_analysis.rb +++ b/lib/ast_transform/abstract_analysis.rb @@ -22,7 +22,7 @@ module ASTTransform # end # end # - # SendCounter.new.run(ASTTransform.parse(source)).count + # SendCounter.new.run(SourceParser.new.parse(source)).count class AbstractAnalysis < AbstractProcessor # Runs this analysis on +node+, discarding the rebuilt tree. # Note: If you want to add one-time setup or result finalization, override this, then call super. diff --git a/lib/ast_transform/source_parser.rb b/lib/ast_transform/source_parser.rb index f9e3709..df5b252 100644 --- a/lib/ast_transform/source_parser.rb +++ b/lib/ast_transform/source_parser.rb @@ -6,8 +6,8 @@ module ASTTransform # Owns source → AST parsing: Prism's C parser through its whitequark translation layer, so every consumer gets the - # node vocabulary Parser::AST::Processor understands. Transformer parses through this seam, and ASTTransform.parse - # exposes it to analysis-only consumers that never emit. + # node vocabulary Parser::AST::Processor understands. Transformer parses through this seam; analysis-only + # consumers that never emit instantiate it directly and keep the instance for as many parses as they need. class SourceParser # Parses the given +source+. # diff --git a/test/ast_transform/abstract_analysis_test.rb b/test/ast_transform/abstract_analysis_test.rb index a7cab8d..d9044d7 100644 --- a/test/ast_transform/abstract_analysis_test.rb +++ b/test/ast_transform/abstract_analysis_test.rb @@ -30,7 +30,7 @@ def on_send(node) end test "#run harvests from every matching node in the tree" do - ast = ASTTransform.parse(<<~CODE) + ast = ASTTransform::SourceParser.new.parse(<<~CODE) foo bar(baz) CODE @@ -52,10 +52,11 @@ def on_send(node) end test "#run leaves the analyzed tree equal to a fresh parse" do - ast = ASTTransform.parse("foo(bar)") + source_parser = ASTTransform::SourceParser.new + ast = source_parser.parse("foo(bar)") SendNameCollector.new.run(ast) - assert_equal ASTTransform.parse("foo(bar)"), ast + assert_equal source_parser.parse("foo(bar)"), ast end end end diff --git a/test/ast_transform/source_parser_test.rb b/test/ast_transform/source_parser_test.rb index f37d2e6..bad5d9b 100644 --- a/test/ast_transform/source_parser_test.rb +++ b/test/ast_transform/source_parser_test.rb @@ -40,18 +40,5 @@ def setup assert_equal file_path, ast.loc.expression.source_buffer.name end end - - test "ASTTransform.parse parses through the shared seam" do - assert_equal s(:send, nil, :method_call), ASTTransform.parse("method_call") - end - - test "ASTTransform.parse_file parses the file's source" do - Dir.mktmpdir do |dir| - file_path = File.join(dir, "input.rb") - File.write(file_path, "method_call\n") - - assert_equal s(:send, nil, :method_call), ASTTransform.parse_file(file_path) - end - end end end From eeb12b7d4356528c218501cebbb97fbf5f4e9aae Mon Sep 17 00:00:00 2001 From: Jean-Philippe Date: Wed, 29 Jul 2026 10:17:07 -0400 Subject: [PATCH 3/4] Construct the parser per parse; the builder is the constructor-owned collaborator The memoized reset-and-reuse parser was carried over from Transformer's original code: a micro-optimization that made the instance stateful (and create_buffer's default_encoding lookup re-triggered the reset on every access). Parser instances accumulate per-run state and cost nothing to construct next to the parse itself, so they are per-call; the builder is fixed for the instance's lifetime and moves to the constructor. Co-authored-by: Cursor --- lib/ast_transform/source_parser.rb | 29 ++++++++++++++++------------- 1 file changed, 16 insertions(+), 13 deletions(-) diff --git a/lib/ast_transform/source_parser.rb b/lib/ast_transform/source_parser.rb index df5b252..50ad844 100644 --- a/lib/ast_transform/source_parser.rb +++ b/lib/ast_transform/source_parser.rb @@ -9,6 +9,14 @@ module ASTTransform # node vocabulary Parser::AST::Processor understands. Transformer parses through this seam; analysis-only # consumers that never emit instantiate it directly and keep the instance for as many parses as they need. class SourceParser + # Constructs a new SourceParser instance. + # + # @param builder [Prism::Translation::Parser::Builder] The node builder handed to every parser. Fixed for this + # instance's lifetime; the framework's default preserves the kwargs/hash distinction Unparser needs. + def initialize(builder: KwargsBuilder.new) + @builder = builder + end + # Parses the given +source+. # # @param source [String] The input source code. @@ -17,8 +25,10 @@ class SourceParser # # @return [Parser::AST::Node] The AST. def parse(source, file_path: "tmp") - buffer = create_buffer(source, file_path) - parser.parse(buffer) + # A fresh parser per parse: parser instances accumulate per-run state (lexer position, diagnostics), and + # constructing one is trivial next to the parse itself. + parser = Prism::Translation::Parser.new(@builder) + parser.parse(create_buffer(source, file_path, parser.default_encoding)) end # Parses the source in the given +file_path+. @@ -32,25 +42,18 @@ def parse_file(file_path) private - # Builds a source buffer over +source+ in the parser's default encoding. + # Builds a source buffer over +source+ in the given +encoding+. # # @param source [String] The input source code. # @param file_path [String] The file path recorded on the buffer. + # @param encoding [Encoding] The encoding the buffer's source is coerced to. # # @return [Parser::Source::Buffer] The buffer. - def create_buffer(source, file_path) + def create_buffer(source, file_path, encoding) buffer = Parser::Source::Buffer.new(file_path) - buffer.source = source.dup.force_encoding(parser.default_encoding) + buffer.source = source.dup.force_encoding(encoding) buffer end - - # The memoized parser, reset between parses. - # - # @return [Prism::Translation::Parser] The parser. - def parser - @parser&.reset - @parser ||= Prism::Translation::Parser.new(ASTTransform::KwargsBuilder.new) - end end end From 7b90fbb7cebd773ccc13be16a2a22fc2f4eb63fd Mon Sep 17 00:00:00 2001 From: Jean-Philippe Date: Wed, 29 Jul 2026 10:23:23 -0400 Subject: [PATCH 4/4] Replace the hand-rolled kwargs rewrite with parser's emit_kwargs opt-in MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The associate override predates the parser gem's native flag and rewrote any braceless hash construction; emit_kwargs rewrites at the call sites where kwargs semantics actually live (calls, index, super/yield) and is maintained upstream. The flag is a class-level ivar, so setting it on the subclass opts in this builder only — the global default is untouched. Verified AST-identical on every probed form via the Prism translation layer. Also stop exposing the builder as a SourceParser constructor parameter: the kwargs/hash distinction is required for the framework's emission to round-trip, an implementation detail rather than an injection seam. Co-authored-by: Cursor --- lib/ast_transform/kwargs_builder.rb | 19 +++++++++---------- lib/ast_transform/source_parser.rb | 8 ++++---- 2 files changed, 13 insertions(+), 14 deletions(-) diff --git a/lib/ast_transform/kwargs_builder.rb b/lib/ast_transform/kwargs_builder.rb index a0a02be..cb54b70 100644 --- a/lib/ast_transform/kwargs_builder.rb +++ b/lib/ast_transform/kwargs_builder.rb @@ -3,22 +3,21 @@ require "prism/translation/parser" module ASTTransform - # Extends the default Prism parser builder to distinguish keyword arguments from hash literals in the AST. + # The framework's parser builder: the stock builder with the parser gem's `emit_kwargs` opt-in, which + # distinguishes keyword arguments from hash literals in the AST. # - # The upstream builder always emits :hash nodes for both `foo(bar: 1)` and `foo({ bar: 1 })`. Unparser uses the - # node type to decide whether to emit braces: :hash gets `{}`, :kwargs does not. Since Ruby 3.0+ treats these as - # semantically different (strict keyword/positional separation), we need the AST to preserve the distinction. + # With the flag off (the gem's compatibility default) both `foo(bar: 1)` and `foo({ bar: 1 })` emit :hash nodes. + # Unparser uses the node type to decide whether to emit braces: :hash gets `{}`, :kwargs does not. Since Ruby 3.0+ + # treats these as semantically different (strict keyword/positional separation), the AST must preserve the + # distinction. `emit_kwargs` rewrites at the call sites where kwargs semantics live (method calls, index, + # super/yield); the flag is a class-level ivar, so setting it here opts in this builder only — the global + # Parser::Builders::Default stays untouched. # # NOTE: parsed nodes deliberately stay plain Parser::AST::Node. Custom node classes exist only for registered # custom types (see ASTTransform::Node), which are IR and never reach Unparser: AST::Node#eql? compares class, and # Unparser verifies dynamic-string emission by re-parsing and comparing eql? against the freshly parsed # (plain-class) node — custom-class nodes of standard types would fail that verification. class KwargsBuilder < Prism::Translation::Parser::Builder - def associate(begin_t, pairs, end_t) - node = super - return node unless begin_t.nil? && end_t.nil? - - node.updated(:kwargs) - end + self.emit_kwargs = true end end diff --git a/lib/ast_transform/source_parser.rb b/lib/ast_transform/source_parser.rb index 50ad844..d3be302 100644 --- a/lib/ast_transform/source_parser.rb +++ b/lib/ast_transform/source_parser.rb @@ -11,10 +11,10 @@ module ASTTransform class SourceParser # Constructs a new SourceParser instance. # - # @param builder [Prism::Translation::Parser::Builder] The node builder handed to every parser. Fixed for this - # instance's lifetime; the framework's default preserves the kwargs/hash distinction Unparser needs. - def initialize(builder: KwargsBuilder.new) - @builder = builder + # KwargsBuilder is a required implementation detail, not an injection seam: the framework's emission depends + # on the kwargs/hash distinction it preserves, so every parse goes through it. + def initialize + @builder = KwargsBuilder.new end # Parses the given +source+.