Skip to content

fix(clojure): extract calls whose list head carries metadata (#548) - #715

Closed
himanchalkaushale wants to merge 2 commits into
redhat-et:mainfrom
himanchalkaushale:lane/clj-meta-call-548
Closed

himanchalkaushale wants to merge 2 commits into
redhat-et:mainfrom
himanchalkaushale:lane/clj-meta-call-548

Conversation

@himanchalkaushale

@himanchalkaushale himanchalkaushale commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Clojure extraction was missing valid function calls when metadata appeared before the function name inside a list.

For example:

^{:doc "tagged"} (square 4)
^String (square 5)

The issue was that tree-sitter-clojure represents metadata as a meta_lit node. The existing query did not account for this node before the function name, so these calls were skipped.

This PR fixes the issue by allowing metadata before the list head, just like comments and discard expressions.

The C++ list-head check is also updated to ensure special forms such as if remain excluded correctly.

This PR covers only Arm 1 of #548.

Implementation Details

Updated queries/clojure/tags.scm to allow comment, dis_expr, and meta_lit before the function name.

Updated clojureListHead in src/ingest_clojure.h to skip metadata when identifying the actual list head. This prevents metadata from being mistaken for the function name and keeps special-form filtering working correctly.

Added regression cases in test/clojurefix/core.clj covering normal calls, metadata calls, quoted expressions, special forms, and metadata attached to symbols.

Updated test/clojurecheck.sh with assertions for these cases, including an explicit --callers=square check.

Also updated parser versions:

  • kParserVer: 309 → 311
  • Rich file tag: 312
  • kIngestParserVerMirror: 309 → 311

Updated test/qschemetrip.hash and added a dated justification entry to test/qschemetripcheck.sh.

Testing

The following checks passed locally:

  • clojurecheck.sh: ALL PASS
  • qschemetripcheck.sh: ALL PASS
  • qextractionkeycheck.sh: PASS, mirror 311 == 311
  • langidentitycheck.sh: 371 SAME / 9 DIFF / 0 EMPTY / 0 ERROR

The 9 differences are expected and limited to the Clojure changes.

Two full-repository runs produced byte-identical output, and XML validation passed.

Build and test commands

cmake -S . -B build
cmake --build build -j

bash test/clojurecheck.sh ./build/ripwire
bash test/qschemetripcheck.sh
bash test/qextractionkeycheck.sh ./build/ripwire

Manual verification

./build/ripwire test/clojurefix --no-cache --callees=meta-call
./build/ripwire test/clojurefix --no-cache --callees=tag-call
./build/ripwire test/clojurefix --no-cache --callees=quoted-meta
./build/ripwire test/clojurefix --no-cache --callees=special-form-metadata
./build/ripwire test/clojurefix --no-cache --callers=square

Metadata calls are now detected correctly, while quoted expressions, special forms, and metadata attached to symbols remain excluded.

ASan and the full pargates.py suite were not run because the development environment was Windows.

Scope

This PR fixes only Arm 1 of #548.

The following are intentionally out of scope:

  • anon_fn_lit (Arm 2)
  • Syntax-quote unquotes
  • Qualified-call resolution
  • Reader conditionals
  • Parser refactoring

Closes Arm 1 of #548.

Summary by CodeRabbit

  • Bug Fixes
    • Clojure calls prefixed with metadata are now captured correctly. Quoted metadata, special forms, and metadata on non-call expressions remain excluded.
    • Caller filtering now works for metadata-prefixed calls.

A Clojure list with metadata before its head (`^{:doc "tagged"} (square 4)`,
`^String (square 5)`) parses with a named meta_lit child in front of the head,
so the call pattern's `.` anchor never reached the sym_lit and the call was
dropped. Accept meta_lit in the same leading slot as comments and discards.

The C++ head filter must skip meta_lit too: otherwise a metadata-prefixed
special form would read as head "meta_lit" and bypass the special-form gate.
Metadata on non-call forms and inside quoted code stays inert.

Bump kParserVer 309 -> 311 (rich file tag 312) with quality.h's mirror and
re-pin test/qschemetrip.hash. Add fixture and gate arms for tagged, plain,
quoted, special-form, and non-call metadata, including an explicit
--callers=square assertion. Arm 1 of redhat-et#548 only; anon_fn_lit is untouched.
@coderabbitai

coderabbitai Bot commented Nov 21, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • Trigger review

No actionable comments were generated in the recent review.

Walkthrough

Clojure direct-call extraction now accepts metadata literals between a list head and the captured symbol. The C++ list-head filter also skips metadata literals. Tests cover metadata-prefixed calls, excluded forms, and caller filtering. Parser-version markers and the re-pin log are updated; cache versions remain unchanged.

Changes: Clojure call extraction

Layer / File(s) Summary
Metadata-prefixed call extraction
queries/clojure/tags.scm, src/ingest_clojure.h, src/ingest_cache.h, src/quality.h, test/clojurecheck.sh, test/qschemetripcheck.sh
The direct-call query and C++ list-head filter now skip metadata literals. Tests cover metadata-prefixed calls, quoted metadata, special forms, non-calls, and caller filtering. Parser-version markers change from 309 to 311; the rich-file tag is 312. The cache version and cache scheme remain unchanged.

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix

Suggested reviewers: tag-und-nacht, joyful-ii-v-i

Merge Risk: Minimal · previous review 63a7a

This change makes Clojure calls with metadata before the function name, such as ^String (square 5), appear in extracted call data. Special forms and metadata on non-call forms remain excluded. No merge-blocking risk was found.

Pre-merge checks: Passed 4

Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title names the main change: Clojure call extraction now handles list heads that carry metadata. The PR objectives confirm the query and clojureListHead changes that do this. The title follows con…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Title check

Explanation

The title names the main change: Clojure call extraction now handles list heads that carry metadata. The PR objectives confirm the query and clojureListHead changes that do this. The title follows conventional commit form and cites issue #548. It is specific and accurate.

Recent review info

Run configuration

Commits

Reviewing files that changed from the base of the PR and between f79a94e and 63a7aaf.

Files ignored due to path filters: 2

  • test/clojurefix/core.clj is excluded by !test/*fix/**
  • test/qschemetrip.hash is excluded by !test/*.hash
Files selected for processing: 6

  • queries/clojure/tags.scm
  • src/ingest_cache.h
  • src/ingest_clojure.h
  • src/quality.h
  • test/clojurecheck.sh
  • test/qschemetripcheck.sh

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 7 remain after this review.

# Conflicts:
#	src/ingest_cache.h
#	src/quality.h
#	test/qschemetrip.hash
#	test/qschemetripcheck.sh
@joyful-ii-V-I

Copy link
Copy Markdown
Collaborator

Thanks @himanchalkaushale, this is a clean fix for arm 1. Treating meta_lit like comment and dis_expr before the list head, in both the query and clojureListHead, is the right place for it. The fixture covers the cases that matter: plain, map metadata, tag metadata, quoted (stays inert), a special form under metadata (stays excluded) and metadata on a non-call. The --callers=square check is a good end-to-end proof.

A few notes. None of them blocks:

  1. Versions: main has moved since your branch, and the ingest cache is now 54. 335/336 is also taken by a train already in review. You don't need to chase the numbers. We land community PRs by rebasing them onto main ourselves, so we'll renumber kParserVer and re-pin qschemetrip.hash then. The rebase also replaces the merge commit. Your authorship and the Closes link stay.
  2. Small tidy in clojureListHead: the loop condition calls ts_node_named_child/ts_node_type three times per child. Reading the kind once into a local and testing it against the three kinds would be cheaper and easier to read. If you'd rather not, we can make that change during the rebase.
  3. Optional extra arms, if you want them:
    • stacked metadata, ^:a ^{:b 1} (square 1), which the * should already handle;
    • the legacy #^String (square 1) reader form, which tree-sitter-clojure parses as old_meta_lit. If that form isn't covered, a one-line note in the PR is fine.

CI is waiting for a maintainer to approve the workflow run for your fork, so we'll start that. Thanks again for picking this up!

joyful-ii-V-I added a commit that referenced this pull request Nov 21, 2026
Lane land-715 lands #715 (@himanchalkaushale, #548 arm 1) on train 1118a.
The Clojure query and clojureListHead change bump kParserVer to 337 above
the train's 335/336; re-pin test/qschemetrip.hash once from that build and
log it. kCacheVersion 54 and kQSnapCacheScheme 38 are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
joyful-ii-V-I added a commit that referenced this pull request Nov 21, 2026
The head scan called ts_node_named_child and ts_node_type three times per
leading child (comment, dis_expr, meta_lit). Read the child and its kind
once into locals and return from the first child outside that slot. Same
behaviour; follows the #715 review note.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
joyful-ii-V-I added a commit that referenced this pull request Nov 21, 2026
… metadata

The special-form arm used `if`, which has no in-tree def, so a leak past
the special-form gate could never show as a resolved call: with the
specialForms test removed the gate still passed. Add a `doto` def to the
fixture and a metadata-prefixed `(doto 1)` beside the `if`; that mutant
now fails the arm on {'doto'}. metadata-not-call now names `square` (an
in-tree def) instead of the unresolvable `foo`. Add a stacked-metadata
caller, `^:a ^{:b 1} (square 6)`, from the #715 review.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
joyful-ii-V-I added a commit that referenced this pull request Nov 22, 2026
clojure: extract calls whose list head carries metadata (lands #715, #548 first arm)
@joyful-ii-V-I

Copy link
Copy Markdown
Collaborator

Landed in #718 with your commit and authorship kept (rebased onto main, merge commit dropped), plus a hash re-pin and stronger test arms on our side. Thank you for the fix! The head-symbol metadata case and the legacy #^ form are tracked as follow-ups for the rest of #548.

@himanchalkaushale
himanchalkaushale deleted the lane/clj-meta-call-548 branch November 22, 2026 06:26
moses-y pushed a commit to moses-y/ripwire that referenced this pull request Nov 22, 2026
…ich 338)

Lane land-715 lands redhat-et#715 (@himanchalkaushale, redhat-et#548 arm 1) on train 1118a.
The Clojure query and clojureListHead change bump kParserVer to 337 above
the train's 335/336; re-pin test/qschemetrip.hash once from that build and
log it. kCacheVersion 54 and kQSnapCacheScheme 38 are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
moses-y pushed a commit to moses-y/ripwire that referenced this pull request Nov 22, 2026
The head scan called ts_node_named_child and ts_node_type three times per
leading child (comment, dis_expr, meta_lit). Read the child and its kind
once into locals and return from the first child outside that slot. Same
behaviour; follows the redhat-et#715 review note.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
moses-y pushed a commit to moses-y/ripwire that referenced this pull request Nov 22, 2026
…; stacked metadata

The special-form arm used `if`, which has no in-tree def, so a leak past
the special-form gate could never show as a resolved call: with the
specialForms test removed the gate still passed. Add a `doto` def to the
fixture and a metadata-prefixed `(doto 1)` beside the `if`; that mutant
now fails the arm on {'doto'}. metadata-not-call now names `square` (an
in-tree def) instead of the unresolvable `foo`. Add a stacked-metadata
caller, `^:a ^{:b 1} (square 6)`, from the redhat-et#715 review.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
joyful-ii-V-I added a commit that referenced this pull request Nov 22, 2026
Brings in every community PR main has landed since the train's last main merge: #715
(@himanchalkaushale, #548 arm 1: Clojure calls whose list head carries metadata), its
qschemetrip re-pin and clojurecheck hardening, and main's #717 merge (the train already
carries #717's content). Merged, not rebased: the train contains the pushed
origin/integration/070-1120, and contributor commits stay as merged.

Conflicts — a parser-version collision, resolved by the house rule:
- src/ingest_cache.h kParserVer and src/quality.h kIngestParserVerMirror: the train's 340
  (rich 341) vs main's 337 (rich 338) -> 342 (rich file tag 343), ONE number above both,
  the mirror equal to it; both sides' history notes kept.
- test/qschemetrip.hash: re-pinned with UPDATE_GOLDEN=1 test/qschemetripcheck.sh on the
  merged sources (a606e48d5b…2f4e02ec), never hand-edited; test/qschemetripcheck.sh's
  RE-PIN LOG keeps both sides' entries under a new one for 342.
kCacheVersion 54 and kQSnapCacheScheme 38 unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants