fix(getter): name C-family functions with nested declarators - #1214
Merged
Conversation
A C/C++/Objective-C function whose declared name sits under an extra declarator layer resolved its FuncSpace::name to null. Two spellings were affected — a function returning a function pointer, `int (*fp(int a, int b))(int c)`, and the macro-obscured declarator `RUN_STATS_METHOD(allocate)(JNIEnv *env)` that JNI shims use. get_func_space_name reached the declarator with first_occurrence, a leftmost pre-order search. When the function node's own declarator field is already a function_declarator — the return type's — the search stops there, and its child(0) is a parenthesized_declarator or an inner function_declarator, neither of which is an identifier kind, so the arm fell through to None. nargs has walked this chain correctly since #1200. That walk moves to src/c_declarator.rs and the four getters read the name off the same innermost declarator, so the arity and the name can no longer describe two different nodes. The FunctionDeclarator alias enumerations go with it (grammar-dispatch.md §1), and the name comes off the `declarator` field rather than child(0) (§3). Measured over DeepSpeech and pdf.js (14,269 files): 44 previously nameless function spaces gain a name, 2 lose one. Both losses are inside ERROR-recovery subtrees where no strategy has a defensible answer; one had been reporting an `if` statement's callee as a function name. The two openfst snapshots move for the same reason — their space's arity has come from the newly-named node since #1200. Search::first_occurrence has no production caller left. It is allowed rather than removed here, tracked in #1212. Fixes #1208
The four C-family get_func_space_name impls each spelled the same two steps — innermost_declarator, then its `declarator` field. Name that pairing in c_declarator::declarator_name so the getters carry only the part that differs between them, which is what each grammar accepts as an identifier kind. Also restores a contract line to Search::first_occurrence's doc: the #1212 note is a maintainer comment, not a description of what the method does.
The four C-family get_func_space_name impls each dispatched a single
match arm and dropped everything else through a dead `_ => {}`. The
kind list is a validity gate, not a dispatch table, so it reads as one:
a matches! on the let chain that already binds the name. One nesting
level goes, the dead arm goes, the coverage claim is unchanged.
the_table_reports_a_name_that_does_not_match claimed to prove the
selector reaches a real function space. It did not: the failure message
echoes the fixture source, so its `contains("plain")` matched the
input rather than the output, and the test passed with function_spaces
filtering on SpaceKind::Class — selecting nothing at all.
Match the rendered `Some("plain")` instead, which only the found side
produces. Verified by re-running that same perturbation: both tests in
the module now fail.
Every row in the table was a shape the getters' matches! accepts, so nothing reached its false branch — the branch that exists to emit no name rather than whatever text sits in the name slot. `int (fp)(int a)` is that shape: redundant parentheses around the name are legal C and put a parenthesized_declarator where the identifier kinds would be. Folding it into the shared table as the one row expecting None turns the two expectation types into one Option-typed column, which is also why check() loses its wrapper. Measured: the gate's guard now runs 607 times per C fixture set, 606 true and 1 false, where the false arm had zero hits before.
The last-named-child fallback is documented as serving the three
fieldless *declarator* rules, but it runs on the C++ name forms too,
which have no `declarator` field either. `template_function` and
`template_method` put their `template_argument_list` last, so a type
argument spelling a function type — which carries a `parameters` field
of its own — pulled the chain off the name side and into the argument.
template <> void tspec<int (*)(int x, int y)>(int a) { }
read as taking two arguments rather than one, and resolved to no name
at all, the abstract declarator the chain landed on spelling no
identifier. Both trees parse cleanly, so neither is covered by the
module's ERROR-recovery caveat. The arity half dates from #1200 and the
name half from this branch; both are unreleased, so one entry covers
them.
Exclude `template_argument_list` alongside `attribute_declaration`.
The narrower exclusion rather than a `_declarator`-suffix gate on the
whole fallback: that gate is the more principled-looking fix and it
loses three corpus names, because
`TF_CAPI_EXPORT extern TF_ConcreteFunction* TF_Get…(…)` misparses into
a `qualified_identifier` with a zero-width `::` that the chain has to
descend through. A non-declarator link is not always a name.
Corpus output is byte-identical to the previous commit, so no snapshot
moves. Removing the exclusion fails exactly the two rows added here and
nothing else of the 3,323 lib tests.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #1214 +/- ##
==========================================
+ Coverage 98.30% 98.31% +0.01%
==========================================
Files 276 277 +1
Lines 71858 71904 +46
Branches 71428 71474 +46
==========================================
+ Hits 70643 70696 +53
+ Misses 802 796 -6
+ Partials 413 412 -1
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
…decision
Codecov flagged one production line partial on this patch:
`declarator_name`'s `?`, the arm taken when no link on the chain
carries a `parameters` field. Nothing reached it — every row resolved
an owner and was answered by the identifier-kind gate instead.
Two additions, both perturbation-checked:
- `int *f() TF_ATTRIBUTE_NOINLINE { … }` is the shape that reaches it,
and the grammars split two-two on it — not along family lines.
C and **mozcpp** parse it cleanly and name the function; C++ and
Objective-C bury the declarator in an ERROR and answer None. mozcpp
siding with C rather than with the tree-sitter-cpp it forked from is
the part worth pinning: it owns no file extension, so only a unit
test can see it. Written expecting it to behave like C++, and the
test said otherwise.
- A TensorFlow C-API signature every grammar recovers into a
`qualified_identifier` holding a zero-width `::`. It guards the
reason the chain is not gated on a `_declarator` kind suffix, which
is the tidier rule and loses three corpus names: gating it fails
exactly this row, 2 of 48, and nothing else in the suite noticed
before.
The two collect-then-assert tests now share one `assert_all_matched`,
so the failure formatting — unreachable by construction while the suite
is green — exists once rather than three times.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1208
A C/C++/Objective-C function whose declared name sits under an extra
declarator layer resolved its
FuncSpace::nametonull. Two spellingswere affected:
get_func_space_namereached the declarator withfirst_occurrence, aleftmost pre-order search. When the function node's own
declaratorfield is already a
function_declarator— the return type's — thesearch stops there, and its
child(0)is aparenthesized_declaratoror an inner
function_declarator, neither of which is an identifierkind, so the arm fell through to
None.nargshas walked this chain correctly since #1200. That walk moves tosrc/c_declarator.rs, and the four getters now read the name off thesame innermost declarator, so the arity and the name can no longer
describe two different nodes.
What changed
src/c_declarator.rs(new) —innermost_declarator, moved out ofsrc/metrics/nargs.rswith itsnode-types.jsontable doc intact,plus
declarator_name, which is that node'sdeclaratorfield. Thethree
NArgs::params_ownerimpls and the Objective-Ccomputeoverride point at the new path; no arity moved.
declarator_name::<Self>(node)and gate theresult with
matches!on their own identifier kinds. TheFunctionDeclarator/2/3/4enumerations are gone(
grammar-dispatch.md§1) and the name comes off thedeclaratorfield rather than
child(0)(§3). C++/mozcpp keep theOperatorCastearly return: the shared walk deliberately cuts the chain at a
conversion operator, so removing it would return
None.C++ name forms, and
template_function/template_methodputtheir
template_argument_listlast, so a type argument spelling afunction type pulled the chain off the name side.
template <> void tspec<int (*)(int x, int y)>(int a)lost its nameon this branch and had been misreading its arity since fix(nargs): C/C++ pointer/reference-return functions report nargs = 0 #1200.
template_argument_listis now excluded alongsideattribute_declaration.Search::first_occurrenceloses its last production caller andtrips
dead_codeon the lib target, so it sits behind a scoped#[allow]naming refactor(node): Search::first_occurrence has no production caller #1212. Bothsrc/node.rstest docs that justifiedthe pre-order contract by naming these four getters were rewritten —
that claim is now false.
Measurements
Same method both sides (
bca metrics --no-configoverDeepSpeech+pdf.js, 14,269 files, 232,198 spaces):kind: functionspacesThe issue's 354 is the nameless-function population, not this
defect's. This fixes 44 (46 gained a name, 2 lost one). The residual
310 is ERROR-recovery on input the grammars cannot parse:
schema_generated.h(115, a macro between a struct name and its baseclause), two SWIG
.ifiles (56),gpu_device.cc(32). The clustersthe issue named as this defect — the JNI shims — are all fixed:
object_tracker_jni.cc(18),imageutils_jni.cc(5),run_stats_jni.cc(4), plus 17 XLA files.The 2 losses are both inside recovery subtrees, where no strategy has a
defensible answer; one had been reporting the callee of a misparsed
ifstatement as a function name. The boundary is documented on
innermost_declarator.Snapshots
Two openfst
union-weight.hsnapshots move, where the issue predictednone. Not a nameless space — a name→name change. tree-sitter-cpp cannot
parse
friend bool operator== <>(const UnionWeight<W, O> &, …)andfolds the following comments and constructor into a
reference_declaratorholding two siblingfunction_declarators; theold name came from the leftmost, the new one from the last. That space's
function_argshas read1since #1200 — the arity offirst_(W::NoWeight()), not ofUnionWeight()— so the name and thearity had been describing two different nodes and now describe one.
Submodule bumped in the same parent commit and pushed.
Tests
A table in
src/c_declarator.rsover C / C++ / Mozcpp / Objective-Casserting name and
kind: functionand span in one pass(discharging
grammar-dispatch.md§6): the three #1208 shapes, threecontrols including the array-return
int (*g(void))[4]that resolvedcorrectly before the fix, one row expecting no name, and eight C++
name forms. Mozcpp owns no file extension, so it reaches all of this
only through
space_verbatim.Reverting the getters fails 12 of 36 rows — exactly the three #1208
shapes × four languages. Removing the
template_argument_listexclusion fails exactly the two rows added for it.
a_parenthesised_declarator_still_loses_its_space_name, the #1200bug-lock, is deleted: its
fpfixture andplaincontrol are rows inthe new table, in the module that now owns the behaviour.
Downstream note
Affected C-family functions re-key in
.bca-baseline.tomlfrom theline-dependent
<anon@L…>to their real qualified name, so a baselineholding such an entry needs one refresh — after which the key is stable
across line drift like any other named function.
bca functionsstopsrendering these as a red
error:line.Follow-ups
Search::first_occurrencehas no production caller.nargsreadsRUN_STATS_METHOD(allocate)(env, clazz)as one argument; this makesthe name agree with it, so the two surfaces are consistent. Whether
they are consistent about the right node is open.
make pre-commit→BCA_GATE: pass.