Skip to content

feat(extraction): represent process launches as SPAWNS edges to Process nodes (#158) - #171

Merged
ata-sesli merged 7 commits into
octacity-org:mainfrom
mertcankaratass:feat/spawns-process-nodes-158
Nov 11, 2026
Merged

ata-sesli merged 7 commits into
octacity-org:mainfrom
mertcankaratass:feat/spawns-process-nodes-158

Conversation

@mertcankaratass

Copy link
Copy Markdown
Contributor

What does this PR do?

Closes #158.

Process launches (subprocess.run(["git", ...]), exec.Command("kubectl", ...), child_process.spawn, posix_spawn, Process.Start, Command::new, etc.) were previously either invisible in the graph or resolved to project functions of the same name via guessed CALLS edges.

This PR represents process launches as SPAWNS edges from the calling function to a Process node named after the program, vetoing heuristic call guesses:

  1. Launch API Tables & Program Extraction (internal/cbm/spawn_patterns.h, spawn_patterns.c):

    • Per-language launch API table covering ~30 languages (QUALIFIED, BARE, BARE_IMPORTED, STRING_RECEIVER).
    • Import-aware pattern matching (e.g. import subprocess as sp -> sp.run).
    • Program name extraction: strips directories (/usr/bin/git status -> git), handles literals, and falls back to <dynamic> when not a literal string.
  2. Pipeline Integration (src/pipeline/pass_spawns.c, src/pipeline/pass_parallel.zig):

    • Implemented cbm_pipeline_spawn_site and cbm_pipeline_emit_spawn.
    • Vetoes heuristic name guesses (unique_name, suffix_match, fuzzy, weak same_module) for launch APIs that do not resolve to actual project definitions.
    • Emits Process node (__process__<program>) and SPAWNS edge with call line and confidence.
    • Hooked into pass_parallel.zig call resolution loop (resolveFileCalls).
  3. Build & Test Wiring (Makefile.cbm, build.zig, tests/test_spawns.c, tests/test_main.c):

    • Added spawns test suite covering recognition through imports, bare names, argument extraction rules, graph nodes/edges, and sequential vs parallel parity.
    • Added SPAWNS to ALL_EDGE_TYPES and updated README.md graph data model.

Checklist

  • Every commit is signed off (git commit -s) — required, CI rejects
    unsigned commits (DCO, see CONTRIBUTING.md)
  • Tests pass locally (make -f Makefile.cbm test)
  • Lint passes (make -f Makefile.cbm lint-ci)
  • New behavior is covered by a test (reproduce-first for bug fixes)

…ss nodes (octacity-org#158)

Process launches (subprocess.run, exec.Command, child_process.spawn, posix_spawn, Process.Start, Command::new, etc.) were either omitted from the graph or bound to project functions of the same name via guessed CALLS edges.

Emit SPAWNS edges to Process nodes instead:
- internal/cbm/spawn_patterns: per-language launch API table across ~30 languages and program name extraction rules (literal first argument/word or <dynamic>).
- src/pipeline/pass_spawns: resolve spawn sites and emit Process nodes (__process__<program>) and SPAWNS edges. Veto heuristic name guesses for launch APIs that do not resolve to project code.
- src/pipeline/pass_parallel.zig: integrate spawn site checks into call resolution.
- tests/test_spawns.c: test recognition through imports, bare names, program argument extraction, graph representation, and parallel parity.

Closes octacity-org#158.

Signed-off-by: mertcankaratass <184436995+mertcankaratass@users.noreply.github.com>

# Conflicts:
#	Makefile.cbm
#	build.zig
tests/test_github_workflows.sh validates that every default suite in test_main.c is assigned to exactly one CI test shard in scripts/ci-test-shard.sh.

Signed-off-by: mertcankaratass <184436995+mertcankaratass@users.noreply.github.com>
Remove CBM_LANG_ARKTS, which is not part of this repository's language enum, and drop source_origin field access from CBMCall.

Signed-off-by: mertcankaratass <184436995+mertcankaratass@users.noreply.github.com>
Initialize CBMCall.args as array members rather than pointer assignment, call cbm_mcp_server_new() with no arguments, and use cbm_store_open_path_query().

Signed-off-by: mertcankaratass <184436995+mertcankaratass@users.noreply.github.com>
Signed-off-by: mertcankaratass <184436995+mertcankaratass@users.noreply.github.com>
@mertcankaratass
mertcankaratass force-pushed the feat/spawns-process-nodes-158 branch from 4ce9913 to 449e383 Compare November 11, 2026 13:52

@ata-sesli ata-sesli left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this. The feature is useful, the launch-API tables are thorough, and the veto on name-guess resolutions is the right approach. A few things need to change before merge.

1. Please write the pipeline part in Zig

src/pipeline/ has been fully ported to Zig: PIPELINE_SRCS in Makefile.cbm was empty before this PR, and the last C pieces (lsp_resolve, the sequential resolver) were removed recently. src/pipeline/pass_spawns.c adds C back to that directory, plus new C prototypes in pipeline_internal.h, the PIPELINE_SRCS entry and the native_sources entry in build.zig.

Please move cbm_pipeline_spawn_site / cbm_pipeline_emit_spawn and their helpers into a Zig module (for example src/pipeline/spawns.zig) that pass_parallel.zig imports with @import, and drop pass_spawns.c, its build entries and the pipeline_internal.h additions. src/pipeline/lsp_resolve.zig is a recent example of the pattern. If tests/test_spawns.c needs to call them from C, export them with pub export fn and keep a small header with only those prototypes, as lsp_resolve.h does.

internal/cbm/spawn_patterns.c can stay in C, since internal/cbm (extraction) is still C.

2. The sequential resolver no longer exists

The sequential resolver (pass_calls.c, pass_definitions.c, pass_usages.c) was deleted in faf79b3, and pass_parallel.c is now pass_parallel.zig. Several places still describe two resolvers:

  • the pass_spawns.c header ("Shared by both call resolvers (pass_calls.c, pass_parallel.c)");
  • the pipeline_internal.h comment and the PR description ("sequential vs parallel parity");
  • spawns_parallel_matches_sequential in tests/test_spawns.c. Both runs now go through the same parallel resolver, so this test cannot catch a divergence. Please remove it, or rename it to say what it does check (for example that the result does not depend on the file count).

Also, the pipeline_internal.h comment says the hook runs on "the registry answer after the cross-language veto". In resolveFileCalls the hook runs before the cross-language guard (cbm_suppress_cross_language_suffix_match). Because suffix_match is treated as a guess, I don't think this changes the output, but the comment should match the code.

3. Bare PHP exec likely matches PDO calls (not reproduced)

CBM_SPAWN_BARE compares only the callee text and does not look at whether the call has a receiver. For PHP member calls the extractor records the bare method name (see the comment above the scoped_call_expression case in extract_calls.c: "every $obj->get() would become a route"). So $pdo->exec("DELETE FROM jobs"), a very common PDO call, would likely match the exec entry. It does not resolve to project code, so it would become a SPAWNS edge to a Process node named DELETE. The same may apply to other bare entries called as methods (Ruby obj.exec, Julia run). I found this by reading the code and did not reproduce it.

Suggested fix: skip CBM_SPAWN_BARE entries for calls that have a receiver, and add a test where $pdo->exec("...") produces no SPAWNS edge.

Comment thread src/pipeline/pass_spawns.c Outdated
@@ -0,0 +1,103 @@
/*

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please write this in Zig (see the review summary): a module under src/pipeline/ imported by pass_parallel.zig, instead of C linked through PIPELINE_SRCS and pipeline_internal.h.

Comment thread src/pipeline/pass_spawns.c Outdated
* pass_spawns.c — SPAWNS edges: a call that starts another program, to the
* Process node of that program (internal/cbm/spawn_patterns.h).
*
* Shared by both call resolvers (pass_calls.c, pass_parallel.c), which ask

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pass_calls.c and pass_parallel.c no longer exist. The only call resolver is resolveFileCalls in pass_parallel.zig, and it calls this before the cross-language guard, not after it.

Comment thread tests/test_spawns.c Outdated

/* The sequential (few files) and parallel (past MIN_FILES_FOR_PARALLEL)
* resolvers ask at the same point and emit the same SPAWNS edges. */
TEST(spawns_parallel_matches_sequential) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since faf79b3 there is no sequential resolver, so both runs here use the parallel path and this comparison cannot fail for the reason in its name. Please remove it or rename it to what it does check.

};

static const cbm_spawn_api_t php[] = {
{"exec", NULL, U, 0, -1},

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Likely false positive (not reproduced): PHP member calls are extracted with the bare method name, and CBM_SPAWN_BARE does not check for a receiver, so $pdo->exec("DELETE FROM jobs") would become a SPAWNS edge to a Process named DELETE. Consider skipping bare entries for calls with a receiver, and add a test for the PDO case.

@mertcankaratass
mertcankaratass force-pushed the feat/spawns-process-nodes-158 branch from d6c98b0 to 5d9a6f8 Compare November 11, 2026 15:45
…d calls

- Move cbm_pipeline_spawn_site and cbm_pipeline_emit_spawn into src/pipeline/spawns.zig
- Add src/pipeline/spawns.h with C ABI prototypes and export in src/cbm_zig.zig
- Drop src/pipeline/pass_spawns.c, PIPELINE_SRCS in Makefile.cbm, and native_sources entry
- Update pass_parallel.zig to import spawns.zig directly
- Set call.is_method for PHP member calls and Ruby method calls with receivers
- Skip CBM_SPAWN_BARE and CBM_SPAWN_BARE_IMPORTED when call.is_method is true
- Rename test spawns_parallel_matches_sequential to spawns_result_invariant_to_file_count
- Add PHP regression test ensuring $pdo->exec does not emit a SPAWNS edge

Signed-off-by: mertcankaratass <184436995+mertcankaratass@users.noreply.github.com>
@mertcankaratass
mertcankaratass force-pushed the feat/spawns-process-nodes-158 branch from 5d9a6f8 to efe06ba Compare November 11, 2026 16:02

@ata-sesli ata-sesli left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the quick update. The Zig port is what we asked for: pass_spawns.c is gone, spawns.zig is imported by pass_parallel.zig, the C exports for the tests sit behind a small spawns.h like lsp_resolve, PIPELINE_SRCS is empty again, and checkExports covers the new module. The stale sequential-resolver references are fixed too.

One change from this round needs to come back out, and that is on me.

Correction: the PDO false positive in my previous review did not exist

In point 3 of my previous review I said $pdo->exec("DELETE FROM jobs") would match the bare PHP exec entry. That was wrong. PHP's member_call_expression has an object field, and extract_callee_from_fields joins it to the method name (extract_calls.c, the name + object branch), so the callee text is "$pdo.exec", not "exec". Bare entries only match the exact text, so $pdo->exec(...) never produced a SPAWNS edge, even before this commit. The comment I cited is about Route::get (scoped_call_expression), which has no object field. I misread it. Sorry for the extra work.

The new is_method check disables Ruby launch APIs called with a receiver (found by reading the code, not reproduced)

  • The extractor now sets is_method = true for every Ruby call with a receiver.
  • The Ruby extractor names those calls receiver plus method ("Open3.capture3", "IO.popen"), and the Ruby table lists exactly those spellings as CBM_SPAWN_BARE entries (Kernel.*, Process.*, IO.popen, PTY.spawn, Open3.*).
  • site() now rejects every bare match when is_method is set.

So 13 of the 17 Ruby entries can never match: Open3.capture3("ls"), IO.popen("ls"), Kernel.system("make") and Process.spawn("x") no longer produce SPAWNS edges. Only receiver-less system, exec and spawn still work. No test catches this, because the Ruby test (spawn_match_bare_names) calls cbm_spawn_match directly and never reaches site().

For PHP the change has no effect, since the callee text already includes the receiver. The new $pdo->exec pipeline case passes with or without the fix. No other code reads is_method for PHP or Ruby; the existing readers are gated to Perl, Python/JS/TS and Zig.

Requested changes

  1. Revert the is_method additions in internal/cbm/extract_calls.c, the cbm.h comment change and the new comment above the PHP table in spawn_patterns.c.
  2. Remove the is_method check from site() in spawns.zig. Without the extractor change it does nothing.
  3. Keep the $pdo->exec case in spawns_in_the_graph as a regression guard, and add a Ruby case there (for example Open3.capture3("ls") producing a SPAWNS edge to ls) so receiver-form Ruby calls are covered end to end.
  4. Minor: the failure message in spawns_result_invariant_to_file_count still prints sequential: / parallel:.

Comment thread src/pipeline/spawns.zig Outdated

// A bare launch API (e.g. PHP exec, Ruby exec) called as a method on a receiver
// (such as $pdo->exec("DELETE FROM ...") or obj.exec(...)) is not a process spawn.
if (call.is_method and (api.*.rule == c.CBM_SPAWN_BARE or api.*.rule == c.CBM_SPAWN_BARE_IMPORTED)) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This rejects the Ruby receiver-form entries (Open3.capture3, IO.popen, Kernel.system, Process.spawn, ...). They are CBM_SPAWN_BARE entries whose callee text already includes the receiver, and the extractor now marks those calls is_method. For PHP $pdo->exec the callee text is "$pdo.exec", which never matched exec anyway, so this check can go. See the review summary.

Comment thread internal/cbm/extract_calls.c Outdated
strcmp(ts_node_type(node), "nullsafe_member_call_expression") == 0)) {
call.is_method = true;
}
if (ctx->language == CBM_LANG_RUBY &&

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please revert the PHP and Ruby is_method additions. Ruby: it makes site() drop every receiver-form launch API. PHP: it has no effect, because the callee is already "$obj.method". No other reader of is_method covers PHP or Ruby.

Comment thread tests/test_spawns.c Outdated
sp_cleanup(&p);
ASSERT_TRUE(seq[0] != '\0');
if (strcmp(seq, par) != 0) {
fprintf(stderr, " sequential:\n%s parallel:\n%s", seq, par);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor: the message still says sequential: / parallel:; something like few files: / many files: would match the new test name.

…orm Ruby

- Revert is_method setting for PHP and Ruby in extract_calls.c and comments in cbm.h and spawn_patterns.c
- Drop is_method check from spawns.zig site() so receiver-form Ruby entries (e.g. Open3.capture3) are recognized
- Keep $pdo->exec regression test and add worker.rb with Open3.capture3("ls") to test_spawns.c
- Update file-count invariance test error message to print "few files: / many files:"

Signed-off-by: mertcankaratass <184436995+mertcankaratass@users.noreply.github.com>
@ata-sesli
ata-sesli merged commit c9e4e54 into octacity-org:main Nov 11, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Represent process launches as SPAWNS edges to Process nodes instead of guessed CALLS

2 participants