Skip to content

Multi-source --architecture: one code path for one or many sources, and run sources in parallel #73

Open

Description

@inth3shadows

Two cleanups found while reviewing #72 (--source A,B for --architecture). Neither changes any output.

  • One code path for one source and for many. A single --source still has its own branch in cli.go: its own snapshot and SCIP reads, marker suffix, alt-text notes, and its own way of working out whether codegraph and --path are used. A list goes through loadSources and RunMultiArchitectureMode. The read errors and the undrawn-edge-kinds warning exist twice, so a change to source handling has to be made in both places. Treating one source as a list of one would remove the duplication, provided single-source output stays byte-identical (marker ids, image names, warnings without a label prefix). (inferred, from reading cli.go and sources.go at da8780b)
  • Run the sources concurrently. RunMultiArchitectureMode runs each source's pipeline one after another. With --source codegraph,scip, codegraph's symbol enumeration (CLI subprocesses plus an SQLite read) finishes before the in-memory SCIP work starts. Running them in parallel, with results stored by index so the order stays deterministic, would bring wall time down to roughly the slowest source. Warnings would need buffering per source to keep their order. (inferred; not measured)

Edit (2026-11-18), from the second review round:

  • Build codegraph's source in one place. loadSources leaves the codegraph entry's source nil and cli.go fills in a CodegraphSource later. A new caller that uses the loaded source directly would pass nil into the multi-source run. In list mode cli.go also builds a single-source src it never uses. (inferred)

  • Locate the codegraph index once. A multi-source run with codegraph walks up for .codegraph/ once to work out the path prefix and again to read the index. (inferred)

  • Read each source's symbols once. For a snapshot or SCIP source in a list, the vote on whether its paths start at --path or at codegraph's index root calls Symbols() (list and sort every node), and the edge collection calls it again. That is negligible below a few hundred thousand nodes (about 0.9s per call at 500k). (inferred, timing from the review)

  • Set a group edge's weight once. RollupFileEdges sums the weights and then, for multi-source edges, overwrites the sum with the highest per-source total in a second loop. (inferred)

  • One path-containment helper. fileSourcePrefix repeats the filepath.Rel containment check EmbedPathSuffix does, and the "is a regular file" stat check now exists in three places. fileSourcePrefix deliberately skips EvalSymlinks so it matches how the codegraph index is found; a shared helper has to keep that difference. (inferred)

Edit (2026-11-18), from the #76 review:

  • Look up SARIF lines only for the violating pairs. codegraph.ReadSiteLines runs a GROUP BY over every callee edge in the index, and repeats ReadIndexRows' retry and placeholder code, when only a few violating pairs need a line. It runs only with --sarif and at least one violation. (inferred)
  • One SCIP edge rule. scipindex.RefLines restates FileEdges' rule (definitions, imports, locals, namespaces, cross-file only). The pairs agree today (153 on codeshot's own index), but a change to one would leave the other behind; one pass returning edges with their first line would remove the copy. (verified agreement; drift inferred)

Edit (2026-11-18), from the #77 review:

  • Decide blankness from the drawn graph. IsBlankDot now matches both DOT and Mermaid node lines with one regex; a new format or id scheme would need another case or --embed refuses every diagram. The drawn file list (archDrawn) already knows. (inferred)
  • One embed/check tail for both formats. finishMermaid repeats FinishOutput's doc read, ApplyEmbed, blank refusal, stale messages and write. Separating "produce the bytes and the markdown" from one shared embed/check tail would remove about 40 duplicated lines. (inferred)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions