Skip to content

feat: --sarif writes architecture rule violations as SARIF 2.1.0 - #76

Merged
inth3shadows merged 4 commits into
masterfrom
p5-sarif
Nov 19, 2026
Merged

inth3shadows merged 4 commits into
masterfrom
p5-sarif

Conversation

@inth3shadows

@inth3shadows inth3shadows commented Nov 19, 2026 •

Copy link
Copy Markdown
Owner

What

--architecture can now write its rule violations as SARIF 2.1.0, the format GitHub code scanning reads:

codeshot --architecture --path . --rules .codeshot.yaml --sarif codeshot.sarif --format svg --out arch.svg
- uses: github/codeql-action/upload-sarif@v3
  if: always()
  with:
    sarif_file: codeshot.sarif
    category: codeshot
  • Every --forbid and --rules violation becomes one error result, with the same message as the stderr report (including the reason, the layer names and which sources saw the edge).
  • The result sits in the depending file, at the first line where it calls or refers to the other file, so the alert lands on the pull request line that added the dependency. The depended-on file is a related location.
  • The fingerprint (primaryLocationLineHash, the key GitHub tracks alerts by) comes from the rule and the two files, not the line, so moving the call doesn't open a new alert.
  • The file is written on every run: with an empty results list when nothing breaks a rule (which closes fixed alerts), and with --check. Exit codes don't change. --sarif needs a rule to check, and refuses a scan --max-symbols cut.

This is P5a of the program.

How

  • Line numbers are looked up only for violations, after the rules ran, through an optional SiteLiner interface on each source. The code that builds the diagram is untouched.
    • codegraph: one read-only query over the index, using the same edge kinds --architecture draws (imports and file targets aside). It takes the smallest edge line, or the calling function's start line when codegraph stored none for that edge.
    • snapshot: the smallest line on a drawn-kind edge between the two files (the field is optional in snapshot v1).
    • SCIP: the first non-import, non-definition reference in the depending file to a symbol the other file defines.
    • Multi-source: the smallest line any source gives; rebased paths are mapped back.
    • No line known: line 1 of the file.
  • Two SARIF rules: codeshot/forbid (flags and file forbid rules) and codeshot/layers.
  • No new dependency: encoding/json and crypto/sha256.
  • Dropped: file-level results only (simpler, but they would almost never land on a diff line); threading line numbers through the file-edge code (touches every diagram run for a lookup only violations need); a SARIF library.

Measured (base 0bf0de4 vs edda666, Linux, Go 1.26.5)

  • Schema (verified): every output validates against the SARIF 2.1.0 JSON schema from schemastore, with 0 errors. Checked with jsonschema 4.26.0, outside the test suite. The runs, on this repo with the core and sources layers swapped so there are violations:
    • codegraph (fork v1.6.2-em.44): 34 results, all with a real line.
    • scip-go 0.2.8 index alone: 38 results, all with a real line.
    • codegraph + scip-go: 45 results, all with a real line.
    • With the real .codeshot.yaml: 0 results.
  • Lines spot-checked by hand: internal/codeshot/diff_test.go:67 is that file's first use of a codegraph type (*codegraph.Response). The SCIP index's diff_test.go:14 is x.Name, a field of codegraph's Node.
  • Bug found on the real index, fixed before this PR: the first SCIP lookup read only SCIP's deprecated packed range field, and scip-go 0.2.8 writes only the typed one, so every SCIP-only result fell back to line 1. It now reads both through the bindings' SourceRange, with a regression test for the typed field.
  • Without --sarif (verified): 5 runs (no rules, the real rules, failing rules, --forbid with --group-depth, symbol mode) give identical DOT, stderr and exit code on base and this branch.
  • New tests: 9. They cover the encoder, an empty log, fingerprints that ignore the line, the codegraph and SCIP line lookups (including the typed range), snapshot lines, merging lines across sources with rebased paths, an end-to-end snapshot run (violation at its call-site line, then a clean run with an empty log), and --sarif refused outside --architecture.
  • Full suite: go test ./... ok; golangci-lint 0 issues.; gofmt and go vet clean; both diagram --check runs exit 0 (fresh clone, fork v1.6.2-em.44).

Changes since review (2026-11-18)

Round 1 (/code-review + a code-reviewer agent on 0384dab; the agent read GitHub's SARIF docs and the codeql-action fingerprint code), fixed in fe1bca3:

  • Fingerprints GitHub actually uses. GitHub tracks alerts by primaryLocationLineHash and computes it from the line's text when it's missing. The first version put codeshot's hash under its own key, which GitHub ignores, and two different rules on one edge (a --forbid flag and the same rule in the file) shared it. codeshot's hash, which now includes the rule's wording, goes in primaryLocationLineHash (upload-sarif keeps a given one), and an exact duplicate result is written once.
  • Encoded URIs. A path with a space, #, % or a non-ASCII name was written raw, so a URI parser read part of it as a fragment and codeql-action's decoder turned %20 into a space. Each segment is now percent-encoded, and an absolute path is a file:// URI.
  • Better lines. A real call line now beats the calling function's start line in the same file pair, for codegraph and snapshots alike. The start line is used only when no edge in the pair has a line, and 0 counts as no line.
  • No accidental alert wipe. --sarif without --forbid or --rules is refused: it would upload an empty log and close every open alert.
  • Rules carry help text, and the no-lines warning names the source and says plainly when an index predates the line columns.
  • Filed to Multi-source --architecture: one code path for one or many sources, and run sources in parallel #73: the full-index line query and SCIP's line rule duplicating its edge rule (the agent found the pairs agree today: 180 for codegraph, 153 for SCIP).
  • Real runs re-validated against the SARIF 2.1.0 schema, 0 errors: codegraph 34, scip-go 38 and both together 45 results, all with a real line and unique fingerprints; a clean run writes 0 results. Without --sarif, 5 runs match master. go test ./... ok; golangci-lint v2.15.0 0 issues.

Round 2 (verify on fe1bca3): the round-1 fixes held (13 runs without --sarif matched master; 11 outputs validated). Fixed in 17b1e69:

  • Stable fingerprints. The round-1 key included the --rules path as typed and the rule's reason, so --rules ./r.yaml instead of --rules r.yaml, or a reworded reason, reopened every alert. It is now the rule id, the rule and the two files.
  • A cut scan writes no SARIF. With --sarif, a scan that --max-symbols cut exits 1 and writes nothing, naming the value to rerun with: a partial log would close the alerts of violations past the cut. (Before: exit 0 and a partial log.)
  • A rules file that checks nothing (no forbid rule, fewer than two layers) is refused with --sarif, like no rules at all.
  • URIs: a : in a relative path's first segment is %3A (it read as a scheme), and a Windows drive path is a file:///C:/ URI.
  • Lines: exact call lines beat start-line fallbacks across sources too; an index without edge line numbers still gives start lines; the multi-source warning only covers that source's results.
  • upload-sarif logs one warning per result that codeshot's fingerprint differs from the line-text one it would compute; it keeps codeshot's (noted in USAGE).
  • go test ./... ok; golangci-lint v2.15.0 0 issues.; 5 outputs validate with 0 errors (all results lined, unique fingerprints); without --sarif, 5 runs match master.

Known limits

  • Not uploaded to GitHub yet. How code scanning shows a result at line 1 (a file-level finding) is assumed, not checked.
  • Paths are the diagram's. They match what GitHub checked out when codeshot runs at the repository root (--path .); with a snapshot or SCIP index whose paths start elsewhere, results point at the wrong file.
  • Only the first call site per file pair is reported, not every one.
  • This repo's own CI doesn't upload SARIF. That needs security-events: write on the diagrams job and is a separate decision.
  • Not for --check drift, symbol mode or --diff.

`codeshot --architecture ... --sarif codeshot.sarif` writes every
--forbid and --rules violation as a SARIF result, for GitHub code
scanning: the depending file at its first call-site line (from the
codegraph index, a snapshot edge's line, or the SCIP index's first
reference; line 1 when unknown), the depended-on file as a related
location, and a fingerprint from the rule and the two files so an alert
survives the line moving. Written on every run, with an empty results
list when nothing breaks a rule, and with --check. Line numbers are
looked up only for violations, so the diagram path is unchanged.

P5a of the program.
golangci-lint v2.15.0 (CI) flags SA1019 on the test that covers indexers
writing the deprecated packed range; that coverage is on purpose.
- F1: codeshot's hash goes in primaryLocationLineHash, the key GitHub
  tracks alerts by (upload-sarif keeps a given one); it includes the
  rule's wording, and an exact duplicate result is written once.
- F2: artifact URIs are percent-encoded per segment; absolute paths are
  file:// URIs.
- F3/F4: a real call line wins over the caller's start line, for
  codegraph and snapshots alike; 0 is no line.
- F6: --sarif needs --forbid or --rules (an empty log would close every
  open alert).
- A3/A4/F9: rules carry help text; the no-lines warning names the source
  and an old index plainly; one smallest-line helper.
- V1: the fingerprint is the rule id, the rule and the two files; the
  --rules path as typed and the reason no longer reopen alerts.
- V2: a --sarif run whose scan --max-symbols cut writes nothing and exits
  1, since a partial log would close the alerts of violations past it.
- V3: --sarif refuses a rules file that checks nothing.
- V5: a ':' in a relative path's first segment is %3A; a Windows drive
  path is a file:///C:/ URI.
- V6/V10/V11: an index without edge lines still gives start lines; the
  multi-source warning is about that source's results only; exact call
  lines beat start-line fallbacks across sources.
@inth3shadows
inth3shadows merged commit 33a034b into master Nov 19, 2026
5 checks passed
@inth3shadows
inth3shadows deleted the p5-sarif branch November 19, 2026 02:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant