Skip to content

fix: store the npm audit fixture under non-manifest names - #328

Merged
HemSoft merged 2 commits into
mainfrom
fix/hide-npm-audit-fixture-from-dependency-graph
Nov 7, 2026
Merged

HemSoft merged 2 commits into
mainfrom
fix/hide-npm-audit-fixture-from-dependency-graph

Conversation

@HemSoft

@HemSoft HemSoft commented Nov 7, 2026

Copy link
Copy Markdown
Collaborator

Summary

Dependabot alert 1 (low, GHSA-238p-pmpm-9mq7, katex) comes from scripts/fixtures/npm-vulnerable-markdown/package-lock.json. That fixture is vulnerable on purpose: scripts/tests/npm-audit.test.cjs copies it to a temporary folder and requires npm audit to fail on exactly that KaTeX advisory. So bumping it would break the test.

This PR keeps the fixture's content byte-identical and renames both files to package.json.fixture and package-lock.json.fixture. The test now copies them to their real names in its temporary folder, so the audit still reads the same vulnerable lock. GitHub's dependency graph indexes only real manifest names, and Dependabot alerts come from that graph. dependabot.yml options configure update PRs, not alerts (About the dependabot.yml file), so the rename is the way to keep the fixture out of the alert list.

docs/RUST_DEPENDENCY_SECURITY.md now describes the .fixture suffix and asks future vulnerable npm fixtures to use it.

Validation

  • Before: the GraphQL dependencyGraphManifests listing on main (6493636) includes both scripts/fixtures/npm-vulnerable-markdown/package.json and package-lock.json.
  • npm run test:npm-audit on Home: 3 pass, 0 fail. The vulnerable-lock test still gets the KaTeX GHSA-238p-pmpm-9mq7 low/direct finding and a failing audit, and the root lock audit is clean.
  • The renames are 100% similarity (no content change).
  • markdownlint on the changed doc: clean. git diff --check: clean.
  • Pre-commit hook (fmt, clippy, full cargo test): passed. No Rust or C# file changed.

After merge, I will confirm that the graph no longer lists the fixture and that alert 1 closes. If it doesn't close by itself, I'll dismiss it with a comment citing this PR.

🤖 Generated with Claude Code

The deliberately vulnerable KaTeX 0.18.1 fixture used by
scripts/tests/npm-audit.test.cjs raised Dependabot alert 1, because
GitHub's dependency graph reads every package.json and package-lock.json
in the tree. Store both files with a .fixture suffix and have the test
copy them to their real names in its temporary folder, so the audit
still sees the vulnerable lock while the graph no longer does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@HemSoft HemSoft added the model:claude-opus-5-5 PR authored by claude-opus-5-5 label Nov 7, 2026
@HemSoft

HemSoft commented Nov 7, 2026

Copy link
Copy Markdown
Collaborator Author

@sfl review

@hemsoft-sfl-reviewer hemsoft-sfl-reviewer Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Verdict: PASS • Round 1

Confidence: 9/10

Review done by claude-opus-5-5 (same family: same-family: codex gated: codex Weekly 156% SUS 64 🔴 pause, defer)

The PR renames the deliberately vulnerable npm audit fixture to package.json.fixture and package-lock.json.fixture so GitHub's dependency graph no longer raises Dependabot alert 1. The test now copies those files to their real names in a temporary folder before running the audit. The test change is correct and nothing at the exact head still uses the old paths. The only defect is a low-severity documentation inconsistency, which does not block approval.

🔨 Full-Spectrum Review Summary

Effort: high
Review time: 1m 2s
Token count: Unavailable

Pass 🔴 Critical 🟠 High 🟡 Medium ⚪ Low
🔒 Security 0 0 0 0
✅ Accuracy 0 0 0 0
🧰 Quality & Maintainability 0 0 0 1
Total 0 0 0 1

Key Findings

  1. ⚪ LOW — Quality & Maintainability: Fixture section intro contradicts the new rename-to-clear-alert guidance. docs/RUST_DEPENDENCY_SECURITY.md:138: Observed: the new text at lines 136-141 says the npm fixture is stored with a .fixture suffix so that it no longer raises Dependabot alert 1. The unchanged heading Expected alerts from test fixtures and the intro at lines 129-130 still say both fixtures are expected to alert and must not be rewritten or removed to clear an alert. Trigger: a maintainer reads the section to decide whether the npm fixture should alert, or whether renaming a fixture to clear an alert is allowed. Impact: the two parts of the section give conflicting guidance, and nothing says what should happen to alert 1 once the manifest names are gone. The previous text said it was left open, not dismissed. This is a documentation clarity problem only. It has no runtime effect. (non-blocking)

Coverage

  • Files reviewed:
    • docs/RUST_DEPENDENCY_SECURITY.md
    • scripts/fixtures/npm-vulnerable-markdown/package-lock.json.fixture
    • scripts/fixtures/npm-vulnerable-markdown/package.json.fixture
    • scripts/tests/npm-audit.test.cjs
  • Context read:
    • package.json
    • AGENTS.md
    • .github/workflows/ci.yml
    • .gitattributes
    • .editorconfig
    • .gitleaks.toml
    • .coderabbit.yaml
    • .markdownlintignore
    • .gitignore
    • scripts/Invoke-SecurityScan.ps1
  • Verified:
    • Compare base_commit.sha is 649363626d96a3ab3fafb6aa37e1ee78bc6638a0 and the single commit is a041979a60303ad6ac12d380a283399c9ee4dca5, ahead 1, behind 0
    • PR #328 head and base SHAs match the reserved SHAs
    • Both fixture files are pure renames with 0 additions and 0 deletions
    • The test copies ${file}.fixture to the real package.json and package-lock.json in a mkdtempSync folder, and the audit:npm call uses --prefix on that folder
    • The head tree has no remaining package.json or package-lock.json under scripts/fixtures/npm-vulnerable-markdown
    • Code search and greps of the config, CI and security-scan files found no other reference to the old fixture filenames
    • CI step npm run test:npm-audit still runs the changed test
    • The fixture still pins katex 0.18.1 as a dev dependency, which matches the GHSA assertion in the test
    • The head tree has no .github/dependabot.yml, which matches the doc
  • Not verified:
    • GitHub behavior claims in the doc: that the dependency graph reads only real manifest names, and that dependabot.yml options do not control alerts. These are external platform behavior with no readable implementation at this ref
    • Whether Dependabot alert 1 closes after merge: this depends on live GitHub state, and alert APIs were not queried
    • Runtime result of the npm audit test and markdownlint on the edited doc: execution is forbidden
    • Code search reflects the default-branch index, not the exact head; the tree listing and targeted greps were used to cover the head
  • Tests run: None. Reviewer execution is disabled.

HemSoft full-spectrum review • 3 evidence-based passes • Reviewed head a041979a60303ad6ac12d380a283399c9ee4dca5 • SFL v0

The section heading and intro still said the fixtures were expected to
raise alerts, which contradicted the new .fixture suffix guidance.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@HemSoft

HemSoft commented Nov 7, 2026

Copy link
Copy Markdown
Collaborator Author

@sfl review

@hemsoft-sfl-reviewer hemsoft-sfl-reviewer Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Verdict: PASS • Round 2

Confidence: 9/10

Since round 1

New: 0; still open (repeated): 0; resolved: 1.
Resolved means no matching finding in this review, not independent proof of a fix.

New (0)

  • None.

Still open (repeated) (0)

  • None.

Resolved (1)

  • LOW Fixture section intro contradicts the new rename-to-clear-alert guidance at docs/RUST_DEPENDENCY_SECURITY.md:138

Review done by claude-opus-5-5 (same family: same-family: codex gated: codex Weekly 155% SUS 64 🔴 pause, defer)

The PR renames the deliberately vulnerable npm audit fixture to package.json.fixture and package-lock.json.fixture so GitHub's dependency graph stops raising Dependabot alert 1. The test now copies both files back to their real manifest names in a temp folder before auditing, and the fixture section of RUST_DEPENDENCY_SECURITY.md was rewritten to match. I found no evidenced defect: the rename has no content changes, no other repository files reference the old paths, and the docs agree with the test and the CI wiring.

🔨 Full-Spectrum Review Summary

Effort: high
Review time: 2m 17s
Token count: Unavailable

Pass 🔴 Critical 🟠 High 🟡 Medium ⚪ Low
🔒 Security 0 0 0 0
✅ Accuracy 0 0 0 0
🧰 Quality & Maintainability 0 0 0 0
Total 0 0 0 0

Key Findings

No material defects found.

Coverage

  • Files reviewed:
    • docs/RUST_DEPENDENCY_SECURITY.md
    • scripts/fixtures/npm-vulnerable-markdown/package-lock.json.fixture
    • scripts/fixtures/npm-vulnerable-markdown/package.json.fixture
    • scripts/tests/npm-audit.test.cjs
  • Context read:
    • package.json
    • .github/workflows/ci.yml
    • .gitattributes
    • .editorconfig
    • .markdownlintignore
    • docs/NPM_ADVISORIES.md
    • scripts/tests/fixtures/rust-advisory/README.md
    • scripts/Invoke-RustAdvisoryScan.ps1
    • AGENTS.md
    • README.md
    • TODO.md
  • Verified:
    • Compare base_commit.sha equals 649363626d96a3ab3fafb6aa37e1ee78bc6638a0 and the last commit equals 08440d72e0c34fd9ff45a48c456575a97c7677d4; status ahead by 2, behind by 0
    • Both fixture files are pure renames (0 additions, 0 deletions), so the vulnerable KaTeX 0.18.1 lock content is unchanged
    • The test loop copies ${file}.fixture to the real file name for package.json and package-lock.json, so npm audit --prefix still sees a real manifest pair
    • The byte-identical assertion compares the temp copy before and after the audit, so the source rename does not weaken it
    • Searched all non-crate text files at the head SHA for npm-vulnerable-markdown and .fixture: only the changed test and doc reference them, and no stale old-path links remain
    • .gitattributes has no rule for *.json or *.fixture, so the rename causes no line-ending normalization change
    • Root package.json has no workspaces glob that would have picked up the fixture directory
    • The CI step npm run test:npm-audit invokes the changed test unchanged
    • docs/NPM_ADVISORIES.md qualification-test text (copying the vulnerable lock into a temp directory) remains accurate
    • The revised doc heading and intro no longer say the fixture is expected to raise alerts, which resolves the contradiction the second commit targeted
  • Not verified:
    • The claim that the GitHub dependency graph reads only real manifest names and that dependabot.yml options do not control alerts is GitHub platform behavior; no repository implementation exists to check it against
    • Whether Dependabot alert 1 auto-closes after merge: alert state was not queried and only default-branch behavior applies
    • Live npm registry audit results for the copied fixture, because execution is forbidden
    • Files under crates/ and .claude/skills/gpui* were excluded from the old-path search because they are unrelated Rust and skill references
  • Tests run: None. Reviewer execution is disabled.

HemSoft full-spectrum review • 3 evidence-based passes • Reviewed head 08440d72e0c34fd9ff45a48c456575a97c7677d4 • SFL v0

@HemSoft
HemSoft merged commit 66764b6 into main Nov 7, 2026
7 checks passed
@HemSoft
HemSoft deleted the fix/hide-npm-audit-fixture-from-dependency-graph branch November 7, 2026 19:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

model:claude-opus-5-5 PR authored by claude-opus-5-5

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant