Skip to content

fix(ci): the dependency audit fails loudly when it cannot open its pull request - #1230

Merged
antoinetoussaint-byte merged 1 commit into
mainfrom
fix/dep-audit-cannot-open-its-pull-request
Oct 23, 2026
Merged

antoinetoussaint-byte merged 1 commit into
mainfrom
fix/dep-audit-cannot-open-its-pull-request

Conversation

@antoinetoussaint-byte

@antoinetoussaint-byte antoinetoussaint-byte commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

Dependency audit auto-remediation has failed every day since at least 18 October
— six consecutive runs — and not at the audit. It remediates correctly, passes
module/tools/module-verify.mjs, force-pushes chore/dep-audit-remediation, and then
dies on the next line:

pull request create failed: GraphQL: GitHub Actions is not permitted to create or approve pull requests (createPullRequest)

Both the organisation and this repository have can_approve_pull_request_reviews: false,
so GITHUB_TOKEN cannot open the pull request. combine-deps.yml already documents
this exact constraint and answers it with a DEPS_COMBINE_TOKEN org secret — which is
not set, so that workflow has failed every week since at least 21 September for the
same reason. dep-audit.yml never adopted the token at all.

Why it was worse than a red badge

gh pr create sits before the gh workflow run ci.yml line, so when it failed:

  • the pushed head never got CI, and
  • no pull request existed to make the remediation visible.

Three critical advisories sat correctly remediated on that branch for six days —
unauthenticated RCE in the Next.js Image Optimization API and in use cache, a DoS in
App Router Server Actions, three dompurify bypasses and a @sentry/nextjs middleware
bypass — while Enforce first-party vulnerability gate blocked every open pull request
in the repository. The remediation pull request had to be opened by hand (#1228).

What this changes

  • Adopt DEPS_COMBINE_TOKEN, falling back to GITHUB_TOKEN. An unset secret is
    falsy, so the remediation half keeps working exactly as today without it.
  • A create that cannot succeed records its status instead of aborting the step, so
    the CI dispatch after it still runs and the pushed head always carries evidence.
  • An already-open pull request is reported and the branch updated in place.
  • With no token, print what a maintainer must do, then exit non-zero.

That last point is the one worth reviewing closely, because the obvious choice is wrong.
By the time the create is attempted the working tree holds the remediated lockfiles,
so letting the step pass would let the residual audit gate report a clean audit and the
whole run go green — with an advisory sitting on a branch nobody is looking at. That is
precisely the silence this job exists to prevent. The residual audit gate itself is
untouched.

The contract test caught my first attempt

My first version moved the dispatch before the create and wrapped it in || true.
scripts/ci/release-gates.test.mjs refused it, and it was right to: that test executes
the real step body against stubs and pins both the dispatch-after-create ordering and
the propagation of the dispatch's own exit status. Reordering was the lazy fix.

The restructure here satisfies every assertion that existed — verified by running the
test with HAVE_PR_TOKEN=true against the unmodified test file, where it passes
untouched. What the test never had was a token dimension, which is exactly where
this failure lived: with gh pr create stubbed to always succeed, no amount of
iteration over existing × changed × dispatchStatus could see it. So the test gains
that dimension and asserts a refused create still pushes, still dispatches CI exactly
once, still fails the run, and names DEPS_COMBINE_TOKEN as the remedy. Nothing was
relaxed to make room for it: 125 pass, 0 fail.

One honest trade: the dispatch stays unconditional, so with a PAT configured the push
raises its own workflow event and the dispatch fires, and that head may get two CI
runs. A head with two runs beats a head with none, and narrowing it would break the
pinned ordering contract for a cosmetic gain.

What this does not fix

The token still has to exist. Minting it is a credential decision for a maintainer: a
PAT or GitHub App token with repo scope, added as an organisation secret, shared
with combine-deps.yml. Until then this job still fails when there is something to
remediate — but now it fails having pushed the branch, triggered its CI, and printed the
remedy, instead of dying on a raw GraphQL error.

Turning on "Allow GitHub Actions to create and approve pull requests" would also work
and needs no secret, but it grants every workflow in scope the ability to approve
pull requests as well, which is a review-bypass path. combine-deps.yml already chose
the token over that trade; this follows it rather than reopening it.

Verified

  • node --test scripts/ci/release-gates.test.mjs scripts/ci/ci-performance.test.mjs:
    125 pass, 0 fail, run with env -u HAVE_PR_TOKEN so no value leaked in from the
    shell that wrote it.
  • The pre-existing assertions pass against the unmodified test file when
    HAVE_PR_TOKEN=true, which is what shows the restructure preserves the old contract
    rather than bending it.
  • The workflow parses, and the rendered step body was read back from the parsed YAML
    rather than from the diff.
  • module/tools/naming-gate.mjs message passes on the commit message and on this body.

Not verified: the no-PAT path end to end on a real advisory — that needs a day when
audit fix produces a diff. The paths that run today are the early git diff --quiet
exit (main is clean once #1228 lands) and the already-open-pull-request exit.

🤖 Generated with Claude Code

…ll request

The job remediated correctly and force-pushed its branch, then died on
`gh pr create` with "GitHub Actions is not permitted to create or approve pull
requests". Because that call aborted the step, the `gh workflow run` line after
it never ran, so the pushed head got no CI either, and no pull request existed
to make the remediation visible. Six consecutive daily runs failed that way
while three critical advisories sat remediated on the branch; the pull request
had to be opened by hand (#1228).

Adopt the DEPS_COMBINE_TOKEN that combine-deps.yml already documents for this
exact constraint, falling back to GITHUB_TOKEN so the remediation half keeps
working without it. A create that cannot succeed now records its status instead
of aborting, prints what a maintainer must do, and lets the CI dispatch run
before the step exits non-zero.

The failure is deliberate, and the obvious alternative is wrong: by then the
working tree holds the remediated lockfiles, so a passing step would let the
residual audit gate report a clean audit and take the whole run green with an
advisory on a branch nobody is looking at.

The release-gate contract test pins the dispatch to run after a new or updated
pull request and the dispatch's status to propagate; both still hold. It gains
the token dimension it never had — the one this failure lived in — asserting
that a refused create still pushes, still dispatches, still fails, and names
DEPS_COMBINE_TOKEN as the remedy.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@antoinetoussaint-byte
antoinetoussaint-byte force-pushed the fix/dep-audit-cannot-open-its-pull-request branch from 4c46172 to 5cc34ac Compare October 23, 2026 15:48
@antoinetoussaint-byte antoinetoussaint-byte changed the title fix(ci): the dependency audit pushes and triggers CI before it can be refused fix(ci): the dependency audit fails loudly when it cannot open its pull request Oct 23, 2026
@antoinetoussaint-byte
antoinetoussaint-byte added this pull request to the merge queue Oct 23, 2026
Merged via the queue into main with commit 303ff46 Oct 23, 2026
25 checks passed
@antoinetoussaint-byte
antoinetoussaint-byte deleted the fix/dep-audit-cannot-open-its-pull-request branch October 25, 2026 17:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant