Skip to content

Backstop against the qa-pro.yml/qa-suite.yml synchronize cost drift - #6

Merged
iamprazol merged 1 commit into
mainfrom
fix/qa-suite-trigger-drift-backstop
Sep 11, 2026
Merged

iamprazol merged 1 commit into
mainfrom
fix/qa-suite-trigger-drift-backstop

Conversation

@iamprazol

Copy link
Copy Markdown
Collaborator

Closes the recurrence half of #5. The immediate drift in user-registration-pro's qa-pro.yml (which re-added synchronize to on.pull_request.types — 15+ full pro-suite runs across two days on release PR #1572, ~$5.4/day in billed Actions minutes) was already hand-fixed in that repo before this PR. This is what stops it recurring, in this or any other product repo.

What changed

  1. A cost backstop in the reusable workflows themselves, not just the callers. suite.yml's suite job and pro-suite.yml's gate job now allow-list pull_request events to opened/reopened/ready_for_review in their job-level if:. Any other action — synchronize included — is a skipped job: $0, no runner started, regardless of what any caller's own on: block says. Centralised so every current and future pro repo is protected without depending on every copied qa-pro.yml/qa-suite.yml staying correct (per the issue's own framing: these are per-repo callers with no central version lock, and two repos had already disagreed on this exact line once).

  2. setup-product.mjs check-workflow — a new read-only action (issue's Option A). Reads an existing caller workflow and flags synchronize in pull_request.types or any use of pull_request_target, so drift can be found on request instead of on a bill. Comment lines are stripped before scanning: a naive substring check flagged the template's own "never use pull_request_target" warning as the violation, caught by testing the tool against the templates themselves before trusting it.

  3. CLAUDE.md's build-state log records the incident and both fixes, per this repo's own convention.

Verification

  • 5 fixtures for check-workflow: clean qa-suite.yml, clean qa-pro.yml, both with synchronize added, and one with pull_request_target — correct verdict and exit code on all five (0 clean, 1 for a real finding, 2 for a missing file).
  • Live pull of all three real pro-repo callers (colormag-pro, user-registration-pro, zakra-pro) via the GitHub API and run through check-workflow — all three report clean: true today.
  • Both edited workflow files parse as valid YAML; claude plugin validate ./plugins/claudegrill still passes with only the expected, intentional "no version" warning.

Not done here

Option D from the issue (surfacing runner-minutes somewhere more visible than the org billing page) — each job already reports its elapsed minutes to the run summary, but nothing rolls that up across runs or repos. Left open as a separate, smaller follow-up if it's still wanted.

🤖 Generated with Claude Code

claudegrill#5: qa-pro.yml in user-registration-pro re-added `synchronize`
to on.pull_request.types, re-running the full pro suite on every push to
one release PR — 15+ full runs in two days, ~$5.4/day in billed Actions
minutes, found only on the org bill. That repo's own file was already
hand-fixed before this commit; this is the fix for it recurring.

- suite.yml's `suite` job and pro-suite.yml's `gate` job now allow-list
  pull_request actions to opened/reopened/ready_for_review in their
  job-level `if:`. Any other action (synchronize included) is a skipped
  job — $0, no runner started — regardless of what a caller's own `on:`
  block says. Centralised in the reusable workflows so every current and
  future pro repo is protected without depending on every copied caller
  file staying correct.

- setup-product.mjs gains `check-workflow`, a read-only action that reads
  an existing qa-pro.yml/qa-suite.yml and flags `synchronize` in
  pull_request.types or any use of pull_request_target. Comment lines are
  stripped before scanning — a naive substring check flagged the
  template's own "never pull_request_target" warning as the violation,
  caught by testing against the templates themselves.

Verified against five fixtures (clean qa-suite.yml, clean qa-pro.yml,
both with synchronize added, one with pull_request_target) and a live
pull of all three real pro-repo callers (colormag-pro,
user-registration-pro, zakra-pro) — all three report clean today.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@iamprazol
iamprazol merged commit 8e6e780 into main Sep 11, 2026
2 checks passed
@iamprazol
iamprazol deleted the fix/qa-suite-trigger-drift-backstop branch September 11, 2026 06:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant