fix(release): stop swallowing failed artifact regeneration in the commit step - #174
Merged
Merged
Conversation
…mit step
The \"Commit regenerated artifacts to a review branch\" step ran a single
monolithic \`git add file1 file2 ... file20 || true\`. Standard git
behavior: \`git add\` aborts with a fatal error and stages NOTHING (not
just the missing pathspec -- the whole invocation) if even one listed
pathspec doesn't exist on disk. Six of the earlier generator steps in this
same job (iso26262/iec61508/do178/iec62443/coverage/report gap+compliance
reports) are themselves \`|| true\`-guarded because their cpfusa subcommand
"exits non-zero by design" against certification thresholds -- so their
output files can legitimately be missing if the subcommand crashes before
writing. The trailing bare \`|| true\` on the git-add call (the only \`||
true\` in this file with no inline comment justifying it) silently
swallowed that fatal error, meaning:
guarded generator crashes before writing its file
-> git add hits the missing pathspec, stages NOTHING at all
-> || true swallows the fatal error
-> git diff --cached --quiet sees no staged changes
-> "No artifact changes -- nothing to commit", ARTIFACTS_CHANGED=false
-> review PR step (gated on ARTIFACTS_CHANGED) is skipped
-> job exits 0 (green), no safety-evidence review PR opened,
even if other unrelated artifacts (fmea.json, safety-case.json,
sbom.json, etc.) genuinely changed and were silently dropped too.
Fix: replaced the single git-add call with two passes.
1. The 7 files from `|| true`-guarded generator steps
(iso26262-gap-report.json, iec61508-gap-report.json,
do178-gap-report.json, iec62443-gap-report.json,
coverage-report.json, report.json, report.html) are added one at a
time, gated on `[ -f "$f" ]`, logging a `::warning::` and skipping
cleanly (not aborting the rest) when legitimately absent.
2. Every other artifact (fmea.*, safety-case.*, tooling/tara.*, sbom.json,
provenance.json, artifact-manifest.json, sas.*, sci.json,
audit-pack.zip, fusa-badge.svg, qualify-report.json) -- none of whose
generator steps are `|| true`-guarded -- is added via the original
single git-add call, now with NO trailing `|| true`. If one of these
is unexpectedly missing, git add's fatal pathspec error now propagates
(GitHub Actions runs bash steps with `-eo pipefail` by default) and
fails the job loudly instead of reporting a false green.
Verification:
- `python3 -c "import yaml; yaml.safe_load(open('.github/workflows/release.yml'))"` parses cleanly.
- act/yamllint not available in this environment; instead traced the new
logic by hand and reproduced it standalone (see below).
- Standalone repro in a scratch git repo (not committed) extracting just
the new git-add logic, run against three scenarios:
(a) all 22 files present -> all 22 staged, identical to prior
behavior, exit 0.
(b) coverage-report.json (guarded-generator) missing, fmea.json
genuinely changed -> coverage-report.json skipped with a
::warning::, all other 21 files (including the real fmea.json
change) still staged, exit 0 -- no longer silently reports
"nothing to commit".
(c) sbom.json (non-guarded, required) unexpectedly missing ->
`git add` fails with "fatal: pathspec 'sbom.json' did not match
any files", script exits 128 immediately -- step now fails loudly
instead of a false green.
- Confirmed via grep that this step's name/ID and the ARTIFACTS_CHANGED/
ARTIFACTS_BRANCH env vars it sets are referenced only within this same
job in release.yml; nothing in ci.yml or any other workflow references
them, so no downstream breakage from this change.
Closes a finding from the cpp-RCP v3.0.0 deep audit (batch 3).
Signed-off-by: Matt <47545907+SoundMatt@users.noreply.github.com>
17 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bug
.github/workflows/release.yml's "Commit regenerated artifacts to a review branch" step ran a single monolithic:Standard git behavior:
git add existing.txt nonexistent.txtexits 128 and stages zero files, not just the missing one. Six of the earlier generator steps in the same job (iso26262,iec61508,do178,iec62443,coverage,report) are themselves|| true-guarded because their cpfusa subcommand "exits non-zero by design" against certification thresholds -- so their output files can legitimately be absent if the subcommand crashes before writing. The trailing bare|| trueon thegit addcall (the only|| truein this file with no inline comment justifying it, unlike every other one) silently swallowed that fatal error.Concrete failure chain:
git addhits the missing pathspec and stages nothing at all.|| trueswallows the fatalgit adderror.git diff --cached --quiet(next step) sees no staged changes.ARTIFACTS_CHANGED=false.ARTIFACTS_CHANGED == 'true') is skipped.fmea.json,safety-case.json,sbom.json, etc.) may have genuinely changed and were silently dropped too, since the wholegit addcall staged nothing.Fix
Replaced the single
git add <20 files> || truewith two passes:|| true-guarded generator steps (iso26262-gap-report.json,iec61508-gap-report.json,do178-gap-report.json,iec62443-gap-report.json,coverage-report.json,report.json,report.html) are added one at a time, gated on[ -f "$f" ]. A missing one logs a::warning::and is skipped cleanly, without aborting the rest of the staging operation.fmea.*,safety-case.*,tooling/tara.*,sbom.json,provenance.json,artifact-manifest.json,sas.*,sci.json,audit-pack.zip,fusa-badge.svg,qualify-report.json) -- none of whose generator steps are|| true-guarded -- is added via the original singlegit addcall, now with no trailing|| true. If one of these is unexpectedly missing,git add's fatal pathspec error now propagates (GitHub Actions runsrun:steps withbash -eo pipefailby default) and fails the job loudly instead of reporting a false green.Added inline comments explaining the distinction, mirroring this file's existing convention of commenting every
|| true.Scope is strictly this one step; no other job/step in the workflow was touched.
Verification
CI workflow YAML, so it can't be exercised by "running the test suite" -- and
release.ymlonly triggers onpush: tags: - "v*", so this PR's own CI run will not exercise this job at all. The evidence below is a standalone substitute.YAML syntax:
python3 -c "import yaml; yaml.safe_load(open('.github/workflows/release.yml'))"parses cleanly.act/yamllintwere not available in this environment.Manual trace of both scenarios:
git added exactly as before -- no regression to the working case.[ -f "$f" ]is false for it, so it's skipped with a::warning::and the loop continues; the rest of the optional files and all required files still get added normally. If any real file changed,git diff --cached --quietnow correctly sees staged changes instead of silently reporting "nothing to commit".Standalone shell reproduction (scratch git repo in
/tmp, not committed) extracting just the new git-add logic, run against three scenarios:coverage-report.json(guarded-generator) missing,fmea.jsongenuinely changed -> output:fmea.jsonchange) still staged, exit 0 -- no longer silently reports "nothing to commit".sbom.json(non-guarded, required) unexpectedly missing -> output:Downstream reference check:
grep -rn "Commit regenerated artifacts\|ARTIFACTS_CHANGED\|ARTIFACTS_BRANCH\|regenerate-safety-artifacts" .github/workflows/shows this step's name/ID and theARTIFACTS_CHANGED/ARTIFACTS_BRANCHenv vars it sets are referenced only within this same job inrelease.yml. Nothing inci.ymlor any other workflow file references them, so there's no downstream breakage from this change.Closes a finding from the cpp-RCP v3.0.0 deep audit (batch 3).