Skip to content

fix(release): stop swallowing failed artifact regeneration in the commit step - #174

Merged
SoundMatt merged 1 commit into
mainfrom
audit-fix/batch3-release-artifact-commit
Aug 22, 2026
Merged

fix(release): stop swallowing failed artifact regeneration in the commit step#174
SoundMatt merged 1 commit into
mainfrom
audit-fix/batch3-release-artifact-commit

Conversation

@SoundMatt

Copy link
Copy Markdown
Owner

Bug

.github/workflows/release.yml's "Commit regenerated artifacts to a review branch" step ran a single monolithic:

git add \
  fmea.csv fmea.json \
  safety-case.json safety-case.md safety-case.mermaid \
  tooling/tara.json tooling/tara.md \
  sbom.json provenance.json artifact-manifest.json \
  iso26262-gap-report.json \
  iec61508-gap-report.json \
  do178-gap-report.json \
  iec62443-gap-report.json \
  coverage-report.json \
  sas.json sas.md \
  sci.json \
  audit-pack.zip \
  fusa-badge.svg \
  report.json report.html \
  qualify-report.json \
  || true

Standard git behavior: git add existing.txt nonexistent.txt exits 128 and stages zero files, not just the missing one. Six of the earlier generator steps in the same job (iso26262, iec61508, do178, iec62443, coverage, report) are themselves || true-guarded because their cpfusa subcommand "exits non-zero by design" against certification thresholds -- so their output files can legitimately be absent if the subcommand crashes before writing. The trailing bare || true on the git add call (the only || true in this file with no inline comment justifying it, unlike every other one) silently swallowed that fatal error.

Concrete failure chain:

  1. A guarded generator step crashes before writing its file.
  2. git add hits the missing pathspec and stages nothing at all.
  3. || true swallows the fatal git add error.
  4. git diff --cached --quiet (next step) sees no staged changes.
  5. Prints "No artifact changes -- nothing to commit", sets ARTIFACTS_CHANGED=false.
  6. The "Open review PR" step (gated on ARTIFACTS_CHANGED == 'true') is skipped.
  7. Job exits 0 (green) with no safety-evidence review PR opened at all, even though unrelated, non-guarded artifacts (fmea.json, safety-case.json, sbom.json, etc.) may have genuinely changed and were silently dropped too, since the whole git add call staged nothing.

Fix

Replaced the single git add <20 files> || true with two passes:

  1. The 7 files produced by || true-guarded generator steps (iso26262-gap-report.json, iec61508-gap-report.json, do178-gap-report.json, iec62443-gap-report.json, coverage-report.json, report.json, report.html) are added one at a time, gated on [ -f "$f" ]. A missing one logs a ::warning:: and is skipped cleanly, without aborting the rest of the staging operation.
  2. Every other artifact (fmea.*, safety-case.*, tooling/tara.*, sbom.json, provenance.json, artifact-manifest.json, sas.*, sci.json, audit-pack.zip, fusa-badge.svg, qualify-report.json) -- none of whose generator steps are || true-guarded -- is added via the original single git add call, now with no trailing || true. If one of these is unexpectedly missing, git add's fatal pathspec error now propagates (GitHub Actions runs run: steps with bash -eo pipefail by default) and fails the job loudly instead of reporting a false green.

Added inline comments explaining the distinction, mirroring this file's existing convention of commenting every || true.

Scope is strictly this one step; no other job/step in the workflow was touched.

Verification

CI workflow YAML, so it can't be exercised by "running the test suite" -- and release.yml only triggers on push: tags: - "v*", so this PR's own CI run will not exercise this job at all. The evidence below is a standalone substitute.

  1. YAML syntax: python3 -c "import yaml; yaml.safe_load(open('.github/workflows/release.yml'))" parses cleanly. act/yamllint were not available in this environment.

  2. Manual trace of both scenarios:

    • (a) all files present: both loops execute, all 22 files get git added exactly as before -- no regression to the working case.
    • (b) one of the 7 guarded-generator files missing: [ -f "$f" ] is false for it, so it's skipped with a ::warning:: and the loop continues; the rest of the optional files and all required files still get added normally. If any real file changed, git diff --cached --quiet now correctly sees staged changes instead of silently reporting "nothing to commit".
  3. Standalone shell reproduction (scratch git repo in /tmp, not committed) extracting just the new git-add logic, run against three scenarios:

    • (a) all 22 files present -> all 22 staged, identical file list to the original behavior, exit 0.
    • (b) coverage-report.json (guarded-generator) missing, fmea.json genuinely changed -> output:
      ::warning::coverage-report.json not present -- its generator step is || true-guarded (exits non-zero by design), treating as legitimately absent and skipping.
      Staged changes:
      artifact-manifest.json
      audit-pack.zip
      do178-gap-report.json
      fmea.csv
      fmea.json
      ...
      
      All other 21 files (including the real fmea.json change) still staged, exit 0 -- no longer silently reports "nothing to commit".
    • (c) sbom.json (non-guarded, required) unexpectedly missing -> output:
      fatal: pathspec 'sbom.json' did not match any files
      
      script exits 128 immediately -- the step now fails loudly instead of a false green.
  4. Downstream reference check: grep -rn "Commit regenerated artifacts\|ARTIFACTS_CHANGED\|ARTIFACTS_BRANCH\|regenerate-safety-artifacts" .github/workflows/ shows this step's name/ID and the ARTIFACTS_CHANGED/ARTIFACTS_BRANCH env vars it sets are referenced only within this same job in release.yml. Nothing in ci.yml or any other workflow file references them, so there's no downstream breakage from this change.

Closes a finding from the cpp-RCP v3.0.0 deep audit (batch 3).

…mit step

The \"Commit regenerated artifacts to a review branch\" step ran a single
monolithic \`git add file1 file2 ... file20 || true\`. Standard git
behavior: \`git add\` aborts with a fatal error and stages NOTHING (not
just the missing pathspec -- the whole invocation) if even one listed
pathspec doesn't exist on disk. Six of the earlier generator steps in this
same job (iso26262/iec61508/do178/iec62443/coverage/report gap+compliance
reports) are themselves \`|| true\`-guarded because their cpfusa subcommand
"exits non-zero by design" against certification thresholds -- so their
output files can legitimately be missing if the subcommand crashes before
writing. The trailing bare \`|| true\` on the git-add call (the only \`||
true\` in this file with no inline comment justifying it) silently
swallowed that fatal error, meaning:

  guarded generator crashes before writing its file
    -> git add hits the missing pathspec, stages NOTHING at all
    -> || true swallows the fatal error
    -> git diff --cached --quiet sees no staged changes
    -> "No artifact changes -- nothing to commit", ARTIFACTS_CHANGED=false
    -> review PR step (gated on ARTIFACTS_CHANGED) is skipped
    -> job exits 0 (green), no safety-evidence review PR opened,
       even if other unrelated artifacts (fmea.json, safety-case.json,
       sbom.json, etc.) genuinely changed and were silently dropped too.

Fix: replaced the single git-add call with two passes.

  1. The 7 files from `|| true`-guarded generator steps
     (iso26262-gap-report.json, iec61508-gap-report.json,
     do178-gap-report.json, iec62443-gap-report.json,
     coverage-report.json, report.json, report.html) are added one at a
     time, gated on `[ -f "$f" ]`, logging a `::warning::` and skipping
     cleanly (not aborting the rest) when legitimately absent.
  2. Every other artifact (fmea.*, safety-case.*, tooling/tara.*, sbom.json,
     provenance.json, artifact-manifest.json, sas.*, sci.json,
     audit-pack.zip, fusa-badge.svg, qualify-report.json) -- none of whose
     generator steps are `|| true`-guarded -- is added via the original
     single git-add call, now with NO trailing `|| true`. If one of these
     is unexpectedly missing, git add's fatal pathspec error now propagates
     (GitHub Actions runs bash steps with `-eo pipefail` by default) and
     fails the job loudly instead of reporting a false green.

Verification:
- `python3 -c "import yaml; yaml.safe_load(open('.github/workflows/release.yml'))"` parses cleanly.
- act/yamllint not available in this environment; instead traced the new
  logic by hand and reproduced it standalone (see below).
- Standalone repro in a scratch git repo (not committed) extracting just
  the new git-add logic, run against three scenarios:
    (a) all 22 files present -> all 22 staged, identical to prior
        behavior, exit 0.
    (b) coverage-report.json (guarded-generator) missing, fmea.json
        genuinely changed -> coverage-report.json skipped with a
        ::warning::, all other 21 files (including the real fmea.json
        change) still staged, exit 0 -- no longer silently reports
        "nothing to commit".
    (c) sbom.json (non-guarded, required) unexpectedly missing ->
        `git add` fails with "fatal: pathspec 'sbom.json' did not match
        any files", script exits 128 immediately -- step now fails loudly
        instead of a false green.
- Confirmed via grep that this step's name/ID and the ARTIFACTS_CHANGED/
  ARTIFACTS_BRANCH env vars it sets are referenced only within this same
  job in release.yml; nothing in ci.yml or any other workflow references
  them, so no downstream breakage from this change.

Closes a finding from the cpp-RCP v3.0.0 deep audit (batch 3).

Signed-off-by: Matt <47545907+SoundMatt@users.noreply.github.com>
@SoundMatt
SoundMatt merged commit dd78cff into main Aug 22, 2026
25 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant