Skip to content

fix: drop a run whose timings cannot be aggregated instead of failing the branch - #213

Merged
MarshallOfSound merged 1 commit into
mainfrom
sam/spec-weights-isolate-run
Oct 6, 2026
Merged

MarshallOfSound merged 1 commit into
mainfrom
sam/spec-weights-isolate-run

Conversation

@MarshallOfSound

Copy link
Copy Markdown
Member

Follow-up to #210: the reviewer's second pass noted that aggregateRun() ran outside the try/catch added to isolate one bad run, so a spec-timings.json that parses as JSON but has no files would still fail the branch's refresh. Aggregation now runs inside the same block, and timingsFromArtifactZip() rejects such a file by name. Test added with a fixture.

… the branch

The try/catch around a sampled run only covered fetching its artifacts;
aggregating them ran outside it, so a spec-timings.json that parsed as
JSON but carried no per-file data would still fail the whole branch's
refresh. Aggregate inside the same block, and reject such a file where
it is read so the message names the artifact.
@MarshallOfSound
MarshallOfSound requested review from a team as code owners September 12, 2026 03:00

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — the fix correctly moves aggregateRun(jobs) inside the try/catch so a bad spec-timings.json no longer fails the whole branch refresh, and the new validation in timingsFromArtifactZip() is covered by a fixture-backed test.
What was reviewed: the restructured try/catch in computeRefresh() (aggregation failures are now caught and the run skipped), the new timings.files guard in timingsFromArtifactZip(), and the added test/fixture. Checked whether the files guard's typeof === 'object' check could admit non-object values like arrays — it can, but that's a narrow edge case unrelated to the bug this PR fixes.

Extended reasoning...

Overview

This is a small follow-up fix (second commit building on PR #210) touching src/utils/roll-spec-weights.ts and src/utils/spec-weights.ts, plus a new test and binary fixture. The change moves aggregateRun(jobs) inside the existing try/catch block in computeRefresh()'s per-run loop, so an aggregation failure is now caught and logged rather than propagating uncaught and failing the whole branch's refresh. It also adds a validation check in timingsFromArtifactZip() that throws a named error when a parsed timings JSON entry lacks a valid files object, rather than silently pushing malformed data downstream.

Security risks

None. This is internal CI tooling that reads GitHub Actions run artifacts and JSON timing data to compute test-shard weights; there is no user input, auth, or crypto surface touched here.

Level of scrutiny

Low-to-moderate. The diff is small (four files, ~18 lines added/changed excluding the binary fixture), self-contained, and directly matches the PR's stated intent (a targeted correction to a try/catch scope from the prior PR). The bug-hunting system found no issues after running to a dry streak (not cut off), and the change includes a genuine regression test using a real malformed zip fixture rather than a mock.

Other factors

One candidate issue was raised and ruled out: the files guard uses typeof timings.files !== 'object' || timings.files === null, which would also accept an array (e.g., {"files": []}) since arrays are typeof object. This is a real minor gap, but it's an edge case outside the scope of what this PR is fixing (the bug being fixed is about files being entirely absent/null, which the fixture reproduces), and it doesn't represent a regression introduced by this diff relative to the code's prior behavior (previously such malformed data would have been pushed through with no validation at all, arguably worse). Given the fix does what it claims, is narrowly scoped, and is backed by a passing test against a real fixture, this is approvable without requiring further human review.

@MarshallOfSound
MarshallOfSound merged commit cf84856 into main Oct 6, 2026
6 checks passed
@MarshallOfSound
MarshallOfSound deleted the sam/spec-weights-isolate-run branch October 6, 2026 22:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants