You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
compat-gen skips old fixture versions in CI and stops at the first failure #10754
The backward-compatibility harness (vortex-test/compat-gen) has three gaps:
No workflow runs compat.py check --mode all on its own. compat-validation.yml defaults to last (inputs.mode || 'last'), and its only caller, publish-dry-runs.yml, passes last. So once a newer release publishes a fixture under the same name, CI stops reading the files of older releases. For example, after Tracking Issue: Fix Canonical Decimal #10701 lands, the old decimal forms that Add compat fixtures for old decimal forms #10753 adds exist only under the versions published before it.
One failing fixture stops the whole check. check.rs returns early on the first fixture write error, and assert_arrays_eq! panics instead of recording a failure.
The README ("Git worktree workflow") says that compat.py publish --git-ref <ref> builds the binary from a worktree at that ref. The script only uses the ref to pick the version (scripts/compat.py:394, :416) and always builds the current tree.
Proposed solution
Run --mode all before each release, or on a schedule.
Record failures per fixture, and report them together at the end.
Either make --git-ref build from the ref, or correct the README.
Additional context
Found while planning #10701. Research notes: #10633.
Activity