Repository navigation
ci(verify): run the wall-clock crash and realtime playback tests first and alone - #2983
Merged
Merged
Conversation
Contributor
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Member
Author
|
@greptileai review |
richiemcilroy
marked this pull request as ready for review
October 23, 2026 17:10
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two macOS Engine verify tests measure against the wall clock, and they fail when they share the runner with the rest of the suite:
crash_safetyhands frames to a real-time VideoToolbox session on a 30 fps schedule. Itscrash/equalitycase lost 3 to 7 consecutive frames in the in-process arm (for example[100, 101, 102, 213, 214, 215, 216]).realtime_loopcounts the frames playback presents in real time.realtime30-from20/avfpresented 65% of its frames against a 75% floor.Both pass when
run.shreruns them alone. Everycrash/equalityfailure in the last 80 Engine verify runs has the same pattern: 6 of 6 failed in the shared run and passed in the solo rerun. The solo reruns were also 7 to 19 s faster.realtime_loopfailed shared and passed solo in #2953. In the shared runs, the seek storms,chaos_recordingand the export jitter tests were running at the same time, and they load the same 6 vCPUs and media engine.This PR gives both tests
threads-required = "num-test-threads"andpriority = 100in thedefaultandcinextest profiles. They start first in their shard and nothing else runs alongside them. The thresholds and checks are unchanged, and no cases are marked as expected failures.Cost: about 140 s (
crash_safety) and 19 s (realtime_loop) of serial time, in whichever shard each one lands in. Both used to overlap with other tests.The other recurring macOS failure,
crash/recorder-kill/*/static-screen (fragment-span), was fixed in the harness by #2785.On this PR's macOS run (fast tier, on top of #2785), nextest started
crash_safetyfirst (1 of 61, 132.8 s, pass) andrealtime_loopsecond (18.3 s, pass) before any other test, andmacos / build and testpassed.

Confidence Score: 5/5
The PR appears safe to merge.Summary
Runs
crash_safetyandrealtime_loopfirst and alone within each nextest shard.Reviews (2) · Last reviewed commit: "ci(verify): run the wall-clock crash and..." · Reviewed by Greptile