Skip to content

fix(verify): start the gated Instant harness screen track at the arm point like the product - #3005

Merged
richiemcilroy merged 2 commits into
mainfrom
fix/instant-gate-shared-zero
Oct 23, 2026
Merged

richiemcilroy merged 2 commits into
mainfrom
fix/instant-gate-shared-zero

Conversation

@richiemcilroy

@richiemcilroy richiemcilroy commented Oct 23, 2026 •

Copy link
Copy Markdown
Member

Problem

Engine verify's macos-vt / build and test fails intermittently on main (run 39951468297, PR #2930 run 39955181352):

XPASS export/instant-gated-system-continuous-mic-late/output/mp4/native

The gated Instant export cases measured audio +13 to +15 ms late against the screen, and the xfails in crates/verify/expectations.toml described it as the screen and audio pipelines taking their own first admitted frame as zero. The offset depends on where the harness's arm lands relative to the next 30 fps screen frame (the arm is Timestamps::now() after a tokio sleep to 0.517 s, frames are stamped on schedule), so it is roughly uniform over one frame period. When the arm wakes late, close to the 0.533 s frame, the offset drops below the 5 ms threshold (+1.73 ms in the failing run) and the strict xfail XPASSes.

Mechanism

The product already shares one zero. Since #2658, instant_recording::create_pipeline calls RecordingStartGate::set_video_origin_at_arm, so the gated Instant screen track starts at the arm point (video_origin_lead / shift_from_origin in output_pipeline/core.rs), the same instant the epoch-anchored audio starts at (audio_epoch). The setter was pub(crate), so cap_verify::pipeline::record_instant, which builds the Instant muxers by hand, could not call it. The harness kept recording the pre-#2658 behaviour: screen zero at the first admitted frame, audio zero at the arm point. The export xfails were measuring the harness, not the product.

Fix

  • RecordingStartGate::set_video_origin_at_arm is now pub.
  • record_instant enables it on its gate, as it already mirrors the product's AudioAnchor::PipelineEpoch audio anchor.
  • Removed the four xfails that only described that gap: export/instant-gated-*mic* sync-mic, export/instant-gated-system-* sync-system, and export/instant sync-mic and sync-system. The flaky instant-gated-system-only audio-system marker-loss entry and the ungated entries stay.
  • Removed report::tests::instant_system_audio_expectation_requires_the_microphone_timeline, which pinned the removed export/instant sync-system bound, and the mic-offset-ms / system-minus-mic-ms measures in instant_case that only fed that bound.

No product behaviour changes. Studio never sets the flag and is unchanged. The fixture cache key includes the recording crate's source hash, so CI re-records the Instant fixtures.

What CI verifies

cap-verify::export instant_output on macOS, Windows and Linux now runs the gated Instant cases with no sync excuse: each must measure mic and system audio within 5 ms of the screen regardless of where the gate opens in the frame cadence.

RetriggerView in GreptileConfidence Score: 5/5

The PR appears safe to merge, with the previous test failure addressed and no new blocking issues found.

Summary

The PR makes the Instant recording harness start its screen timeline at the gate’s arm point, matching the product.

  • Gated Instant exports now start screen timing at the arm point.

Reviews (2) · Last reviewed commit: "test(verify): drop the Instant system-on..." · Reviewed by Greptile

@vercel

vercel Bot commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
cap-status Ready Ready Preview Oct 23, 2026 5:48pm UTC

@capy-ai capy-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Capy found no issues.

View 2 other findings in Capy.

Open in Capy

@@ -134,21 +120,6 @@ bound = { count = 1, missing = 2, lost = 2 }
reason = "NEW: gated loopback-only system audio sometimes loses markers (2 of 11 lost in 1 of 2 fresh recordings, with the track 958 ms early). Bound: at most 2 markers lost, none unexpected"
link = ""

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Existing unit test now fails

Removing the export/instant/output/mp4/native sync-system entry breaks instant_system_audio_expectation_requires_the_microphone_timeline in report.rs. That test parses expectations.toml and requires exactly one matching entry; it now finds zero. Update or remove the obsolete test alongside the expectation so cargo test -p cap-verify can pass.

Prompt To Fix With AI
This is a comment left during a code review.
Path: crates/verify/expectations.toml
Line: 122

Comment:
**Existing unit test now fails**

Removing the `export/instant/output/mp4/native` `sync-system` entry breaks `instant_system_audio_expectation_requires_the_microphone_timeline` in `report.rs`. That test parses `expectations.toml` and requires exactly one matching entry; it now finds zero. Update or remove the obsolete test alongside the expectation so `cargo test -p cap-verify` can pass.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

@blacksmith-sh

This comment has been minimized.

@capy-ai capy-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Capy found no issues.

View 1 other finding in Capy.

Open in Capy

@richiemcilroy

Copy link
Copy Markdown
Member Author

@greptileai review

@richiemcilroy
richiemcilroy merged commit 2012f25 into main Oct 23, 2026
73 of 75 checks passed

This branch was successfully deployed

2 active deployments
Preview – cap-web — df1204d1 Deployed Oct 23, 2026 by vercel[bot]
Preview – cap-status — df1204d1 Deployed Oct 23, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant