[richiemcilroy]

Problem

Engine verify on Blacksmith macOS runners (paravirtualized VMs sharing the host media engine) flakes when many macOS jobs run at once and VideoToolbox sessions fail. Two failure modes, both also reachable by real users when the media engine is busy or the GPU resets:

1. The hardware encoder fails mid-export and the export dies. Log: [h264_videotoolbox] Error encoding frame: -19092 (sometimes followed by Error flushing frames: -19092, once preceded by IOServiceMatchingfailed for: AppleM2ScalerParavirtDriver), then export failed: Encode: Generic error in an external library. Cases: export/backward-hold/backward-jumps/mp460/ffmpeg, export/basic/colour-black/mp430/avf, audio/audio-mix-30s-a2-repair500/full/mp4-plain30 (jobs 120411517788, 119820555095, 120432834871).
2. The FFmpeg decoder burns its whole 30 s init timeout. Log: [h264] Failed setup for format videotoolbox_vld: hwaccel initialisation returned error. hundreds of times, each followed by decode_slice_header error / no frame!, then FFmpeg decoder timed out after 30s initializing: .../camera.mp4. Case: export/basic/camera-user-offset/mp430/ffmpeg (job 120411517788, 60 s = probe + exporter build, 30 s each).

Mechanism

Encoder: -19092 is not a public VTErrors.h code; it is a status the VideoToolbox output callback hands to FFmpeg's vtenc_output_callback, which logs it and stores AVERROR_EXTERNAL as the session's async error. The next avcodec_send_frame returns it, H264Encoder::queue_frame_reusable maps it to QueueFrameError::Encode, and the encoder thread in crates/export/src/mp4.rs turns it into a plain export failure. Encoder selection (get_codec_and_options / H264EncoderBuilder::build in crates/enc-ffmpeg/src/video/h264.rs) only falls back to libx264 when opening fails, not after frames are in flight, and the MP4 cannot switch codecs partway.

Decoder: ffmpeg-hw-device's get_format callback returned only the hardware pixel format and otherwise AV_PIX_FMT_NONE. When VideoToolbox hwaccel init fails, FFmpeg's ff_get_format drops that format and calls get_format again with the software formats; the callback chose nothing, so the slice fails, the h264 context stays uninitialised, and the next packet retries the (slow, failing) hwaccel init. first_frame in crates/rendering/src/decoder/ffmpeg.rs skips decode errors on the ordinary path, so it walked the whole file one failed hwaccel init per packet. Its existing software fallback only runs after the iterator is exhausted, which the 30 s timeout in spawn_ffmpeg_decoder (crates/rendering/src/decoder/mod.rs) beat.

Fix

• ffmpeg-hw-device::get_format: when the hardware format is not offered (its setup failed), return avcodec_default_get_format, which picks the best software format, and log a warning. Healthy machines never reach this branch.
• cap-export MP4: the encoder thread records when a hardware encoder (H264Encoder::is_hardware, new) fails while queueing, wrapping a VideoToolbox surface, or flushing. export_nv12 then logs a warning and runs the export once more from the start with the encoder priority overridden to libx264 (zero-copy VideoToolbox input skipped). ExporterBase::reopen_streaming reopens the staged output, streaming audio and its cancellation that the first attempt consumed. No retry when the user cancelled, when the failing encoder was already software, or for size-estimate sample exports. Output on a machine whose encoder does not fail is unchanged.

Test

• cap-export gets a test-hooks feature (enabled only by cap-verify, like cap-editor's) with fail_video_encoder_at(frames), which fails the first attempt's video encoder after N frames the way a hardware encoder does.
• New Engine verify test audio::encoder_fallback (cases audio/<mix>/encoder-fallback/mp4-app30 and mp4-plain30) injects the failure at frame 90 and checks the restarted export against the audio and video models (PCM, frame sequence, A/V sync, audio duration), plus that the file was encoded by libx264. Covers the streaming-audio app path and the in-memory plain path on every platform. Audio case floors raised to match.
• Unit test for the get_format fallback; ffmpeg-hw-device added to the sync-tests cargo test line so it runs.

No xfails added or changed.

Confidence Score: 5/5

The PR appears safe to merge, with both previous findings addressed and no new actionable issues found.

Summary

The PR restarts failed hardware MP4 exports once with libx264 and lets FFmpeg choose software decoding when hardware setup fails.

• MP4 exports restart with libx264 when a hardware encoder fails.
• FFmpeg keeps decoding in software when hardware setup fails.

_{Reviews (3) · Last reviewed commit: "fix(export): keep the encoder restart st..." · Reviewed by Greptile}

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
cap-status	Ready	Preview	Oct 23, 2026 6:53pm UTC

[capy-ai]

Capy found 1 potential issue (1 medium).

[greptile-apps]

Real encoder errors go untested

The injected failure sets hardware_encoder_failed directly, so encoder_fallback skips the new checks that make real hardware errors trigger a retry. The test would still pass if queueing or flushing errors stopped setting that flag.

Add focused tests showing that hardware queueing and flushing errors request a retry, while software encoder errors do not.

Prompt To Fix With AI

This is a comment left during a code review.
Path: crates/export/src/mp4.rs
Line: 655-658

Comment:
**Real encoder errors go untested**

The injected failure sets `hardware_encoder_failed` directly, so `encoder_fallback` skips the new checks that make real hardware errors trigger a retry. The test would still pass if queueing or flushing errors stopped setting that flag.

Add focused tests showing that hardware queueing and flushing errors request a retry, while software encoder errors do not.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

[richiemcilroy]

Fixed in 6defbb6. The decision is now two small functions:

• video_encoder_failure(hardware, restart, error) sets the restart flag only for a hardware encoder. Every queue, VideoToolbox surface and flush error site goes through it.
• restart_on_software_encoder(attempt_was_software, sampled, hardware_failed, cancelled) decides whether a failed attempt restarts.

New unit tests:

• hardware_encoder_errors_restart_on_software covers a queue error (QueueFrameError::Encode(External)) and the flush error (Video encoding failed: ...).
• software_encoder_errors_do_not_restart covers a software encoder error, plus an attempt already on libx264, a size-estimate sample and a cancelled export.

The injected failure no longer sets the flag directly. It marks the encoder as hardware and returns the real QueueFrameError::Encode(ffmpeg::Error::External) through the same video_failure path.

[richiemcilroy]

@greptileai review

[greptile-apps]

Software decode claims hardware use

When hardware setup fails, get_format now accepts a software format, but FFmpegDecoder keeps hw_device and reports is_hardware_accelerated() == true. The first software frame also skips the outer fallback that would report FFmpegSoftware.

This lets software decode pass the worker’s RF_REQUIRE_GPU check. A slow worker can keep rendering instead of rejecting the job for a retry on a healthy worker. Track the format actually selected and use it when reporting the decoder type.

Prompt To Fix With AI

This is a comment left during a code review.
Path: crates/ffmpeg-hw-device/src/lib.rs
Line: 49

Comment:
**Software decode claims hardware use**

When hardware setup fails, `get_format` now accepts a software format, but `FFmpegDecoder` keeps `hw_device` and reports `is_hardware_accelerated() == true`. The first software frame also skips the outer fallback that would report `FFmpegSoftware`.

This lets software decode pass the worker’s `RF_REQUIRE_GPU` check. A slow worker can keep rendering instead of rejecting the job for a retry on a healthy worker. Track the format actually selected and use it when reporting the decoder type.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

[richiemcilroy]

Fixed in edeb507. When get_format makes its choice, it now writes it to a per-device flag. install_get_format points the codec context's opaque at that flag, which lives in HwDevice and is exposed as is_software_fallback(). Picking the hardware format clears the flag. Falling back to avcodec_default_get_format sets it.

FFmpegDecoder::is_hardware_accelerated() and a new FramesIter::is_hardware_accelerated() return false once the flag is set. The rendering FFmpeg worker (crates/rendering/src/decoder/ffmpeg.rs) now picks DecoderType after the first frame is decoded, through frames.is_hardware_accelerated(). A fallback therefore reports FFmpegSoftware, and the render farm's RF_REQUIRE_GPU=1 check (crates/render-farm/src/video.rs, which reads decoder_type.is_hardware_accelerated() for screen and camera) rejects the chunk.

Those are the only non-example consumers. CUDA frame handling decides by frame format, so it is unaffected. FFmpegDecoder declares decoder before hw_device, so the codec context is freed before the flag.

fallback_reason stays None: DecoderInitResult doesn't carry a reason, and the get_format warning logs the fallback. The unit test now also checks that the flag is clear after the hardware choice and set after the fallback.

[capy-ai]

Capy found no issues.

View 2 other findings in Capy.

[richiemcilroy]

@greptileai review