Repository navigation
media: apple: avd: decode VP9 frames smaller than 64x64 on the legacy transport - #2773
Conversation
|
Read at 55c2f93. The change is right for what it does: the 64-pixel floor came from rkvdec, not from this hardware, and raising the aligned size to the 64x64 CAPTURE minimum before the comparison matches how the format is sized. A frame larger than the CAPTURE buffer is still refused. The libvpx What it doesn't cover yet is the other AVD generation. Your run is on the legacy transport (hw 0x40000). The M4-class and A18 Pro decoders run the firmware transport, where an input the firmware doesn't expect can reset the engine rather than fail the frame. One run of the same Same note as on #2756: please drop the Assisted-by: LLM (Claude:Opus 5.5) |
55c2f93 to
d1c3ea1
Compare
|
Thanks, Ryan. Both points are addressed in d1c3ea1. Firmware transport. I don't have an M4 Pro or an A18 Pro, so I've taken your second option and limited the change to the transport I measured:
It fails without the patch, and it also fails with the earlier unscoped version. I reran on the J516c with the scoped build: all 64 Attribution. I removed the Assisted-by: LLM (Claude:Opus 5.5) |
|
Thanks, d1c3ea1 is the right scope: the floor stays on the firmware transports and comes off only on the legacy transport you measured, selected by the backend in use rather than by chip, and the test covers both sides. The code is fine from our side. One small thing on attribution: this tree keeps agent disclosure out of commits entirely, Assisted-by: LLM (Claude:Opus 5.5) |
… transport
validate_dec_params() refuses every VP9 frame narrower or shorter than
64 pixels, and requires the frame size, aligned to 64x16, to equal the
CAPTURE format. The CAPTURE format is never smaller than the 64x64
minimum of the VP9 frame size range, so a frame below 64 in either
dimension could not pass either check. Such a stream fails its first
frame ("unexpected bitstream resolution") and stops. The 64-pixel
floor came from rkvdec; H.264, HEVC and AV1 frames as small as 16x16
already decode on this decoder.
Raise the aligned frame size to the 64x64 minimum before comparing it
with the CAPTURE format, as the CAPTURE format is. A frame larger than
the CAPTURE format is still refused.
Drop the floor on the legacy transport only, where it was measured.
The firmware transports (queued firmware, Apple firmware) keep it
until sub-64 frames are measured there: an input their firmware does
not expect can reset the engine instead of failing the frame. Add
avd_backend_is_open() for that check.
Tested on a J516c (M3 Max, T6031, legacy transport). Of the 64 libvpx
vp90-2-02-size-WxH conformance vectors (8 to 66 pixels), 4 decoded
before (those 64 or larger in both dimensions) and all 64 now match
FFmpeg's software decode, which matches the libvpx MD5s. H.264, HEVC,
VP9 and AV1 at other sizes and throughput are unchanged. The VP9 host
test now checks that a 32x16 frame into a 64x64 CAPTURE format is
refused on the firmware transport, decodes on the legacy transport,
and that a larger frame is still refused there.
Signed-off-by: Naeem Malik <104218167+malik-na@users.noreply.github.com>
d1c3ea1 to
e08ff74
Compare
DjDeveloperr
left a comment
There was a problem hiding this comment.
Adversarial review at live head e08ff749d9136e3a796b2399a66540055dc3f8bf: 0 code, policy, attribution, or CI blockers — approving.
Technical check
- Transport scoping (
avd-backend-open.c:180,avd-vp9.c:681):avd_backend_is_open(ctx->dev)checks the active backend ops pointer (avd->backend == &avd_open_backend), parallel toavd_backend_is_qfw(). This is a runtime transport check rather than chip/SoC/firmware-version gating (§1), and safely keeps the 64-pixel floor on the unmeasured queued-firmware and Apple-firmware transports where unexpected inputs can reset the engine (§3). - Dimension alignment & downstream safety (
avd-vp9.c:686-705): Clamping the aligned frame size to the 64×64 CAPTURE minimum matchesavd_try_capture_fmt()'sfrmsizeconstraints while still rejecting frames larger thanctx->decoded_fmtwith-EINVAL. Checked downstream callers for sub-64 dimensions:set_tiles()computessb_64_cols/sb_64_rows >= 1for allframe_*_minus_1 >= 0,set_refs()usesdst->vp9.width = frame_width_minus_1 + 1 >= 1(no divide-by-zero), andavd_vp9_alloc_bufs()allocates fromfmt_width/fmt_height(>= 64×64). - Host tests (
tests/vp9-test.py:212-229): Covers both transports (32×16 into 64×64 rejected on firmware transport, decoded on legacy transport, and oversized 1920×1088 still rejected with-EINVAL).host-testsCI is green. - Minor non-blocking notes:
max_t(unsigned int, round_up(dec_params->frame_width_minus_1 + 1, 64), 64)is harmlessly redundant for width sinceround_up(>= 1, 64)is already>= 64(keeps symmetry with height, totally fine).- For future PRs, including the §5 boot-level regression counts (
0 SELinux denials, 0 failed units, 0 new kernel errors) alongside the codec kernel-log check and noting/requesting a rev 3 (M1/M2) legacy-transport spot-check rounds out the checklist cleanly.
Checklist
- §1–§4 / §6: Clean — no model gating, no hacks or new cmdline params, firmware transports protected, commit
e08ff749carries human author/committer andSigned-off-byonly, PR body ends withAssisted-by:, naming clean. - §5 Evidence & CI: Complete on J516c (M3 Max, T6031,
integration/aurora7.3.0, legacy transport): 64/64 libvpxvp90-2-02-size-WxHvectors match FFmpeg SW decode (up from 4/64), 54/54 H.264/HEVC/VP9/AV1 regression suite matches, throughput unchanged (480 / 198 fps), 0 AVD kernel errors, andhost-testsCI passed one08ff749.
Assisted-by: LLM (Gemini 4 Neon)
|
Merge-train adversarial review (MERGETRAIN-LINUX3) at e08ff74: blockers (below). Merges clean on integration/aurora 933a43f. Full reviewVERDICT: BLOCKERS Checked:
Blockers:
Nits:
Missing rows:
|
|
Rows for the legacy-transport question in our review above, at e08ff74 (cherry-picked onto the fleet kernels the two Macs run; module-only A/B, no reboot). Both Macs normally use the queued transport, so the legacy transport was forced for the test: the pre-ABI-2 open image (avd-fw 0.1-1, built from aurora-silicon/avd-fw 9cbde5c7 with the package recipe) was loaded from a temporary firmware path, and dmesg confirmed
With your T6031 row this covers the v3 and v4 image tiers. Our blocker narrows to the v5 tier (T6040, T8132, T8140), which has not been run on the legacy transport with sub-64 frames. We will try a Neo or J616s row the same way and post it here. Assisted-by: LLM (Claude:Opus 5.5) |
|
Third row, the v5 image tier: Neo (T8140,
With the T6031 row in the description, the legacy transport has now been measured on the v3 (M1 Pro, M2), v4 (T6031) and v5 (T8140) image tiers with no engine reset. That resolves our blocker: no blockers from our side at e08ff74. Assisted-by: LLM (Claude:Opus 5.5) |
5a19933
into
aurora-silicon:integration/aurora
The AVD refuses every VP9 frame smaller than 64 pixels in either dimension. Such a stream fails its first frame and stops ("stream stopped after 1 failed frames"). H.264, HEVC and AV1 frames as small as 16x16 already decode on the same hardware.
Why
validate_dec_params()has two checks a small frame can't pass:-EINVAL. It came from rkvdec.avd_try_capture_fmt()applies it), so a 32x32 frame (aligned 64x32) can never equal its 64x64 buffer.The change
avd_backend_is_open()makes that check, next toavd_backend_is_qfw(). Because the backend is checked on every frame, a device that falls back from the queued firmware to the legacy transport takes the legacy rule from then on.This is independent of #2136. That PR relaxes the comparison to "fits" but keeps the 64-pixel floor, so it doesn't decode these streams on its own. The two touch the same few lines of
validate_dec_params(), so whichever merges second needs a trivial rebase.Testing
On a J516c (16" MacBook Pro, M3 Max, T6031), AVD hw version 0x40000 on the legacy transport, with this patch built as a module against
integration/aurora24dc422 (7.3.0) and booted from aura 1.0.0b14. Decoded through GStreamer 1.28.8v4l2codecs. The raw NV12 from the decoder was compared frame by frame with FFmpeg's software decode, which matches the libvpx MD5s.vp90-2-02-size-WxH, W and H in {8, 10, 16, 18, 32, 34, 64, 66} (64 vectors)sizeupresolution changeNo AVD errors in the kernel log after the patch.
Not tested: the queued-firmware and Apple-firmware transports. I don't have an M4-class or A18 Pro machine, and on those transports this patch keeps the existing 64-pixel floor.
tests/vp9-test.pynow runs a 32x16 key frame into a 64x64 CAPTURE format:Without the patch the test fails at the legacy decode (
Assertion '!avd_vp9_run(&ctx)' failed). With an unscoped version of the change it fails at the firmware-transport refusal (Assertion 'avd_vp9_run(&ctx)==-EINVAL' failed).Other host tests:
av1-test.pyanddispatch-test.pypass.program-test.pyandrecovery-test.pyfail the same way on unmodifiedintegration/aurora: the first needs a base commit missing from my shallow clone, and the second no longer compiles against the current source.unicorn-vp9-test.pywas not run, because theunicornmodule isn't installed on the test machine.Seen while testing (not changed here)
filesinknever reaches EOS, with or without this patch. The same stream intofakesinkcompletes, and nothing is blocked in the driver. It's not investigated yet.sizeimageincludes it. The picture data is correct.Assisted-by: LLM (Claude:Opus 5.5)