Skip to content

media: apple: ave: add the T6021 (M2 Max) encoder - #2374

Open
ludagoo wants to merge 7 commits into
aurora-silicon:integration/aurorafrom
ludagoo:ave/t6021
Open

ludagoo wants to merge 7 commits into
aurora-silicon:integration/aurorafrom
ludagoo:ave/t6021

Conversation

@ludagoo

@ludagoo ludagoo commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Both video encoders of the M2 Max, constant-quality H.264 and HEVC, and two sessions per firmware boot on every 6070.11.1 encoder.

Base: integration/aurora 0abc2f385e54. Seven commits on branch ave/t6021 (ludagoo/linux), tip 9aadf56a25ad. Needs aurora-silicon/aura#341 (the boot loader hand-off). The encode-quality changes are #2375, on top of this one, and the device tree is #2373 and #2376.

  1. pin the T6000 and T8112 6070 command bytes (9a4f94ef751e). A KUnit case builds Start and the first two Process commands from the 6070.11.1 layout with default parameters and compares their CRC32 with the base. Every later commit here and in media: apple: ave: 6070.11.1 encode quality: H.264 references, CBR/VBR, VUI timing #2375 passes it.
  2. give each encoder its own debugfs directory (dce6e177789b). apple-ave-<device>, because the second encoder could not create apple-ave. The path changes on every chip; the two documents that give it are updated.
  3. read the firmware CRC and IOVA per encoder (851ad30ab202). aurora,crc32 on each memory-region reservation, with /chosen as the fallback, and several iommu-addresses tuples (the encoders share TEXT); the admin guide describes the reservation property. Every chip goes through this code. With today's boot loader (no aurora,crc32, one tuple per reservation) it does what it did before; only T6021 was run.
  4. map an extra0 window that is not page aligned (aa930a5963ae). Neither T6021 window is (ave0 0x7c44 bytes at 0x2902803e0, ave1 0x3dfc at 0x28e680228). The mapping covers the larger of the resource and the variant size. T6000's and T8112's mappings do not change (both windows are 16 KiB aligned). The alignment check is dropped for every chip; see arm64: dts: apple: t6001, t6002: enable every video encoder #2295 for the one other unaligned window, on a disabled node.
  5. add the T6021 (M2 Max) encoder (805c89fc6e67). AppleAVE2FW_H14C, a larger build of 6070.11.1 (TEXT 0xf0000, DATA 0x134000) with T8112's registers, boot argument and timebase. Sessions run the T6000/T8112 commands unchanged: one constant-quality session per encoder at this commit.
  6. bound the sessions kept for a power cycle (b4de0709d3f8, new). A session the firmware saw keeps its buffers until the encoder is power cycled, which happens only after the last session closes. Once four wait, a session that would join the running firmware gets EBUSY with a rate-limited warning; the count resets at the power cycle. This holds for every layout (9003 already runs two sessions per boot).
  7. run two sessions on one 6070.11.1 boot (9aadf56a25ad). max_sessions is 2 on the 6070.11.1 layout, so T6000, T8112 and T6021 all get it. Four on an M2 Max; a fifth STREAMON gets EBUSY.

The lifecycle is upstream's, unchanged: Halt after every session, VENC power cycle before the next.

What each T6021 value was checked against on the J475c (also in abi.rst):

  • Config shared area, address at +0x60 and size at +0x68: with both zero, as on T8112, the firmware never answers Config. With a 64 KiB, 16 KiB-aligned area both encoders answer Config, Open and Start and encode. It is a variant flag, config_shared, because it belongs to the H14C build: T8112's build of 6070.11.1 answers Config with both words zero. The area is allocated at the first boot and kept until the device is removed.
  • Boot device ID 20 and device type 17: both encoders boot and answer the handshake.
  • TEXT word 0x423c: holds the low 32 bits of DATA's device address on both encoders. TEXT with that word cleared has CRC 0d56db87, which selects the 6070.11.1 layout.
  • extra0: the windows above, mapped whole; Config takes the IOVA of the window's first byte.

Overlap: #2218 and #1900 (T8142) and #1990 (Neo) conflict textually with this PR if they merge first (ave.h; #1990 also ave-abi-test.c). A rebase needs KUnit again, not hardware. #1861 merges cleanly.

Tested

  • KUnit apple-ave (arm64, LLVM) at every commit: 44 at the base, 45 at commits 1-5, 46 at commits 6 and 7 (ave_quarantine_count: four handed sessions and one never handed, the count is four, and the reap frees them and clears it). All pass, no compiler warnings.
  • checkpatch.pl --strict: 0 errors, 0 warnings, 0 checks on every commit.
  • tools/asahi/run-host-tests.py --only ave: 6/6 at the base and at this tip.
  • Mac Studio (M2 Max, J475c), macOS 13.5 firmware, kernel 7.3.0 with the previous version of this stack (the two-session cap as a T6021 variant field, media: apple: ave: 6070.11.1 encode quality: H.264 references, CBR/VBR, VUI timing #2375's settings on T6021 only) and the aura PR. This version has not run on hardware; the session code is the same, and the J475c row for it is listed below.
    • Both encoders bound (/dev/video11 = 40d100000, /dev/video12 = 507100000). All VENC domains are off after boot and again a few seconds after each session.
    • Six open/encode/close cycles per encoder, H.264 and HEVC alternating: 6/6 each. Every session boots the firmware, every close ends in a confirmed Halt.
    • lifecycle_test 10 times per encoder: 10/10 each, and the encoders encode afterwards.
    • Two sessions on each encoder at once, both encoders, paced at 30 fps, one session of each pair closing first: 30 rounds, 120/120 sessions, and a third session on either encoder refused. These were 45 and 90 pictures of 720p at CBR 1 Mbit/s with media: apple: ave: 6070.11.1 encode quality: H.264 references, CBR/VBR, VUI timing #2375's one-frame-at-a-time submission, so the firmware never had frames from two sessions in flight at once. Then 10 more rounds (40/40) with the kernel's iommu-dma mappings at idle counted before and after (/proc/vmallocinfo lines whose caller contains dma): 393 mappings, 199901184 bytes, both times, so the quarantine is emptied. That output was read in the session; no log file was kept.
    • 182 firmware starts on that kernel, 2 without a hello (ave1, status 0x2e); the existing power-cycle retry recovered both and the sessions completed. This matches what media: apple: ave: H.264 encoding on the M1 Pro and M2 #1550 saw on T6000/T8112.
    • echo platform > /sys/power/pm_test; echo freeze > /sys/power/state, idle after use and again while a paced H.264 session streamed: both returned, the streaming session finished all 300 pictures, both encoders encoded afterwards, and the VENC domains powered off again.
    • No DART faults.

The lifecycle depends on the device tree. With the apple,t6000-ave-dma blocks next to each encoder enabled, apple-ave-dma binds them, ps_venc_sys and ps_venc1_sys stay on, the power cycle no longer resets the coprocessor, and after the first session every boot goes unanswered (10 tries, then EAGAIN). #2376 leaves them disabled.

Not done yet

  • This version on the J475c: two sessions and the new EBUSY bound (four sessions closed while one stays open, then a fifth refused), with the buffer size per closed session measured. Planned for the next window.
  • Only the J475c and one firmware (macOS 13.5, 22G74) were run. The M2 Max MacBook Pros (J414c, J416c) get the encoders from arm64: dts: apple: t6021: enable the video encoders #2376 but were not booted.
  • No other chip was run with this series. T6000 and T8112 now get two sessions per boot and were not run with two; with this PR alone they would pipeline both sessions with B pictures, which no chip has run. T8140, T6041, T6050 and T8142 go through commits 2-4 and 6.
  • Four for the bound is a judgement, not a measured limit.
  • Unbind and rmmod of the driver were not exercised on T6021.
  • System sleep was run with one streaming session, not two. Real s2idle (with a wake source) was not run; pm_test=platform was.
  • t6020 and t6022 are not matched by the driver.

🤖 Generated with Claude Code

T6000 and T8112 run the 6070.11.1 Start and Process commands that were
qualified on hardware byte for byte. Nothing tests that those bytes
stay the same when the shared 6070 writer changes for another chip.

Add a KUnit case that builds Start and the first two Process commands
for H.264 and HEVC, at 1280x720 30/1 and 1920x1080 30000/1001, with
default session parameters, and compares their CRC32 with the values
the builder produces at this commit.

Assisted-by: Grok 4.7
Assisted-by: Claude Opus 5.5
Signed-off-by: Lucas Goossen <lucas.goossen@hey.com>
Every instance creates the debugfs directory apple-ave. On a machine
with two encoders the second one fails to create it and has no
lifecycle_test, inject_wedge or last-command files.

Name the directory apple-ave-<device name>, for example
apple-ave-40d100000.video-codec, and update the two documents that
give the old path. The path changes on every chip.

Assisted-by: Grok 4.7
Assisted-by: Claude Opus 5.5
Signed-off-by: Lucas Goossen <lucas.goossen@hey.com>
@RyanTheTide

Copy link
Copy Markdown
Contributor

Thanks for the T6021 encoder bring-up. The pinned-bytes KUnit case is a nice way to show T6000 and T8112 are untouched.

One convention point. We select behaviour by a hardware or firmware property, not by the list of machines it was run on. drivers/media/platform/apple/ave/ave-drv.c:114 gives two sessions per encoder to T6021 only (.max_sessions = 2), while ave-layout.c treats T6000, T8112 and T6021 as one 6070.11.1 firmware. Commit 6 says T6000 and T8112 "stay at one", but not why.

If the two-session capacity comes from the larger T6021 firmware build (its bigger DATA, for example), please say so in a comment at ave-drv.c:114. That is a firmware difference and is fine as variant data. If it is a property of 6070.11.1 itself, it belongs on the layout, so every encoder running that firmware gets it. We can run the two-session rows on our M1 Pro (T6000) and M2 (T8112) once it is the default.

A small note: the quarantine growth while one session stays open has no cap. A bound or a log line would help.

@RyanTheTide

Copy link
Copy Markdown
Contributor

Thanks, this is careful work. The parts that let one driver serve several encoders are general: a debugfs directory per device, a CRC and an iommu-addresses tuple per reservation, and the unaligned extra0 window. They follow the device tree's node count on any SoC. Two requests:

  1. ave-drv.c:114 .max_sessions = 2: T6000 and T8112 run the same 6070.11.1 layout with one session. If two sessions are a property of the firmware, please put the cap on ave_layout_6070_11_1 so that every 6070 part gets it. We will run the two-session rows on an M1 Pro and an M2. If there is a hardware reason it is T6021-only, please state it in the comment at ave-v4l2.c:767.
  2. Please describe each T6021 value in the PR text and abi.rst (in media: apple: ave: 6070.11.1 encode quality: H.264 references, CBR/VBR, VUI timing #2375) by the behaviour it was checked against on the J475c, rather than by how it was located.

We will run no-regression rows for commits 2-4 on M1 Pro and M2: bind on today's loader, lifecycle_test three times at the new debugfs path, and a 1080p H.264 and HEVC encode at the usual rate, with no DART faults. Your J475c rows cover T6021.

A boot loader that hands over two encoders cannot name both DATA images
with the one /chosen aurora,ave-data-crc32 property, and the encoders
share one TEXT reservation, so its iommu-addresses property carries
one tuple per encoder.

Read aurora,crc32 from each memory-region reservation and fall back to
the /chosen property when it is absent. Accept several iommu-addresses
tuples and use the one whose phandle is this device; a reservation with
none, or with two for this device, is refused as before. Describe the
reservation property in the admin guide, next to the /chosen ones.

Every chip goes through this code. With today's boot loader (no
aurora,crc32, one tuple per reservation) it does what it did before;
only T6021 was run.

Assisted-by: Grok 4.7
Assisted-by: Claude Opus 5.5
Signed-off-by: Lucas Goossen <lucas.goossen@hey.com>
The driver requires the extra0 register window to start on a 16 KiB
boundary. On T6021 neither encoder's window does: ave0's is 0x7c44
bytes at 0x2902803e0, ave1's 0x3dfc bytes at 0x28e680228.

dma_map_resource() maps the pages that contain an unaligned range and
returns the IOVA of its first byte, which is the address Config takes.
Drop the alignment check, and map the larger of the resource size and
the variant size, so that the whole window is covered: with the
variant size alone, the last 0x24 bytes of ave0's window would be
unmapped.

T6000's window is 0x24 bytes at a 16 KiB aligned address and T8112's is
page aligned and exactly its variant size, so both map 0x4000 bytes as
before. The alignment check goes for every chip: an unaligned window is
now mapped, not refused. The only such window outside T6021 is on the
second M1 Max encoder, which the device trees leave disabled.

Assisted-by: Grok 4.7
Assisted-by: Claude Opus 5.5
Signed-off-by: Lucas Goossen <lucas.goossen@hey.com>
The M2 Max runs a larger build of the 6070.11.1 firmware the T6000 and
T8112 support uses, AppleAVE2FW_H14C: TEXT 0xf0000 and DATA 0x134000,
with the link cell at TEXT 0x423c. Its two encoders share TEXT and
each has its own DATA. The coprocessor registers, boot argument and
timebase are T8112's. With boot device ID 20 and device type 17 both
encoders of a J475c boot, answer the handshake and encode. The
identity CRC, TEXT with the link cell cleared, is 0x0d56db87; select
the 6070.11.1 layout for it.

Config on T6021 needs a shared area. With the pointer and size at
+0x60 and +0x68 left zero, as on T8112, the firmware never answers
Config. With a 64 KiB, 16 KiB-aligned area both encoders answer
Config, Open and Start. Allocate it at the first boot, keep it until
the device is removed, and pass it for a variant with config_shared.
That is a property of the H14C build rather than of 6070.11.1: the
T8112 build answers Config with both words zero.

The 6070 sessions on T6021 run the T6000/T8112 commands unchanged in
this commit: one constant-quality session per encoder.

abi.rst said 6070 streams H.264 without B pictures; the 6070 path
already offers HEVC Main and B pictures, so describe that, and say
what T6021 adds and what each value was checked against. The Kconfig
help now says what each chip offers.

Assisted-by: Grok 4.7
Assisted-by: Claude Opus 5.5
Signed-off-by: Lucas Goossen <lucas.goossen@hey.com>
A session the firmware saw keeps its buffers until the coprocessor is
proved to have lost power, and that happens only after the last
session closes. With two sessions per boot, one long-lived stream and
a second that is opened and closed again and again keep every closed
session's buffers, a full set per session, with no bound.

Count the sessions waiting for the reap. Once four wait, refuse a new
session that would join the running firmware with EBUSY and a
rate-limited warning. The refused session was never handed to the
firmware, so it is freed at once. When the last stream closes, the
power cycle frees all of them and resets the count. This holds for
every layout; 9003 already ran two sessions per boot.

Assisted-by: Claude Opus 5.5
Signed-off-by: Lucas Goossen <lucas.goossen@hey.com>
The 6070.11.1 layout allows one session per firmware boot, while the
driver already keeps the process buffers per session. Allow two, on
every SoC that runs this firmware. On a J475c, four H.264 sessions,
two on each encoder, ran in real time for five minutes at their target
rates, and a fifth STREAMON got EBUSY. T6000 and T8112 were not run
with two sessions.

When the first of two sessions closes, its buffers stay quarantined
until the last session closes and the encoder is power cycled, as on
9003.

Assisted-by: Claude Opus 5.5
Signed-off-by: Lucas Goossen <lucas.goossen@hey.com>
@ludagoo

ludagoo commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor Author

Thanks for the review. The updates are pushed; replies to each of your comments below.

Re: .max_sessions = 2 only on T6021; quarantine growth

There is no T6021-specific reason, so the cap is now on the layout: ave_layout_6070_11_1.max_sessions is 2, and the variant field and ave_session_cap() are gone (commit 7, 9aadf56a25ad). T6000 and T8112 get two sessions too. What ran on the J475c: four 720p H.264 CBR 1 Mbit/s sessions at once, two per encoder, paced at 30 fps for 5 minutes (9000 frames each) with one frame in flight per session (#2375's default); all four finished (1.000 Mbit/s coded / 1.005 Mbit/s whole, all 8700 P slices references), a fifth STREAMON on either encoder immediately returned EBUSY, and VENC domains returned off-0. Two pipelined sessions on one encoder, which is what T6000/T8112 would run with only this PR, has not run on any chip.

The quarantine is now bounded (new commit 6, b4de0709d3f8). Once four stopped sessions wait for the power cycle, a session that would join the running firmware gets EBUSY with a rate-limited warning; the count resets when the last stream closes and the reap runs.

Measured on the J475c (/dev/video11, awk '/dma/ {n++; s+=$2} END {print n" mappings "s" bytes"}' /proc/vmallocinfo):

  • Baseline idle: 242 mappings 105086976 bytes.
  • One long paced session active: 258 mappings 128155648 bytes (+16 mappings, +23,068,672 bytes).
  • Closes 1–4 of a second 30-frame session each added 54 mappings and 34,177,024 bytes (~32.6 MiB) (312 -> 366 -> 420 -> 474 mappings; up to 264,863,744 bytes).
  • Fifth STREAMON: failed with EBUSY (2 ms), and dmesg logged: apple-ave 40d100000.video-codec: 4 stopped sessions wait for a power cycle; close every stream first.
  • After closing the long session and VENC returning off-0: 458 mappings 241795072 bytes (216 mappings / 136,708,096 bytes, ~130.4 MiB, waiting for the next boot's ave_sess_reap()).
  • After starting and closing a new 30-frame session: exact return to 242 mappings 105086976 bytes.

So four stopped sessions cap the quarantine at ~130 MiB per encoder; KUnit also covers the count. Also run on the J475c: lifecycle_test 1 and 2 three times per encoder (all returned 0, Halt confirmed, restored DATA CRC 9e559ea2 / d142e9b5, encodes ok afterwards), three pm_test=platform freeze cycles with 60-frame H.264 encodes on both encoders after each, and 0 DART faults across the whole run.


Re: two requests; no-regression rows for commits 2-4

  1. Done as above: on the layout, all 6070.11.1 parts.
  2. The PR text and abi.rst now say what each T6021 value does on the J475c, not how it was found:
    • Config shared area at +0x60/+0x68: with both words zero the firmware never answers Config; with a 64 KiB, 16 KiB-aligned area both encoders answer Config, Open and Start and encode. It stays a variant flag (config_shared), because it is a property of the H14C build: T8112's 6070.11.1 build answers Config with both words zero.
    • Boot device ID 20, type 17: both encoders boot and answer the handshake.
    • TEXT word 0x423c: holds the low 32 bits of DATA's device address on both encoders; the identity CRC is TEXT with that word cleared, 0d56db87.
    • The extra0 windows: the sizes and addresses as the ADT gives them, mapped whole.

Thanks for the commit 2-4 rows. The debugfs path in those rows is now apple-ave-<device>, for example /sys/kernel/debug/apple-ave-<node address>.video-codec/lifecycle_test.

@RyanTheTide

Copy link
Copy Markdown
Contributor

Thanks for round 4. Putting max_sessions = 2 on ave_layout_6070_11_1, bounding the quarantine at four stopped sessions with a logged EBUSY, and the vmalloc accounting answer both points. Keeping config_shared as variant data is fine, since T8112's build answers Config without the area; that is a firmware-build difference. As you note, two pipelined sessions on one encoder have not run anywhere, so on our M1 Pro and M2 we will run that, plus the commit 2-4 rows at the new apple-ave-<device> debugfs path, and post the results here.

@RyanTheTide

Copy link
Copy Markdown
Contributor

@ludagoo, a tidy-up pass over the open PRs here. These of yours need something:

If any of these are no longer needed, closing them helps; otherwise a merge or rebase onto integration/aurora is usually all they need. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants