Skip to content

arm64: dts: apple: t6021: enable the video encoders - #2376

Open
ludagoo wants to merge 13 commits into
aurora-silicon:integration/aurorafrom
ludagoo:ave/t6021-j475c
Open

ludagoo wants to merge 13 commits into
aurora-silicon:integration/aurorafrom
ludagoo:ave/t6021-j475c

Conversation

@ludagoo

@ludagoo ludagoo commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Stacked on #2375 (ave/t6021-rc, and so on #2374, ave/t6021) and #2373 (ave/t602x-dma). Review the top commit.

Base: the T6021 PRs (ave/t6021, ave/t6021-rc). Two commits on branch ave/t6021-j475c (ludagoo/linux), tip a38dade589ee: the t602x DMA description (9966801f4b0e, the same patch as #2373; dropped here once #2373 is merged) and this enable. Needs aurora-silicon/aura#341.

Enable both encoders and their two DARTs each (m2_dart_aveN_0, m2_dart_aveN_1) in t6021-peripherals.dtsi, so every M2 Max machine gets them: J414c, J416c and J475c. The board files do not change. This is safe on a machine nobody ran: the boot loader hands each encoder the firmware iBoot preloaded or disables the node, and without a handoff (an older boot loader) the driver refuses the reservation and does not bind. The DARTs then bind with nothing behind them.

The auxiliary DMA blocks (m2_aveN_dma) stay disabled, as on every other SoC, with the reason in a comment above each. With them enabled, apple-ave-dma binds them; it has no runtime PM, so ps_venc_sys and ps_venc1_sys never power off, the encoder's power cycle does not reset the coprocessor, and after the first session's Halt the firmware never answers a boot again. #1861 (draft) adds runtime PM to apple-ave-dma; once it is in, these blocks can be enabled and tested again.

Tested

  • make CHECK_DTBS=y for the eight t602x trees (dtschema in a venv, aarch64 build host) at the base, at arm64: dts: apple: t602x: describe the encoder DMA apertures as one block #2373 and at this tip: all 8 build, no dtc warnings, 481 schema findings at each. Compared with the base, one finding on t6022-j180d (a PCIe reset-gpios line) differs only in a phandle number (461 at the base, 465 here), because the DMA nodes renumber phandles; nothing new or removed for the encoders or DARTs.
  • checkpatch.pl --strict: clean on both commits.
  • KUnit apple-ave at both commits: 49/49.
  • Mac Studio (M2 Max, J475c), kernel 7.3.0 with the T6021 PRs and the aura PR, with an earlier version of this change (the same six nodes, enabled in t6021-j475c.dts instead of the dtsi):
    • Both encoders handed over (FDT: AVE /soc/video-codec@40d100000 from ADT /arm-io/ave0, the same for ave1) and bound. VENC domains off at idle and after each session; 6/6 sessions and 10/10 lifecycle_test per encoder; the two-session stress and pm_test=platform runs are in media: apple: ave: add the T6021 (M2 Max) encoder #2374.
    • Control boot, same kernel and module; the boot loader differs by one log line. Only the two m2_aveN_dma nodes set to "okay": apple-ave-dma binds both (runtime PM "unsupported"), venc_sys and venc1_sys are on from boot, the first session on each encoder works, every later one fails after 10 unanswered boots (status 0x2e, "coprocessor did not power down; cannot restart it").

Not done yet

  • J414c and J416c now get the encoders but were not booted. The row for them: the loader hands over ave0 and ave1, both bind, one H.264 and one HEVC session on each, VENC domains off at idle, three suspend/resume cycles each followed by an encode.
  • A boot without the handoff (encoders unbound, DARTs bound, VENC off) has not been run with the DARTs enabled; it is in the next J475c row.

🤖 Generated with Claude Code

T6000 and T8112 run the 6070.11.1 Start and Process commands that were
qualified on hardware byte for byte. Nothing tests that those bytes
stay the same when the shared 6070 writer changes for another chip.

Add a KUnit case that builds Start and the first two Process commands
for H.264 and HEVC, at 1280x720 30/1 and 1920x1080 30000/1001, with
default session parameters, and compares their CRC32 with the values
the builder produces at this commit.

Assisted-by: Grok 4.7
Assisted-by: Claude Opus 5.5
Signed-off-by: Lucas Goossen <lucas.goossen@hey.com>
Every instance creates the debugfs directory apple-ave. On a machine
with two encoders the second one fails to create it and has no
lifecycle_test, inject_wedge or last-command files.

Name the directory apple-ave-<device name>, for example
apple-ave-40d100000.video-codec, and update the two documents that
give the old path. The path changes on every chip.

Assisted-by: Grok 4.7
Assisted-by: Claude Opus 5.5
Signed-off-by: Lucas Goossen <lucas.goossen@hey.com>
@RyanTheTide

Copy link
Copy Markdown
Contributor

Thanks. The control boot with the DMA blocks enabled explains well why those stay off, and we agree they should until #1861 lands.

One request. t6021-j475c.dts enables the two encoders and their four DARTs on the Mac Studio only, but J414c and J416c have the same T6021 blocks from t6021-peripherals.dtsi. The handoff already fails closed: the loader hands each encoder its preloaded firmware or disables the node, and without a handoff the driver refuses to bind. So could the six nodes be enabled in t6021-peripherals.dtsi instead of in the board file, keeping m2_ave0_dma/m2_ave1_dma disabled with the runtime-PM reason in a comment?

Your J475c rows cover the Mac Studio. The extra row this needs is one boot on an M2 Max MacBook Pro (J414c or J416c): the loader shows the handoff for ave0 and ave1, both encoders bind, one H.264 and one HEVC session on each, the VENC domains are off at idle, and three suspend/resume cycles each followed by an encode. #2013's author has a J414c and may be able to help.

Once #2373 is in, this branch can drop its copy of that commit.

A boot loader that hands over two encoders cannot name both DATA images
with the one /chosen aurora,ave-data-crc32 property, and the encoders
share one TEXT reservation, so its iommu-addresses property carries
one tuple per encoder.

Read aurora,crc32 from each memory-region reservation and fall back to
the /chosen property when it is absent. Accept several iommu-addresses
tuples and use the one whose phandle is this device; a reservation with
none, or with two for this device, is refused as before. Describe the
reservation property in the admin guide, next to the /chosen ones.

Every chip goes through this code. With today's boot loader (no
aurora,crc32, one tuple per reservation) it does what it did before;
only T6021 was run.

Assisted-by: Grok 4.7
Assisted-by: Claude Opus 5.5
Signed-off-by: Lucas Goossen <lucas.goossen@hey.com>
The driver requires the extra0 register window to start on a 16 KiB
boundary. On T6021 neither encoder's window does: ave0's is 0x7c44
bytes at 0x2902803e0, ave1's 0x3dfc bytes at 0x28e680228.

dma_map_resource() maps the pages that contain an unaligned range and
returns the IOVA of its first byte, which is the address Config takes.
Drop the alignment check, and map the larger of the resource size and
the variant size, so that the whole window is covered: with the
variant size alone, the last 0x24 bytes of ave0's window would be
unmapped.

T6000's window is 0x24 bytes at a 16 KiB aligned address and T8112's is
page aligned and exactly its variant size, so both map 0x4000 bytes as
before. The alignment check goes for every chip: an unaligned window is
now mapped, not refused. The only such window outside T6021 is on the
second M1 Max encoder, which the device trees leave disabled.

Assisted-by: Grok 4.7
Assisted-by: Claude Opus 5.5
Signed-off-by: Lucas Goossen <lucas.goossen@hey.com>
The M2 Max runs a larger build of the 6070.11.1 firmware the T6000 and
T8112 support uses, AppleAVE2FW_H14C: TEXT 0xf0000 and DATA 0x134000,
with the link cell at TEXT 0x423c. Its two encoders share TEXT and
each has its own DATA. The coprocessor registers, boot argument and
timebase are T8112's. With boot device ID 20 and device type 17 both
encoders of a J475c boot, answer the handshake and encode. The
identity CRC, TEXT with the link cell cleared, is 0x0d56db87; select
the 6070.11.1 layout for it.

Config on T6021 needs a shared area. With the pointer and size at
+0x60 and +0x68 left zero, as on T8112, the firmware never answers
Config. With a 64 KiB, 16 KiB-aligned area both encoders answer
Config, Open and Start. Allocate it at the first boot, keep it until
the device is removed, and pass it for a variant with config_shared.
That is a property of the H14C build rather than of 6070.11.1: the
T8112 build answers Config with both words zero.

The 6070 sessions on T6021 run the T6000/T8112 commands unchanged in
this commit: one constant-quality session per encoder.

abi.rst said 6070 streams H.264 without B pictures; the 6070 path
already offers HEVC Main and B pictures, so describe that, and say
what T6021 adds and what each value was checked against. The Kconfig
help now says what each chip offers.

Assisted-by: Grok 4.7
Assisted-by: Claude Opus 5.5
Signed-off-by: Lucas Goossen <lucas.goossen@hey.com>
A session the firmware saw keeps its buffers until the coprocessor is
proved to have lost power, and that happens only after the last
session closes. With two sessions per boot, one long-lived stream and
a second that is opened and closed again and again keep every closed
session's buffers, a full set per session, with no bound.

Count the sessions waiting for the reap. Once four wait, refuse a new
session that would join the running firmware with EBUSY and a
rate-limited warning. The refused session was never handed to the
firmware, so it is freed at once. When the last stream closes, the
power cycle frees all of them and resets the count. This holds for
every layout; 9003 already ran two sessions per boot.

Assisted-by: Claude Opus 5.5
Signed-off-by: Lucas Goossen <lucas.goossen@hey.com>
The 6070.11.1 layout allows one session per firmware boot, while the
driver already keeps the process buffers per session. Allow two, on
every SoC that runs this firmware. On a J475c, four H.264 sessions,
two on each encoder, ran in real time for five minutes at their target
rates, and a fifth STREAMON got EBUSY. T6000 and T8112 were not run
with two sessions.

When the first of two sessions closes, its buffers stay quarantined
until the last session closes and the encoder is power cycled, as on
9003.

Assisted-by: Claude Opus 5.5
Signed-off-by: Lucas Goossen <lucas.goossen@hey.com>
Two Start and Process words make 6070.11.1 H.264 P pictures much
larger than they need to be. Both were checked on a J475c (T6021):

- With the Process timescale word nonzero, every P slice had
  nal_ref_idc 0, so no P picture was a reference. With it zero, P
  slices have nal_ref_idc 1 and frame_num counts up. Submit 0 there
  for H.264; the PTS still counts in the same timescale.
- With the motion lambdas at Start 0xff98, 0xffa4 and 0xffa8 zero,
  about 94% of P macroblocks were 8x8 partitions, and constant
  quality 0 on a 720p lecture clip came out at 8.4 Mbit/s. With 64
  (full-pel search), 256 (inter mode decision) and 64 (intra mode
  decision), 8x8 became rare, the clip came out at 0.30 Mbit/s, and
  quality 50 took 1.8 times the bits of libx264 at the same PSNR.

Name the lambda offsets in the 6070 layout, and key both on a new
layout flag, h264_inter, set for 6070.11.1: they are words of that
firmware's commands, and T6000 and T8112 run the same firmware
version. Those two were not run with them; their encoded H.264
changes. The lambdas were tuned on one 720p clip and checked with
CBR at 720p and 1080p.

The pinned T6000/T8112 command bytes are built with the flag off and
do not change; a new test checks what the flag writes.

Assisted-by: Grok 4.7
Assisted-by: Claude Opus 5.5
Signed-off-by: Lucas Goossen <lucas.goossen@hey.com>
@ludagoo ludagoo changed the title arm64: dts: apple: t6021-j475c: enable the video encoders arm64: dts: apple: t6021: enable the video encoders Nov 19, 2026
@ludagoo

ludagoo commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor Author

Thanks for the review. The updates are pushed; replies to each of your comments below.

Re: enable in t6021-peripherals.dtsi; J414c/J416c row; drop the copy

Done (4001510ea360): the two encoders and four DARTs are enabled in t6021-peripherals.dtsi, the board file no longer touches them, and m2_ave0_dma/m2_ave1_dma stay disabled with the runtime-PM reason in a comment above each. All eight t602x trees build, with the same dtbs_check findings as the base (one J180d PCIe line differs only in a phandle number).

Both boots ran on the J475c with this tree (4001510ea360):

  • With handoff (r4/t6021-fw 0f591198bf7b): both encoders (/dev/video11, /dev/video12) and all four DARTs bound, dapf: Initialized /arm-io/dart-ave0 and /arm-io/dart-ave1, every venc_* / venc1_* domain off-0 at idle, 720p CQ H.264/HEVC, 4-session stress, quarantine, lifecycle_test (3x 1 and 2 per encoder), and 3x pm_test=platform freeze + post-resume encodes all passed with 0 DART faults.
  • Without a handoff (#334 head 0421b4e0):
    • Encoders fail cleanly without binding (apple-ave 40d100000.video-codec: error -ENODEV: no preloaded firmware, same for 507100000.video-codec; v4l2-ctl --list-devices shows only avd).
    • All four DARTs bind to apple-dart (40d030000.iommu, 40d040000.iommu, 507030000.iommu, 507040000.iommu), 0 DART faults, 0 failed systemd units.
    • In pm_genpd_summary, venc_me1, venc_me0, venc_pipe5, venc_pipe4, venc_dma and venc1_* are all off-0, while venc_sys and venc1_sys stay on (apple-pmgr-pwrstate ...: sync_state() pending due to 40d100000.video-codec and 507100000.video-codec, and the four DARTs remain active while their video-codec consumer never finishes probe — the same behavior as ane_sys/ane_cpu when 284000000.ane has no driver).

I do not have a J414c or J416c. The row you describe is in my notes for #2013's author if they can help. The copy of #2373's commit stays here until #2373 is merged, then goes.

6070.11.1 offers constant quality only, because the driver's CBR
value, 2, holds the Start QP on that firmware. On a J475c (T6021),
mode 1 moved slice QP with the bit rate. Without B pictures the coded
rate stayed under the target; for CBR the host's filler data then
brought 1.0 and 2.5 Mbit/s (720p) and 4.0 Mbit/s (1080p) within 15% of
the target over the file and over its last 150 frames. Submit mode 1
for CBR and VBR and offer both.

With a frame interval of 1000/30000 s every frame sat at the QP
ceiling: about 0.31 Mbit/s coded for a 1.0 Mbit/s target. Give the
firmware the frame rate rounded to an integer, over 1. The client's
timeperframe and G_PARM are not changed, and the session keeps
timeperframe exactly (client_fps_num, client_fps_den) for the VUI
timing, which must not follow the firmware's rounded rate.

A new frame rate written into the Process frame-rate words while
streaming stepped the HEVC picture clock backwards. Keep the Start
rate there; store a rate set by S_PARM while streaming in the context,
not the session (the worker frees the session without vdev_lock), and
scale the next Process bit rate and VBR limit by start rate / new rate
so the bits per frame follow. An HEVC stream at 30, then 15, then 30
frames per second kept every picture. The CBR filler budget uses the
live bit rate and frame rate, and drops a surplus above the buffer
size after a rate cut.

Key this on a new layout flag, rate_control, set for 6070.11.1: the
mode numbering is that firmware's, like rc_mode_fixed_qp beside it.
T6000 and T8112 run the same firmware version and were not run with
it. They now also offer CBR and VBR; constant quality stays the
default, and its frame rate is rounded the same way.

Assisted-by: Grok 4.7
Assisted-by: Claude Opus 5.5
Signed-off-by: Lucas Goossen <lucas.goossen@hey.com>
Default the B-frames control to 0 on a low_delay layout, and without
B-frames wait for each frame's reply before submitting the next, as
for the first frame of every session. A second session on the same
encoder waits behind that frame.

Set it for 6070.11.1. Every T6021 measurement in this series was made
this way; submitting frames ahead without B-frames was not measured
on any 6070.11.1 SoC. On T6000 and T8112 this changes the defaults:
B-frames 3 to 0, and one frame in the firmware at a time. A client
that asks for B-frames still gets them and their pipelining.

Assisted-by: Grok 4.7
Assisted-by: Claude Opus 5.5
Signed-off-by: Lucas Goossen <lucas.goossen@hey.com>
6070 firmware writes the parameter sets to a buffer of their own. On
a J475c (T6021) the SPS there had no VUI timing, and with B-frames off
no VUI at all, so a raw Annex B reader guessed 25 fps. When an SPS
from that buffer has no timing_info, rewrite it in the output buffer:
set timing in an existing VUI, or add a minimal one. The timing is the
client's timeperframe exactly, not the integer rate the firmware gets:
H.264 gets num_units_in_tick equal to the denominator and time_scale
twice the numerator, with fixed_frame_rate_flag, which is how H.264
codes a frame rate; HEVC gets vui_num_units_in_tick equal to the
denominator and vui_time_scale equal to the numerator. The rate is
taken in lowest terms, so 60/2 gives the bytes of 30/1, and 30000/1001
gives 1001 and 60000 (H.264) or 1001 and 30000 (HEVC). ffprobe on a
raw H.264 stream prints the field rate (60/1 at 30 fps) and an MP4 mux
reports 30/1.

When twice the numerator does not fit 32 bits, H.264 halves an even
denominator instead; with an odd one there is no exact form and the SPS
is left as written.

The rewrite is keyed on the stream, not on a SoC or firmware flag: an
SPS that already has timing is left alone, and so is one this walk
cannot follow (scaling lists, POC type 1, more than one HEVC
sub-layer, more than 256 bytes). Firmware that puts its parameter
sets in the coded buffer (9003, 26A428) is not touched.

The SPS is read with the RBSP reader of the T8142 slice-type probe
in aurora-silicon#2218 (struct ave_rbsp, ave_rbsp_init(),
ave_rbsp_bit(), ave_rbsp_ue()), with ave_rbsp_get() and
ave_rbsp_get_ue() added and the buffer raised to 256 bytes so a whole
SPS fits. Whichever of the two
lands second drops its copy.

Assisted-by: Grok 4.7
Assisted-by: Claude Opus 5.5
Signed-off-by: Lucas Goossen <lucas.goossen@hey.com>
…lock

The t602x trees describe the ranges at *020000 and *044000 next to
each encoder as two more DARTs. On the M2 Max (J475c) apple-dart
rejects the *044000 nodes: the stream count it reads from them is 385
at 40d044000 and 350 at 507044000, above its 256 limit. The *020000
nodes probe as DARTs.

T600x describes the same two ranges as one apple,t6000-ave-dma block,
with the DARTs at *030000 and *040000. Do the same on t6020 (ave0),
t6021 (ave0, ave1) and t6022 (ave0 to ave3), and drop the two ranges
from the encoders' iommus. The binding allows exactly these two
ranges. A maintainer may prefer an apple,t6020-ave-dma compatible in
front; the binding has none today.

The encoders on J475c ran with only the *030000 and *040000 DARTs and
showed no DART faults. t6020 and t6022 were not booted.

Assisted-by: Grok 4.7
Assisted-by: Claude Opus 5.5
Signed-off-by: Lucas Goossen <lucas.goossen@hey.com>
Enable both encoders and their two DARTs each in t6021-peripherals,
for every M2 Max machine (J414c, J416c, J475c). That is safe on a
machine nobody ran: the boot loader hands each encoder the firmware
iBoot preloaded or disables the node, and without a handoff the
driver refuses the reservation and does not bind.

The auxiliary DMA blocks stay disabled, as on every other SoC. With
them enabled, apple-ave-dma binds them; it has no runtime PM, so
ps_venc_sys and ps_venc1_sys never power off, the encoder's power
cycle does not reset the coprocessor, and after the first session's
Halt the firmware never answers a boot again. With them disabled,
every session boots the firmware afresh and the VENC domains are off
between sessions.

The encoders were run on J475c only.

Assisted-by: Claude Opus 5.5
Signed-off-by: Lucas Goossen <lucas.goossen@hey.com>
@RyanTheTide

Copy link
Copy Markdown
Contributor

Blockers: 2.

I reviewed the top commit (a38dade). It removes status = "disabled" from m2_dart_ave{0,1}_{0,1} and m2_ave{0,1} in arch/arm64/boot/dts/apple/t6021-peripherals.dtsi:810-847 and 899-936, and leaves m2_ave{0,1}_dma disabled with the reason above each. Both enabled DARTs carry their ps_venc*_sys power domain, and the encoders' iommus point only at the *030000/*040000 DARTs, which matches the DMA-block commit below.

  1. Every commit in the branch carries Assisted-by: Grok 4.7 and/or Assisted-by: Claude Opus 5.5 above the Signed-off-by. Commits should hold only the human sign-off, so drop those trailers. The PR body ends with "Generated with Claude Code"; it should end with Assisted-by: LLM (<Vendor>:<Model>) naming the tools used.
  2. With the encoders enabled in the SoC dtsi, every M2 Max booting a loader without aura#341 (still open) leaves m2_ave0/m2_ave1 enabled but unbound (-ENODEV: no preloaded firmware). Your own no-handoff J475c boot shows that holding venc_sys and venc1_sys on for the whole uptime, with sync_state() pending on both video-codec nodes. Merge this after aura#341 is in the loader the fleet boots, where nodes without a handoff are disabled. That boot should also show venc_sys/venc1_sys off-0.

HW row (besides the J414c/J416c row already requested): J475c with this head and the aura#341 loader, with the handoff withheld for one encoder. The loader must disable that node, its two DARTs must bind with 0 faults, and venc_sys/venc1_sys must reach off-0 at idle. Pass: 0 failed units and 0 new kernel errors.

Assisted-by: LLM (Claude:Opus 5.5)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants