Skip to content

arm64: dts: apple: t6050: enable the media engines on every board - #2278

Draft
RyanTheTide wants to merge 2 commits into
integration/aurorafrom
dtfill-t6050/media
Draft

RyanTheTide wants to merge 2 commits into
integration/aurorafrom
dtfill-t6050/media

Conversation

@RyanTheTide

@RyanTheTide RyanTheTide commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Problem

The video encoder (ave0), the first ProRes engine (apr0) and both JPEG engines are enabled only on the J714s. The video decoder (avd0) is enabled only on the J714c, J714s, J775d and K116s. Every T6050 board's ADT has these engines at the same addresses, with the same interrupts, DART streams and power states. So the J716c, J716s, J775c, J873s, K114c, K114s, K116c and K116s have no hardware video decode, and no board but the J714s has hardware encode, ProRes or JPEG.

Change

  • New t6050-media.dtsi: the J714s enables for the decoder, encoder, ProRes and JPEG engines, their DARTs and power states (pmgr4, FAB1_SOC always-on, AVD_SYS, FAB0_PIOGW and FAB7_SOC firmware-owned, SCODEC, FAB1_PIOGW firmware-owned, FAB0_SOC always-on, VENC0_*, JPG).
  • Every T6050 board includes it after t6050-peripherals.dtsi. The J714s, J714c and K116s lose their copies of these blocks.
  • J775d: the decoder part is skipped (#ifndef T6050_J775D), because t6050-ultra.dtsi already describes both dies' decoders enabled.
  • Every DART enabled here carries its engine's power state: dart-ave0 (both instances) VENC0_SYS, dart-apr0 SCODEC, dart-jpeg0 JPG, dart-avd0 AVD_SYS.
  • ave1 and apr1 (the M5 Max's second encoder and ProRes engine) stay disabled. The boot loader hands over only the first T6050 encoder's preloaded firmware (aura dt_set_ave() takes the first apple,t6050-ave node and ADT ave0), and the AVE driver refuses an encoder without its two preloaded segments (ave-fw.c ave_fw_probe()). The ProRes driver's T6050 variant needs the first engine's core/wrapper/aux/ctrl windows and SCODEC rows. apr1 is in the PRORES power state with a different window layout, and no variant describes that. The new file says this in a comment.
  • The AVE firmware preload is handed over by compatible, not by board, so no loader change is needed for the new boards. AVD uses the open firmware file (apple/avd-fw-v5-t2.bin). ProRes and JPEG need no firmware.

The J714s blocks were written by DjDeveloperr, the J714c decoder block by Eduard Moldovan, and the K116s decoder block by Jens Axboe.

Testing

  • Full kernel tree make dtbs CHECK_DTBS=y (dtschema) for every T6050 DTB on this branch: builds clean, no dtc warnings, and no schema message that integration/aurora does not already have.
  • DTBs built for all 11 T6050 boards with kbuild's cpp and dtc flags. No dtc warnings before or after.
  • J714s: byte-identical DTB with the first commit; the second commit adds only the three reg-names properties.
  • All other boards changed. On every board, each media node now has the same status, apple,always-on and apple,firmware-owned as on the J714s. The only change to an already-enabled node is that the J714c's FAB1_SOC gains apple,always-on, which the J714s, K116s and J775d already have.
  • dt-validate: the second commit names the second register window of dart-ave0 (both instances) and dart-jpeg0 (dapf / auxiliary, from the DART instances in the ADT), which the binding requires. That removes the 3 reg-names messages the J714s already had, and no board gains a message. No new compatibles. The driver behaves the same: it uses a dapf window only with apple,dapf-ranges, and never uses auxiliary.

Still to run (owner rows, per board)

Same four rows on every board. Device numbers come from v4l2-ctl --list-devices.

  1. Decode: H.264, HEVC and AV1 clips through gst-launch-1.0 filesrc ! parsebin ! v4l2slh264dec|v4l2slh265dec|v4l2slav1dec ! video/x-raw,format=NV12 ! filesink. The output must be bit-exact (md5) with ffmpeg -i <clip> -f rawvideo -pix_fmt nv12. Use real content, not test-source clips.
  2. Encode: gst-launch-1.0 videotestsrc num-buffers=600 ! video/x-raw,format=NV12,width=1920,height=1080,framerate=60/1 ! v4l2h264enc ! h264parse ! mp4mux ! filesink location=ave.mp4. Expect a 10 s file with 600 frames in ffprobe. dmesg must show the encoder probing with a known firmware build.
  3. ProRes: one frame through tools/asahi/prores-enc on the apr0 node. ffmpeg must decode the output.
  4. JPEG: gst-launch-1.0 videotestsrc num-buffers=1 ! video/x-raw,format=NV12,width=1920,height=1080 ! v4l2jpegenc ! filesink location=t.jpg on each of the two JPEG nodes. djpeg t.jpg must decode.

Owners: J714s DjDeveloperr (no-regression only). J714c Eduard Moldovan. J716c erikwb. J716s rolotrealanis98. J775c DjDeveloperr. J775d funkzy. K114c joshuaswarren. K116s axboe. J873s, K114s and K116c have no owner yet.

Dependencies:

  • Encode needs a boot loader with the T6050 AVE handoff (aura main has it), on a boot whose iBoot preload is a build the driver admits. The driver admits macOS 27.0 (26A428) and 27.3 beta 1 (26C5035). A board running a 27.2 stub needs the 27.2 T6050 build admitted in drivers/media/platform/apple/ave/ave-drv.c (ave_t6050_builds). No open PR does this.
  • J775c: arm64: dts: apple: support M5 Max Mac Studio (J775c) #2102 replaces t6050-j775c.dts with a flattened tree in which these nodes stay disabled. Whichever lands second needs the enables (or the include) carried over.
  • K116s: T6050: firmware-driven deep sleep on the K116s (16-inch M5 Pro, firmware 27.2) #2201 enables FAB1_PIOGW, FAB0_SOC and VENC0_SYS at the end of t6050-k116s.dts for deep sleep. The values match this file. The text conflicts with the removed decoder block, and resolving it is trivial.
Per-board values and DTB diffs

Values: every board's ADT has avd0/dart-avd0, ave0/dart-ave0, apr0/dart-apr0, jpeg0/jpeg1/dart-jpeg0 with the same reg, interrupts, power and clock gates as the J714s (compared across all 11 boards). The node descriptions themselves are unchanged in t6050-peripherals.dtsi and t6050-ultra.dtsi.

Nodes enabled per board (status disabled -> okay unless noted):

  • J714s: none (byte-identical).
  • Common set (call it E): ave0, dart-ave0 x2, apr0, dart-apr0, jpeg0, jpeg1, dart-jpeg0. FAB7_SOC, FAB1_PIOGW (firmware-owned), FAB0_SOC (always-on), SCODEC, JPG, VENC0_SYS/DMA/PIPE4/PIPE5/ME0/ME1.
  • J714c: E, FAB0_PIOGW (firmware-owned), and FAB1_SOC gains always-on.
  • J716c: E, plus pmgr4, FAB1_SOC (always-on), AVD_SYS, avd, dart-avd0. FAB0_PIOGW was already enabled.
  • J716s, J775c, J873s, K114c, K114s, K116c: as J716c, plus FAB0_PIOGW (firmware-owned).
  • J775d: E and FAB0_PIOGW (firmware-owned). The decoders were already enabled by t6050-ultra.dtsi.
  • K116s: E. The decoder and FAB0_PIOGW were already enabled.

ave1, dart-ave1 x2, VENC1_*, apr1, dart-apr1 and PRORES stay disabled on every board.

DTB diffs (decompiled, phandles normalised) are in the branch's build evidence. Each one only flips status and adds apple,always-on/apple,firmware-owned on the nodes above.

Assisted-by: LLM (Claude:Opus 5.5)

The video encoder (ave0), the first ProRes engine (apr0) and both JPEG
engines were enabled only on the J714s, and the video decoder (avd0) only
on the J714c, J714s, J775d and K116s. Every T6050 board's ADT has these
engines at the same addresses, with the same interrupts, DART streams and
power states, so describe their enables once.

Add t6050-media.dtsi with the J714s enables for the decoder, encoder,
ProRes and JPEG engines, their DARTs and their power states, and include it
from every T6050 board after t6050-peripherals.dtsi. The J714s, J714c and
K116s lose their copies of these blocks. The J775d keeps the decoders that
t6050-ultra.dtsi describes enabled; the shared file leaves them alone there.

Every DART enabled here already carries its engine's power state
(dart-ave0 both instances: VENC0_SYS, dart-apr0: SCODEC, dart-jpeg0: JPG,
dart-avd0: AVD_SYS). The encoder's firmware preload is handed over by the
boot loader by compatible, so it needs no per-board change.

The second encoder (ave1) and the second ProRes engine (apr1) of the M5
Max stay disabled: the boot loader hands over only the first encoder's
preloaded firmware, which the AVE driver requires, and the ProRes driver's
T6050 variant only describes the first engine's window layout.

DTBs: the J714s is byte-identical. The J714c gains apple,always-on on
FAB1_SOC, as the J714s, K116s and J775d already have. Every other board
gains whichever of these engines it lacked, so all eleven now enable the
same media nodes as the J714s.

The J714s blocks were written by DjDeveloperr, the J714c decoder block by
Eduard Moldovan and the K116s decoder block by Jens Axboe.

Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
…DARTs

The apple,dart binding requires reg-names once a DART has two register
windows. ave0_dart, ave0_dart1 and jpeg0_dart have two each but no names,
so every board that enables them fails dtbs_check.

Name them after the DART instance each window is in the board's ADT:
ave0_dart's second window is its address filter ("dapf"); ave0_dart1's
second window and jpeg0_dart's (a shared gateway page) are neither a
DAPF nor a GAPF, so they are "auxiliary".

No functional change: the driver reads a "dapf" window only together
with apple,dapf-ranges, which these nodes do not have, and it does not
use "auxiliary" windows.

Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
@satchlj

satchlj commented Nov 19, 2026

Copy link
Copy Markdown
Member

Claude Fable 5.2 on @satchlj's behalf. J775c (Mac Studio M5 Max) probe row for this PR at head 93af6fd, one one-shot boot, nothing installed. OS firmware 27.0 (26A428), system firmware 27.2. 0 failed units.

How it was booted, since it is not the PR tree as-is: the trial used a bootable J775c DTB carrying this PR's media bindings and enable properties, with references remapped to the board tree, and all four media drivers built in. The SD-card-reader PCI probe was skipped on the command line to get past a separate early boot failure in that path that we reproduce independently on the current integration tip.

Block Platform device Probe result
Decoder (AVD) 2309080000.avd probed; missing apple/avd-fw-v5-t2.bin (-2); unbound
Encoder (AVE) 2283100000.video-encoder probed; no preloaded firmware (-19); unbound
ProRes (APR0) 48d000000.prores bound, hardware ID 0x40018, video12
JPEG 0 and 1 2281000000.jpeg, 2281060000.jpeg both bound, hardware revision 0x110010, video10 and video11
  • The encoder never reached its firmware build admission check because the loader handed it no preloaded image, so this row says nothing about which encoder builds are admitted.
  • New media lines against our default boot: the AVD firmware-load failure, the resulting AVD probe failure, and the AVE no-preload failure. A few other trial-only lines (three CPU feature masks, one AOP probe failure -61) came with a different kernel and DTB and cannot be attributed to this change.
  • Not run: any decode or encode workload, and a boot on the 27.2 handoff.
  • Console recorded; warm return to the default was clean.

@RyanTheTide

Copy link
Copy Markdown
Contributor Author

Thanks @satchlj for the J775c row. It shows the media nodes this PR enables are described well enough to probe on the J775c: ProRes (apr0) and both JPEG engines bind and register their video nodes, and the decoder and encoder reach their drivers.

Still open for the J775c:

  • The decoder stopped at its missing firmware file (apple/avd-fw-v5-t2.bin), and the encoder got no preloaded image from the loader, so neither ran. A boot through aura main with the decoder firmware installed would get both further.
  • The DTB was remapped onto the board tree rather than built from this branch as-is, and arm64: dts: apple: support M5 Max Mac Studio (J775c) #2102 replaces the J775c tree (whichever lands second carries the enables over). A boot on the final J775c tree is still needed.
  • The four workload rows from the description (decode md5, 600-frame encode, one ProRes frame, JPEG on both nodes) have not run on any board yet, and the other boards' owners have not posted.

The PR stays a draft until those are in.

@DjDeveloperr

Copy link
Copy Markdown
Contributor

J714s audit & T6050 AVE dependencies check against current integration/aurora (a84637dca209):

  1. DTB verification (clean, no rebase needed):

    • Clean 3-way merge (git merge-tree) of 93af6fdc9d68 onto integration/aurora a84637dca209.
    • All 11 t6050-*.dtb boards compile with 0 dtc warnings.
    • t6050-j714s.dtb decompiled diff vs integration/aurora is strictly the 3 reg-names properties ("dart", "dapf" on ave0_dart, "dart", "auxiliary" on ave0_dart1 and jpeg0_dart). In drivers/iommu/apple-dart.c (apple_dart_dapf_parse()), "dapf" without apple,dapf-ranges returns 0 immediately and leaves the loader's DAPF setup untouched.
    • All 10 other T6050 boards (j716s, j873s, j714c, j716c, j775c, k114s, k116s, k114c, k116c, j775d) flip ave0 (video-encoder@2283100000) from disabled to okay; ave1 stays disabled.
  2. Row drm/apple: release a stale DPTX link before borrowing a Type-C route #2 (framerate=60/1, no B-frames) note for testers:

  3. macOS 27.2 (26B123 and 26B8125) T6050 AVE firmware builds:

    • Checked AppleAVE2FW_H17D segment sizes and __TEXT CRC32 across the public macOS 27.0 and 27.2 restore bundles:
      • 26A428 (27.0, already admitted): TEXT 0x154000, DATA 0x134000, TEXT CRC 87279429
      • 26B123 (27.2 — j714s/j716s/j873s/j714c/j716c/j775c/j775d, not yet admitted): TEXT 0x158000, DATA 0x134000, TEXT CRC d33ed3b4
      • 26B8125 (27.2 — k114/k116, not yet admitted): TEXT 0x158000, DATA 0x134000, TEXT CRC 1e27a926
      • 26C5035r (27.3b1, already admitted): TEXT 0x158000, DATA 0x134000, TEXT CRC c25b4ddb
    • Because 26B123, 26B8125, and 26C5035r all share the exact same 0x158000/0x134000 segment sizes, admitting the 27.2 T6050 builds in ave_t6050_builds[] also depends on media: apple: ave: T8142: admit the macOS 27.2 (26B123) encoder firmware #1900's (text_size, data_size, text_crc) build-table disambiguation (plus a live hardware encode row on a 27.2 T6050 stub to confirm ave_layout_26a428 vs ave_layout_26b123).

@funkzy

funkzy commented Nov 22, 2026

Copy link
Copy Markdown
Contributor

J775D (M5 Ultra Mac Studio) row on 93af6fd, with #2422: the engines probe, and the encoder is refused at probe.

Setup: the 27.0 stub, a read-only root. Linux is integration/aurora cacf355 (2026-11-21) with #1650, #1973 and the J775D's test set, plus this PR, #2422 and #2291. The boot loader is aura main dc2313f8 plus #333 and one commit outside the media path. Three boots with this PR, and one baseline of the same kernel without it.

  • ProRes and JPEG: apple-prores 48d000000.prores: ProRes encoder, id 0x40018 and both apple-jpeg ... JPEG encoder, HWREV 0x110010, with video nodes. ProRes and JPEG encodes were not run.

  • Decoders: unchanged. Both of the J775D's AVD nodes bind as before.

  • Encoder: the boot loader hands ave0 over with its firmware segments reserved. The driver then refuses it: apple-ave 2283100000.video-encoder: error -ENODEV: unsupported firmware (text CRC 26a46d96). There is no encoder node, so the encode rows could not run.

  • The checksums change on every boot. The TEXT/DATA CRCs the boot loader exported differ on all four boots of this window:

    • 26a46d96/e2fff326 and d5fa98b3/8fba324d, two boots of the same kernel and boot loader;
    • 62ba33d7/0a0f1c75 and 04d65f21/b14a624e on the other two.

    So the preloaded segments do not hold a stable image when the boot loader reads them on this machine. The driver's refusal is the right outcome, and admitting one of these CRCs would be wrong. Whether no image is there on this boot path, or something overwrites it before the handoff, is not known yet. We'll look at the segment contents at the boot loader next.

  • Power: five VENC0 power states report sync_state() pending due to 2283100000.video-encoder, because the encoder never binds.

  • Regression: apart from the encoder line, no kernel error line is new against the baseline (the Type-C usb 3-1 line also shows on earlier boots without these PRs). 1 failed unit on each boot, the read-only root's networkd persistent-storage unit, as on the baseline. WARN/BUG/Oops 0. ALARM has no SELinux, so there are no AVCs to count.

Merge note: on a tree with #1973, this PR's first commit conflicts in t6050-j775d.dts. Both add an include at the end of the file (t6050-media.dtsi here, t6050-j775d-gpu.dtsi there). Keeping both resolves it, and that is how we tested.

Assisted-by: LLM (Claude:Opus 5.5)

@satchlj

satchlj commented Nov 22, 2026 •

Copy link
Copy Markdown
Member

Claude Opus 5.5 on @satchlj's behalf. J775c functional row for this PR's ProRes part. It follows the probe row of 2026-11-19.

Setup: lobata's flattened J775c DTS with this PR's apr0 changes applied: apr0, its DART, scodec okay, and fab7-soc/fab0-piogw firmware-owned. The kernel is cand-15 with #2620 merged; default cmdline, one-shot boots.

Result: apr0 probes as ProRes codec, id 0x40018, with encode and decode nodes. Decode gives SHA 6/6, unchanged after s2idle, plus encode, an 800-frame ring wrap, and s2idle with jobs in flight. No DART fault, failed units 0. Details: #2620 (comment).

Without #2620, the same DTB probes as ProRes encoder: the encode node appears and there is no decode node. Encode was not run on that boot. So the J775c DT part here is enough to probe the engine, and decode also needs #2620.

Assisted-by: LLM (Anthropic:Claude Opus 5.5)

@funkzy

funkzy commented Nov 22, 2026

Copy link
Copy Markdown
Contributor

J775D follow-up to our row: the encoder's firmware is the 26A428 build the driver already admits. Only the build check refuses it.

  • With the T6041 variant's text_boot_word applied, TEXT's CRC on the J775D is 87279429 (AVE_FW_TEXT_CRC_T6050_26A428) on both boots we checked.
  • ave_t6050 sets no text_boot_word, so ave_fw_text_id() compares the unmodified CRC, and that does not match the admitted CRC on the J775D.
  • Setting text_boot_word on the T6050 variant, as on T6041, would select the admitted build here. A boot where that word is already zero gives the same CRC with or without it.

That variant is dj's, and text_boot_word is 7robertodantas's T6041 mechanism, so we've left the change to them. We'll add the J775D encode row once a kernel has it.

Assisted-by: LLM (Claude:Opus 5.5)

@RyanTheTide

Copy link
Copy Markdown
Contributor Author

Probe rows are in on the J775c and J775D, and ProRes encode and decode pass on the J775c together with #2620. Still needed before this leaves draft: a decoder md5 row and a JPEG encode row on a board built from this branch, and an encoder row once the T6050 encoder firmware check is fixed in the driver.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants