Skip to content

media: apple: jpeg: encode 4:2:2, 4:4:4 and grey on the T6050 wrapper - #2548

Draft
m0d16l14n1 wants to merge 1 commit into
aurora-silicon:integration/aurorafrom
m0d16l14n1:j815/t8142-jpeg-422-444
Draft

m0d16l14n1 wants to merge 1 commit into
aurora-silicon:integration/aurorafrom
m0d16l14n1:j815/t8142-jpeg-422-444

Conversation

@m0d16l14n1

@m0d16l14n1 m0d16l14n1 commented Nov 20, 2026 •

Copy link
Copy Markdown
Contributor

Draft: the 4:2:2, 4:4:4 and grey values are being re-derived. The per-mode values in this head were not obtained from our own register traces. They will be replaced by values from one of two sources: a trace of macOS encoding NV16, NV24 and grey input on the hardware, as the 4:2:0 sequence was traced, or a direct probe of the engine with the decoded output checked. The code comments, commit message and this description will then describe only those traces and the measured results. Until then this stays a draft.

JPEG encode 4:2:2, 4:4:4 and grey on the T6050 wrapper, which the M5's engines also use (#2325 enables them on every M5). The wrapper refused 4:2:2 and had no 4:4:4 or grey: only the traced 4:2:0 sequence was known.

The change (its structure stays):

  • The 4:2:0/4:2:2 flag becomes a subsampling enum with a per-mode table for the T6050 wrapper (hw type 0x110010). The table holds the codec, source and IO codes, the MCU size, the tile geometry and the encoder control word.
  • NV24 and GREY join the raw formats. NV24's plane 1 is twice the luma stride, and grey has no plane 1, with its luma padded to the 8-row MCU.
  • The legacy layout keeps 4:2:0/4:2:2 only.
  • KUnit covers the table and the new raw layouts.

Tested on J815 (MacBook Air 15" M5), with the current values: NV12, NV16, NV24 and GREY at 640x480, 1920x1080 and 3840x2160 all encode and decode with the subsampling asked for, at 39.5–43.8 dB. There were no DART faults or timeouts. After the values are re-derived, the same four-format, three-size table will be rerun on the J815, plus one T6050 machine.

The commit now carries only the human Signed-off-by.

Part of the J815 bring-up; board results are summarised in discussion #135.

Assisted-by: LLM (Claude:Opus 5.5)

@RyanTheTide

Copy link
Copy Markdown
Contributor

Blockers: 2.

  1. The 4:2:2, 4:4:4 and grey values in aj_t6050_chroma[] (drivers/media/platform/apple/jpeg/apple-jpeg-hw.c:668-680) are taken from running the macOS driver's own setup code under an emulator, and the code comments (apple-jpeg-hw.c:655-669, :716; apple-jpeg-hw.h:153-156; apple-jpeg-test.c:323-328), the commit message and the PR body name its classes and methods and quote the values it maps. Values obtained by executing the vendor binary can't go into this tree or be described in public. The traced 4:2:0 sequence is fine because it comes from our own register trace. Re-derive the other three modes the same way: trace macOS encoding NV16/NV24/grey input on the hardware, or probe the engine directly and check the decoded output. Then rewrite the comments, commit and PR text to describe only the traces and the measured results. The PSNR table shows the values work; the problem is only where they came from.
  2. The commit carries an Assisted-by: Claude Opus 5.5 trailer. Keep only the human Signed-off-by in the commit, and make the PR body's last line Assisted-by: LLM (Claude:Opus 5.5).

The rest checks out. apple_jpeg_has_chroma() keeps the legacy layouts at 4:2:0/4:2:2, and apple_jpeg_build_job() refuses the other two there. apple_jpeg_raw_layout() sizes NV24 at twice the luma stride and pads grey's luma to the 8-row MCU. The enumeration, try_fmt and framesize paths share aj_raw_ok().

HW row: after re-deriving the values, the same four-format, three-size table on the J815, plus one T6050 machine, with no DART faults or timeouts.

Assisted-by: LLM (Claude:Opus 5.5)

The T6050 wrapper (hw type 0x110010, which the M5's engines also use)
refused 4:2:2 and had no 4:4:4 or grey: only the traced 4:2:0 sequence
was known. Replace the 4:2:0/4:2:2 flag with a subsampling enum and a
per-mode table for this wrapper: codec, source and IO codes, MCU size,
tile geometry and the encoder control word.

The values come from the kext. AppleJPEGDriver::startEncoder maps the
user subsampling to (codec, IO code) pairs: 4:2:0 (3,2), 4:2:2 (1,1),
4:4:4 (0,0), grey (4,3). Running AppleJPEGWrapperControl::encoderHWSetup
(27.2.1, V14 wrapper) on an emulated register page reproduces the traced
4:2:0 sequence in order and value for every non-table word, and gives the
remaining words for the other inputs. Grey clears USE_PLANE1, as the kext
does for L008 input.

Add NV24 and GREY to the raw formats with the kext's plane-1 stride
rules (2 x width for 4:4:4, none for grey). The legacy layout keeps
4:2:0/4:2:2 only. KUnit covers the table and the new raw layouts.

Tested on a MacBook Air 15" M5 (J815, T8142 engines programmed as on
T6050): one frame per format and size, decoded with ffmpeg and compared
with the input:

            640x480    1920x1080   3840x2160
  NV12      39.5 dB    41.4 dB     41.7 dB
  NV16      40.6 dB    42.8 dB     43.2 dB
  NV24      42.2 dB    43.5 dB     43.8 dB
  GREY      41.8 dB    42.9 dB     43.2 dB

Each file decodes as 4:2:0, 4:2:2, 4:4:4 or grey as asked, with no
DART faults or timeouts. T6050 itself has not run it.

Signed-off-by: Nick <nekreshen@yandex.ru>
@m0d16l14n1
m0d16l14n1 force-pushed the j815/t8142-jpeg-422-444 branch from d7ac310 to 6a8b461 Compare November 21, 2026 12:51
@m0d16l14n1
m0d16l14n1 marked this pull request as draft November 21, 2026 12:52
@m0d16l14n1

Copy link
Copy Markdown
Contributor Author

@RyanTheTide understood. I've moved this to draft. The 4:2:2/4:4:4/grey values will be re-derived from a hardware trace of macOS encoding NV16/NV24/grey, or from a direct engine probe. After that, the comments, the commit and the body will describe only those traces. The trailer is fixed now, and the body no longer names the driver's classes.

Assisted-by: LLM (Claude:Opus 5.5)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants