Skip to content

media: apple: isp: H17 stream settings, lens-circle crops and crash light for the J813/J815 IMX958 - #2843

Closed
m0d16l14n1 wants to merge 12 commits into
aurora-silicon:integration/aurorafrom
m0d16l14n1:j815/isp-camera-settings
Closed

m0d16l14n1 wants to merge 12 commits into
aurora-silicon:integration/aurorafrom
m0d16l14n1:j815/isp-camera-settings

Conversation

@m0d16l14n1

Copy link
Copy Markdown
Contributor

Draft. This replaces #2645, without MLVNR. It is stacked on #2194 (988af2c7) on integration/aurora bb9f8ec, and only the last nine commits are new. It is a fresh branch, as asked in the #2645 review, so none of its history contains the MLVNR code or the firmware-derived input table. MLVNR will come back as its own PR, rewritten from a hypervisor trace and its write-up (see #2647).

1. macOS's stream settings on H17 (52da9a9d, made fixed in f4366f78)

These are the commands and arguments macOS 27.2 sends the same J815 ISP firmware, taken from a hypervisor IPC trace. They started as test parameters, and after review they are fixed behaviour on H17:

  • AE: stability 16/8 (other firmware keeps 32/20) and an AE gain cap of 96.0.
  • TNR: on band-0 setfiles, the start record is (1, 1, 0) with high strength. isp_test_temporal_filter_band0 now expects that.
  • Camera LED (T8142 only, 84243371):
    • CH_STATUS_LED_MBRIGHTNESS_SET is sent at macOS's start value, 529;
    • it is clamped to the range CH_STATUS_LED_THR_GET reports (204..1023 on the J815), so the LED can never be set dark.
  • New commands, as packed structs with size asserts: CH_AE_GAIN_CAP_SET, CH_STATUS_LED_MBRIGHTNESS_SET and CH_STATUS_LED_THR_GET.
  • Removed:
    • the LPDP tuning, which made no measurable difference in the J815 A/B;
    • the bench-only sensor modes 7 and 9;
    • highlight recovery and the SIFR skip interval. They only apply in a SIFR mode, and the presets stay in the DT's binned config 8, which is not one. They come back with a full-resolution mode.

2. Crop inside the lens circle (f6704110, 81878d47; binding c68edc73, dtsi 1e18af37)

The J813/J815 ultra-wide lens draws an image circle smaller than the sensor. The DT crops take the full sensor width, so they show black corners.

  • Measured on one J815: the 1520x856 16:9 crop goes black at 0.855 and 0.895 of its half-diagonal at the bottom corners, and at 0.96 at the top.
  • The fix: apple,lens-circle gives the radius, per mille of the input width (470 on the J813/J815). Each crop is scaled about its centre until its corners are inside the circle.
    • 16:9 now crops 1242x700 of the binned frame (1.22x).
    • 4:3 crops 1140x856.
    • The portrait crops are unchanged, and each changed crop is logged.
  • Scope: only boards whose DT has the property. The J813 is assumed to have the J815's lens; that is not measured.
  • Sensor mode: the presets keep config 8, the DT's binned mode. In the J815 A/B its centre luma noise was 1.29, against about 3.0 at full resolution.

3. Camera light after a firmware crash (69dc771e)

A crashed ISP firmware is forced off, so it never powers the camera module down. The sensor and its green light then stay on until the next camera use; this was seen on the J815 after an ISP assert.

  • Where the firmware can start again (hw->fw_restart), a halt after an IPC error now boots it once more and stops it the normal way.
  • This runs once.
  • On hardware, the light went off.

4. Host tests (b095664b)

isp-preview-flow-tests.py and the isp-tests.py session suite get inert stubs for the new calls. The checks themselves are unchanged.

Provenance

  • Section 1: a hypervisor IPC trace of macOS 27.2 streaming on this J815: the command payloads and arguments.
  • Command names: the ISP firmware prints each command's name in its log when it receives one. All three new names appear in such logs: Linux dmesg on the J815 for CH_AE_GAIN_CAP_SET, and macOS's log of the same firmware for the two LED commands.
    • To be complete: when I first named these opcodes, I looked them up in the name table behind that log line, extracted from the firmware image. The names now rest on the logs alone, and nothing else here came from the image.
  • Section 2: measurements on the J815.
  • Section 3: observed on the J815.

Apart from that first name lookup, nothing here was read from macOS binaries or the firmware image.

Testing

  • ISP KUnit (arm64, QEMU, LLVM, fresh build dir): 61 tests, 60 pass. The one failure, isp_test_sensor_sysfs, also fails on plain bb9f8ec (59 tests, 58 pass).
  • Host tests: isp-tests.py, isp-preview-tests.py and isp-preview-flow-tests.py pass with clang 19 and ASan/UBSan.
  • checkpatch --strict: clean on all nine commits.
  • DTBs: J813 and J815 build.
  • Hardware, J815, earlier code (the test-parameter form):
    • every mode streamed at 30 fps;
    • the lens crop removed the black corners (dmesg: crop 1520x856+0+292 -> 1242x700);
    • the crash recovery turned the light off.
  • Owed at this head, on the default command line:
    • a J815 row: stream at 30 fps, the LED value inside the firmware's range, the crop lines, no command timeouts;
    • the regression line: 0 SELinux denials, 0 failed units, 0 new kernel errors;
    • a J813 row.

Assisted-by: LLM (Claude:Opus 5.5)

Some Macs pair a sensor with a camera module that has its own tuning
file. Let the ISP node name it, e.g. "2226_02" for the J815's IMX958.

Signed-off-by: Nick <nekreshen@yandex.ru>
The J813 and the J815 (MacBook Air 13" and 15" M5) both have an IMX958
sensor, and both have run its default 2224_01 setfile. The J815's camera
module has its own tunings, 2226_01 and 2226_02, in the H17-native form
(lens-shading records without the xx24 stride). With 2224_01 the J815's
raw picture is dark in the corners. Corner/centre brightness on one
J815, raw camera, same scene, each file loaded into a freshly started
firmware:

  2224_01  0.50
  2226_01  0.63
  2226_02  0.85

Load the tuning the ISP node's apple,module-tuning names. A board
without the property, or with a name that is not a setfile of its
sensor, keeps the sensor's default. The 2226 files are not in every
vendor firmware set yet, so a missing one falls back to 2224_01 instead
of leaving the camera untuned.

Tested on J815, with the file picked by a module parameter at the time:
all three files load and stream (no command timeouts), 2226_02 also
across several camera restarts.

KUnit: isp_test_module_tuning (match, another sensor, unknown names)
and isp_test_setfile_fallbacks (each fallback is the same sensor, no
larger, and final).

Signed-off-by: Nick <nekreshen@yandex.ru>
The J815's IMX958 module runs 2226_02: with the sensor's default
2224_01 its corners read half the centre's brightness, with 2226_02
0.85 of it.

Signed-off-by: Nick <nekreshen@yandex.ru>
Add the ISP commands macOS 27.2 sends at a J815 stream start (from a
hypervisor IPC trace) and test parameters that send them, for an A/B on
the J815: binned SIFR with highlight recovery (config 9), the AE and
gain settings, and the camera status LED. A later commit in this series
makes the useful ones fixed behaviour and drops the parameters.

Signed-off-by: Nick <nekreshen@yandex.ru>
…nned SIFR default

The J813/J815 ultra-wide lens draws an image circle smaller than the
sensor; the DT crops take the full sensor width and show black corners
(measured on a J815: the 16:9 crop goes dark at 0.855-0.96 of its
half-diagonal). lens_circle (per mille of the input width, default 470)
and lens_center_y scale a crop about its centre until its corners are
inside the circle: 1920x1080 and 1280x720 crop 1242x700 of the binned
1520x1440 frame instead of 1520x856 (1.22x).

sensor_config now defaults to 9 (binned + SIFR + highlight recovery):
config 6 measured 2.4x the centre luma noise of the binned modes.

Signed-off-by: Nick <nekreshen@yandex.ru>
Config 9 + HR measured 1.52 centre luma noise against 1.29 for config 8
on the J815 A/B; for a grain complaint keep the DT's binned mode and
leave 9 as a knob. Log each crop the lens circle changes, so a run shows
that it applied.

Signed-off-by: Nick <nekreshen@yandex.ru>
A crashed firmware is forced off: the stop command times out ("coproc
not in WFI"), so the firmware never sends the camera module its
power-down. The sensor stays powered, and its green status light stays
on until the next camera use. Seen on J815 (2026-11-21) after an ISP
firmware assert.

On hardware whose firmware starts again (hw->fw_restart), a halt after
an IPC error now schedules fw_recover_work. It boots the firmware once
more and stops it the normal way, which
powers the module down. It runs once: a second crash is left as it is.
On J815 the light went off after the crash, and the log said "crashed
firmware: restarted and stopped it to power the camera down".

Signed-off-by: Nick <nekreshen@yandex.ru>
The host tests compile driver functions against fake boundaries, so they
stopped building with this series:

- isp-preview-flow-tests: isp_ch_configure_capture() now reads the
  knobs that match macOS (isp_lpdp_macos, isp_macos_tune) and the
  remapped preset flag, and calls the SIFR and camera LED helpers. They
  get inert stubs; the order this test checks is unchanged.
- isp-tests session: the firmware paths read the crash recovery work
  and flag. The fixtures have no fw_restart firmware, so recovery is
  never scheduled and the idle-work check stays as strict as before.

Signed-off-by: Nick <nekreshen@yandex.ru>
An optional property of the sensor-presets node, for what belongs to the
camera module rather than to the driver: apple,lens-circle, the radius
of the image circle the lens draws on the sensor, per mille of each
preset's input width. The J813/J815 ultra-wide lens is black outside
it, so the crops are kept inside.

Signed-off-by: Nick <nekreshen@yandex.ru>
Ryan Murray's review: merged code picks its behaviour itself, and what
belongs to the sensor or the lens goes in the camera's DT node. So the
TEST module parameters go:

- macos_tune, led_brightness: what macOS 27.2 sends the H17 firmware on
  the J815 is now fixed behaviour on H17: AE stability 16/8 and an AE
  gain cap of 96.0; TNR (1, 1, 0) with high strength on band-0
  setfiles; and the camera LED at macOS's start brightness, 529. The
  LED value is still clamped to the range the firmware reports, so the
  privacy indicator can never be set dark. Other firmware keeps its old
  values. isp_test_temporal_filter_band0 now expects (1, 1, 0).
- hr: highlight recovery needs a SIFR sensor mode, and the presets stay
  in the DT's binned mode, which is not one. HR, the SIFR skip interval
  and their two commands go until a full-resolution mode is used.
- lpdp_macos: the 0x18-byte LPDP tuning made no measurable difference
  in the J815 A/B, so it goes with its command; the driver's (1, 15)
  stays.
- sensor_config, presets_full: the presets keep the DT's binned mode
  (config 8): in the J815 A/B its noise was 1.29 against about 3.0 at
  full resolution. Configs 7 and 9 were bench modes only and go.
- lens_circle, lens_center_y: the radius comes from apple,lens-circle.
  The centre shift was never used and goes.

Signed-off-by: Nick <nekreshen@yandex.ru>
The ultra-wide IMX958 module's lens draws an image circle of radius
0.47 of the sensor width (measured on a J815; the J813 is assumed to
have the same lens).

Signed-off-by: Nick <nekreshen@yandex.ru>
macOS's start brightness (529) was traced on the J815 only. Other H17
firmware keeps its own default until it is measured there: a per-SoC
hw->led_brightness, set for T8142, replaces the H17-wide check.

Signed-off-by: Nick <nekreshen@yandex.ru>
@m0d16l14n1

Copy link
Copy Markdown
Contributor Author

Closing this: I'm taking the J815 camera work out of the upstream trees for now and continuing it on my own machine. Thanks for the reviews.

Assisted-by: LLM (Claude:Opus 5.5)

@m0d16l14n1 m0d16l14n1 closed this Nov 22, 2026
@m0d16l14n1
m0d16l14n1 deleted the j815/isp-camera-settings branch November 22, 2026 19:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant