Repository navigation
accel/ane: run vendor-compiled camera programs: long HWX names, several outputs, wide LOAD - #2647
m0d16l14n1 wants to merge 1 commit into
Conversation
6c37c7c to
9b18f64
Compare
1c5e53a to
b91ac5e
Compare
|
J815 run, 2026-11-21: MLVNR streams with the IO_T2H change (this ANE build as the boot driver, the ISP side from #2645 as a module, sensor config 6)
Noise: the two Open: macOS sends the same auto-mode words ( |
|
Read the HWX and program side with untrusted input in mind. The new parsing stays bounded:
On the MLVNR side, the AFPP image is size-capped ( Two notes:
Assisted-by: LLM (Claude:Opus 5.5) |
|
Thanks. Both are in f8947c9, which changes comments only:
Assisted-by: Claude Opus 5.5 |
|
One thing to settle before this merges: at f8947c9
The same applies to the ISP side's last parameter ( Assisted-by: LLM (Claude:Opus 5.5) |
|
Thanks for the MLVNR work and the detailed J815 rows. Two things, on top of the module-parameter note above:
Happy to run no-regression rows on our M1 Pro / M2 / M4 Pro / A18 Pro once the parameters are gone. |
DjDeveloperr
left a comment
There was a problem hiding this comment.
Adversarial review against pre-pr-checklist.md (§1–§6) at f8947c905533 (host-tests: green).
Part 1 (long HWX names, multi-output programs, ANE_H16_LOAD_WIDE, heap-allocated parser temporaries, minor 2 UAPI) is bounded and clean in structure, but several hard checklist blockers and code bugs in Part 2 must be resolved before merge.
Blockers (§4 — Attribution & clean-room)
- AI trailer inside all 5 git commits (§4): Commits
e1a1a460,33e01b08,292d7e54,b91ac5e7, andf8947c90all containAssisted-by: Claude Opus 5.5in their commit messages. Under §4, commits must carry human identity and humanSigned-off-by:only — noAssisted-by:(or other AI trailers) inside commits (keepAssisted-by:at the end of the PR body only). - Apple macOS driver internal symbol names and firmware assert strings in public code/commits (§4): None of these exist in
integration/aurora's baselineane_fw_cold.c/ane_h16_protocol.hand must be removed/reworded as clean-room wire observations (or dropped if sourced from Apple binaries):drivers/accel/ane/ane_fw_cold.c:2105-2106,2135,2143,2147,2158,2180,2271(AppleH16ANEInterface,ANEFirmwareManager,ANE_IPC_PowerOnByPeer,ANEHWDevice::handleISPClientMCacheRequest, exact firmware assert string"pSwitchedOutReq->request.mcacheDsid != (0xFF)",ANEHWDevice::ANE_processISPSurfaceMapRequest,DecodeFirmwareAllocationUsageField,ANE_processISPSurfaceUnMapRequest).drivers/accel/ane/ane_fw_transport.h:108,739anddrivers/accel/ane/ane_fw_transport_test.c:788(ANE_processISPSurfaceMapRequest,ANE_sendISPSurfaceRemapAck).drivers/accel/ane/ane_h16_protocol.h:18-20,27-30,34-38(AppleH16ANEInterface,ANEFirmwareManager::sendAFPPLoadCmd,sendIPCEndpointSetCmd,getMCacheSize,handleISPClientMCacheRequest,setMCacheDSIDs).- Commit messages (
33e01b08,b91ac5e7) also referenceAppleH16ANEInterface,ANEFirmwareManager::..., and firmware source/line strings (CRPCClient.cpp:303,CAneClient.cpp:1375).
Blockers (§2 / §3 — Production-ready, no hacks / module parameters / hardcoded macOS values)
- Six new module parameters & bring-up switches (
drivers/accel/ane/ane_fw_cold.c:2125-2169):isp_link,isp_fwlog,isp_gate_off,isp_mcache_query,isp_surface_map, andisp_mcache_dsidviolate §2 (default kernel command line, no lab-only switches or dead off-by-default code paths).- Remove
isp_linkandisp_surface_mapand make them fixed behaviour when the ISP requests the link and providessurface_get/surface_put. - Remove
isp_fwlog(usedev_dbg/ dynamic debug). - Either drop
isp_gate_offandisp_mcache_queryif J815 MLVNR does not need them, or wire required behaviour unconditionally without switches. - Hardcoded
isp_mcache_dsid = 0xc(ane_fw_cold.c:2167): Do not carry a macOS-observed magic constant; derive any spill DSID from firmware/hardware or omit the setting until Linux allocates/provides it.
- Remove
Code bugs to fix (Part 2 — drivers/accel/ane/ane_fw_cold.c)
- Dangling IOMMU mapping & leaked resources after firmware reboot (
ane_fw_isp_replay()/ane_isp_endpoint_disable()/ane_isp_link_put(), lines 2540–2549, 2774–2874):- When a firmware reboot/sleep calls
ane_fw_isp_replay()under a live link, it setsl->live = falsewithout cancellingb->isp_service, unmappingb->isp_rings_mapped, releasingb->isp_maps, or clearingb->session.gate_pinned. - When the ISP then calls
ane_isp_endpoint_disable(l), line 2778 (if (IS_ERR_OR_NULL(l) || !l->live) return;) returns immediately — skippingcancel_delayed_work_sync(&b->isp_service), leaving the ISP's ring pages mapped in the ANE DART while the ISP frees them, leavingb->isp_mapspinned, and leavinggate_pinnedstucktrue. - In
ane_isp_link_put(), lines 2866–2870 touchb->isp_map_countandb->isp_maps[i].put = NULLwithoutb->session.lockwhileisp_servicemay still be active ifl->livewas cleared byreplay.
- When a firmware reboot/sleep calls
- Off-by-one IOMMU unmap on
ane_fw_map()failure (ane_fw_isp_afpp_map(), lines 2467–2485):- When
ane_fw_map(b, dva + off, pa, PAGE_SIZE, IOMMU_READ)fails insideiommu_map(), the page atdva + offwas never mapped intob->domain, yet line 2469 executesoff += PAGE_SIZE; goto unwind;soiommu_unmap(b->domain, dva, off)attempts to unmap the unmapped page as well. (Conversely, ifane_fw_map()fails on the DART error check afteriommu_map()succeeded,ane_fw_map()itself leaks that mapping.) Makeane_fw_map()clean up on DART error and unwinddma_unmap_page()for the failed page without inflating theiommu_unmap()length.
- When
- Ring unmap size mismatch (
ane_fw_isp_rings_map(), lines 2515–2530 &unmap:at line 2746):- Setting
b->isp_rings_mapped = truebeforeif (mapped != total)causes the caller'sunmap:path to calliommu_unmap(b->domain, b->isp_ring_dva, b->isp_ring_span)on a partial mapping instead ofmapped(and likewise over-unmaps if a later enable passestotal < b->isp_ring_span). Track the exact mapped length (and unmapmappeddirectly on partial map failure).
- Setting
Hardware/runtime evidence still owed after fixes (§5)
- Full regression run on the final rebased head (kernel/build ID +
0 SELinux denials, 0 failed units, 0 new kernel errors) on J815 (default cmdline, both Part 1 segmentation and Part 2 MLVNR) plus non-regression on other H16/H17 machines (and confirmingane-h16-programKUnit passes cleanly).
Assisted-by: LLM (Gemini 4 Neon)
…al outputs, wide LOAD What the camera's segmentation program needs from the H16 program path: - HWX surface names up to 64 bytes and procedure names other than 'main'; - programs with several outputs and a BAR slot table of any size; - ANE_H16_LOAD_WIDE, for programs of up to 16 surfaces (plain H16_LOAD refuses more than 4 with -E2BIG, since its arrays cannot report them); - driver minor 2: plain H16_LOAD now also takes a program with several outputs, where minors 0 and 1 refused one, so userspace that relies on several outputs checks for minor 2 (the uapi header says so); - the parser's large temporaries on the heap (stack use); - h16-sections.py builds the sections of a program with several outputs. Run on a J815 (2026-11-21): the user's person and skin segmentation program (two outputs) ran at 15 runs/s for the camera effects in aurora-silicon/camera-denoise#16. Signed-off-by: Nick <nekreshen@yandex.ru>
f8947c9 to
470794e
Compare
|
@RyanTheTide @DjDeveloperr thanks. Force-pushed
Provenance (Ryan's question). You asked where the names and layouts came from. Not all of it is from captures on the wire, so here is each source:
Under your rule ("anything read out of the macOS binaries can't go into this tree"), point 3 means Part 2 doesn't qualify as it stands. I didn't want to hide that behind reworded comments. Part 1 doesn't use point 3, as far as my notes show: it was worked out from the user's own program file and from J815 runs. The body now has the same section. For Part 2, I see two options:
Which do you prefer? Until you decide, I'll treat Part 2 as blocked on this. The ISP half in #2645 has the same sourcing (its endpoint config and the surface ID answer come from the same reading), so your decision covers both PRs. Still owed on this head: the J815 run without the system-cache query, and the regression line. I'd be glad to take the M1 Pro / M2 / M4 Pro / A18 Pro rows you offered. Assisted-by: LLM (Claude:Opus 5.5) |
|
Thanks for the rework and for laying out the sources so plainly. Earlier points, at 470794e:
On provenance: the For Part 2, the route that works is the first one you offered: a hypervisor trace of the command ring and the SHAREDMALLOC slots at a macOS camera start, kept as a capture with a short write-up of what it shows. The commands and the answers then get rewritten from that write-up, and the commit message cites only the log and the trace. Ideally someone who hasn't read the disassembly writes the code from the write-up. Part 1 (c39ec3c) looks good:
Some things to carry into the Part 2 rewrite:
Hardware for Part 1 at its new head: a J815 row on the default command line, plus no-regression rows on the other machines that run this program path (M4 Pro J616s, M4 Air J713, A18 Pro). Each row should show an existing single-output program loading and running, and plain Assisted-by: LLM (Claude:Opus 5.5) |
470794e to
c39ec3c
Compare
|
@RyanTheTide thanks, understood. The MLVNR commit is dropped from this PR. The head is now
I'll post the J815 row at this head on the default command line, with a single-output program, the two-output segmentation program through For the ISP endpoint, I'll follow the route you describe:
That rewrite will also take your three points: hold a page reference until the deferred unmap, or clear the PTEs now; the doorbell from the ANE's DT node, refusing the link without it; and the dead Assisted-by: LLM (Claude:Opus 5.5) |
|
Thanks. At c39ec3c this is Part 1 alone, the part I'd already read as clean: One housekeeping point: the dropped commits are still reachable from this PR's timeline. For the MLVNR table, please ask a maintainer to have them purged, or say if you'd prefer this reopened from a clean branch. Assisted-by: LLM (Claude:Opus 5.5) |
|
@RyanTheTide thanks for the approval. I'll post the J815 row at On the housekeeping point: could a maintainer please have the dropped heads purged? The MLVNR code is in both PRs' old history, and #2645's also has the firmware-derived input table:
I've deleted the old Assisted-by: LLM (Claude:Opus 5.5) |
|
Thanks, that list is exactly what's needed. I've passed it, with the #2645 heads and the camera-denoise#16 ones, to the repository owner for a purge request to GitHub. Deleting the fork branch was the right step on your side. Assisted-by: LLM (Claude:Opus 5.5) |
|
Closing this: I'm taking the J815 camera work out of the upstream trees for now and continuing it on my own machine. The M4 Pro and A18 Pro rows aren't needed anymore. The purge request for the dropped heads above (and for #2645) still stands. Thanks for the reviews. Assisted-by: LLM (Claude:Opus 5.5) |
This lets the H16/H17 ANE driver run vendor-compiled camera programs with several outputs. One example is the person/skin segmentation program that the camera effects plugin in aurora-silicon/camera-denoise#16 loads from the user's own macOS install. No Apple files are added here.
Scope: one commit,
c39ec3c9, onintegration/aurora17075e3. The ISP MLVNR endpoint (the former Part 2) was dropped from this PR after review. It will come back separately, rewritten from a hypervisor trace and its write-up (see the review thread).Changes
main.ANE_H16_LOAD_WIDE, for programs of up to 16 surfaces. PlainH16_LOADrefuses more than 4 with-E2BIGbefore anything loads, since its arrays cannot report them.H16_LOADalso takes a program with several outputs, where minors 0 and 1 refuse one.ane_accel.hsays what minor 2 adds.h16-sections.pybuilds the sections of a program with several outputs.ANE_H16_SURFACES= 16 madestruct ane_h16_hwxtoo large for a test's stack.Provenance
This was worked out from the user's own program file (its HWX layout) and from runs on the J815. It does not use the macOS kernelcache reading that the dropped MLVNR part was based on.
Testing
tools/config/ane.kunitconfig, fresh build dir) atc39ec3c9: 298 tests, 297 pass, 1 skipped (apple_g17_rx_util_min_test, needsCONFIG_UCLAMP_TASK), 0 compiler warnings.H16_LOAD_WIDE; plainH16_LOADof a program with more than 4 surfaces returns-E2BIG.Assisted-by: LLM (Claude:Opus 5.5)