Skip to content

accel/ane: h16, h17: load programs with int8 and uint8 surfaces - #2748

Merged
niteria merged 1 commit into
aurora-silicon:integration/aurorafrom
niteria:ane/h16-h17-int8-io
Nov 22, 2026
Merged

niteria merged 1 commit into
aurora-silicon:integration/aurorafrom
niteria:ane/h16-h17-int8-io

Conversation

@niteria

@niteria niteria commented Nov 22, 2026 •

Copy link
Copy Markdown
Contributor

Problem

hwx_tensor() in drivers/accel/ane/ane_h16_hwx.c refuses every tensor descriptor whose format code (+0x24) is not 5 or whose element size (+0x68) is not 2, and enforces a minimum row stride of width * 2. As a result, any H16/H17 program with an 8-bit input or output (quantized networks, quantize/dequantize edges, uint8 image inputs, bool outputs) fails H16_LOAD with -ENOEXEC.

Change

Accept the three (format, element bytes) pairs seen on H16 and H17 surfaces that the driver already sizes cleanly:

  • (5, 2) — fp16
  • (2, 1) — int8
  • (3, 1) — uint8

and check the row minimum against width * element instead of width * 2 (while also requiring the upper 32 bits of the u64 element size at +0x68 to be zero).

The rest of the layout is unchanged: observed 8-bit objects align rows to 64 bytes like fp16 ones (for example, a 224-wide uint8 image has 256-byte rows against 512 bytes for fp16), plane = height * row, batch = channels * plane, and span = n * batch. 797 of the 2057 8-bit surfaces in the qweights sweep have row < 2 * width, so updating the row minimum alongside the format table is required to admit them (hwx_tensor() checks only the width * element floor, not row alignment, for all accepted formats).

Other pairs seen in Apple's objects (fp32 (6, 4), int16 (4, 2), uint16 (15, 2)) and any unknown pairs stay refused until tested on hardware.

KUnit (hwx_element_format_test in ane_h16_hwx_test.c) covers all three accepted pairs on input and output (including mixed int8-in / uint8-out), 14 refused pairs (mismatched sizes, unknown codes, fp32/int16/uint16, non-zero upper 32 bits), and the 256-byte row width boundary (256 int8 accepted, 257 int8 refused, 256 fp16 refused, 128 fp16 accepted).

Testing

What Result
scripts/checkpatch.pl --strict (f03100f28f2e) Clean (0 errors, 0 warnings, 0 checks)
arm64 build of drivers/accel/ane/, W=1, J700 config with ANE built in (f03100f28f2e) No new warnings
KUnit under arm64 QEMU (ane.kunitconfig, KASAN + UBSAN, f03100f28f2e) 268/268 passed (ane-h16-hwx: 18/18 including hwx_element_format_test)
tools/asahi/run-host-tests.py (ANE host tests) ane-tm-transient-fault-tests passed (2/2; commit only touches ane_h16_hwx.c and ane_h16_hwx_test.c)
Userspace HWX validation (ane_h16_hwx.c, ASan/UBSan) against wrapped H17 objects 2838/2838 accepted (2823 qweights + 15 8-bit op-survey; 1315 previously refused); the 1523 previously accepted give identical surfaces, spans, and allocations
J713 (T8132 / H16G, M4 MacBook Air 13", dc771820 + f03100f28f2e, ane_h16_hwx.ko) — negative control Before patch: qweights.conv/held1/k3_c33_o79_34x45_b0_none__f16__us (uint8 -> int8, w=34, row=64 < 2*width) fails H16_LOAD with -ENOEXEC (Exec format error)
J713 (T8132 / H16G) — 8-bit + fp16 qweights (67 wrapped Apple H16 cases) & add128 67/67 passed (0 LOADFAIL, 0 hangs/recoveries, 3.37 s wall; qweights.conv 42/42, qweights.linear 11/11, qweights.multi 14/14). Covers all 9 I/O combos (50 8-bit I/O, 17 with row < 2*width, 17 fp16; max relL2 6.26e-3 8-bit / 1.90e-3 fp16 vs tol=0.02). fp16 add128 10/10 runs 16384/16384 exact, 0 ulp (0.6 ms load, p50 109.7 us). Baseline: 0 failed systemd units, 0 new dmesg -l err..emerg entries, 0 ANE/DART/PMGR errors (NixOS, no SELinux)
J700 (T8140 / H17, MacBook Neo, b8c5bb03 + identical ane_h16_hwx.c code + H17 carries) — live-head qweights + add1024 fp16 add1024 10/10 runs 1024/1024 exact, 0 ulp (p50 120.3 us); qweights 73 staged cases (146 Apple + Lean programs; 1 case refused at emit by Lean's D10 gate): 0 LOADFAIL, 0 hangs/recoveries (7 s device time), 70/73 pass C (remaining 3 l4-v2/l4-v4 match Apple bit-for-bit, bitdiff 0). Baseline: 0 failed systemd units, 0 new dmesg -l err..emerg entries during the run (NixOS, no SELinux). Earlier 2026-11-19 run also verified mhc 960x540 ×2 exact (md5 34ff134b) and standalone uint8-act dw_c3_o3_35x65_b1_relu__au8-t-z0__fs (relL2 1.57e-3)

Overlap

Checked against open PRs on aurora-silicon/linux: PR #2647 (accel/ane: run Apple's camera programs, serve the ISP's MLVNR endpoint) also edits hwx_tensor() in ane_h16_hwx.c (procedure name "net" vs "main"), orthogonal to format/element-size admission.

Still to run

None — tested on both H16G (J713 / T8132) and H17 (J700 / T8140).

  • Clean-room: no Apple code, strings, firmware, RE output or tuning values
  • No chip/board/firmware-version gate where a hardware property can decide
  • Works on the default kernel command line; no hacks, delays or lab-only paths
  • Safety limits come from the machine, not from constants in code
  • Hardware safety rules followed for every test boot
  • Third-party commits unchanged; reporters and testers credited
  • Commits: human Signed-off-by only; no agent trailers of any kind
  • Tested on the affected hardware; no regression (J713/H16G and J700/H17 tested)

Assisted-by: LLM (Claude, Gemini)

@RyanTheTide

Copy link
Copy Markdown
Contributor

The change is small and the bound that keeps a program inside its buffers, n * batch <= section.size, is untouched, so admitting 8-bit surfaces cannot widen what a program can reach. The format table is named and commented and the u64 element check is equivalent to the old one. Two points:

  • drivers/accel/ane/ane_h16_hwx.c:372-374: the comment says 8-bit surfaces keep the 64-byte row alignment, but nothing checks it (the fp16 path never did either). Either check row % 64 == 0 for all accepted formats, or word the comment as what the objects show, so that a reader does not assume the validator enforces it.
  • The title and the comments cover H16, but the hardware rows are J700 only. An H16 row on an M4 (J713 or J616s) at this head, with the same 8-bit qweights set and the fp16 add/mhc regression, would cover the second generation the validator is shared with.

Assisted-by: LLM (Claude:Opus 5.5)

@DjDeveloperr DjDeveloperr left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 046f9839d76b against pre-pr-checklist.md.

0 code/policy/attribution/CI blockers.

  • §1–§3 (Correctness, bounds, alignment, safety):
    • hwx_element_size() safely reads +0x24, +0x68, and +0x6c (after length >= 0xd48 is checked), rejects any non-zero upper 32 bits of the u64 element size, and admits strictly (5, 2) (fp16), (2, 1) (int8), and (3, 1) (uint8).
    • Checked overflow/bounds invariants in hwx_tensor() and all downstream consumers (ane_h16_load.c, ane_h16_program.c, ane_h16_accel.c): lowering the row floor from width * 2 to width * element keeps row <= plane <= batch <= n * batch == b->span <= b->section.size <= b->allocation, and no downstream path assumes 2 bytes/element or allows DMA/mmap access past the validated section/allocation.
    • hwx_element_format_test thoroughly exercises all 3 accepted pairs, 14 rejected pairs (including mixed sizes and non-zero upper u32), mixed int8-in/uint8-out, and the exact 256-byte row width boundary.
    • Non-blocking nit: drivers/accel/ane/ane_h16_hwx.c:370-371 says 8-bit surfaces keep the 64-byte row alignment of fp16 ones, though hwx_tensor() does not actually enforce IS_ALIGNED(row, 64) for either format—worth phrasing as what observed objects do (or enforcing it explicitly) if you respin.
  • §4 & §6 (Clean-room, attribution, naming): Human author/committer and Signed-off-by: only on 046f9839d76b, PR disclosure present, clean-room and naming clean.
  • CI: host-tests green on 046f9839d76b.

Holding only for §5 hardware verification on 046f9839d76b:

  • H16 hardware row (e.g., M4 J713 or J616s) at 046f9839d76b: 8-bit (int8/uint8) program load/execute plus fp16 regression (add/mhc), since the commit and validator cover both H16 and H17 (§1 requires proving across affected generations or requesting a test run).
  • Live-head regression row (on H17/J700 and H16) reporting the §5 baseline checks: 0 SELinux denials, 0 failed systemd units, 0 new kernel errors (dmesg -l err..emerg) on 046f9839d76b (the existing J700 row is from an earlier 2026-11-19 commit and omits the regression checks).

Assisted-by: LLM (Gemini 4 Neon)

The HWX validator refused every tensor descriptor whose format code
(+0x24) was not 5 or whose element size (+0x68) was not 2, so any program
with an 8-bit input or output failed H16_LOAD with ENOEXEC: quantized
networks, quantize/dequantize at the edges, uint8 image inputs.

Apple's H16 and H17 objects carry three (format, element bytes) pairs on
surfaces that this driver can size: fp16 (5, 2), int8 (2, 1) and uint8
(3, 1). The rest of the descriptor is the same as for fp16: rows keep the
64-byte alignment (a 224-wide uint8 image has 256-byte rows, the fp16
one 512), plane = height * row, batch = channels * plane, and the span is
still n * batch, so the buffer sizes need no change. Only the row minimum
assumed two bytes per element; it is now width * element bytes, which
an 8-bit surface needs whenever its row is shorter than 2 * width (797 of
the 2057 8-bit surfaces in the qweights sweep).

Accept exactly those three pairs. Other pairs seen in Apple's objects
(fp32 (6, 4), int16 (4, 2), uint16 indices (15, 2)) and anything else
stay refused until they are tested on hardware.

Checked in userspace against Apple's H17 objects in Lean's Linux
container (anec --wrap): all 2823 qweights programs and 15 8-bit
op-survey programs validate (1315 of them were refused before), and the
1523 that validated before give the same surfaces and spans.

Signed-off-by: Bartosz Nitka <niteria@gmail.com>
@niteria
niteria force-pushed the ane/h16-h17-int8-io branch from 046f983 to f03100f Compare November 22, 2026 13:27
@niteria

niteria commented Nov 22, 2026

Copy link
Copy Markdown
Contributor Author

Addressed both review points and force-pushed f03100f28f2e (re-verified checkpatch.pl --strict clean, arm64 W=1 clean, arm64 QEMU KUnit 268/268):

  1. drivers/accel/ane/ane_h16_hwx.c:369-377 comment: reworded to describe what observed 8-bit objects do (64-byte row alignment like fp16, so row can be shorter than 2 * width) and explicitly note that hwx_tensor() checks only the width * element floor, not row alignment.
  2. H16G hardware test (M4 MacBook Air 13", J713 / T8132, dc771820 + f03100f28f2e, ane_h16_hwx.ko):
    • Negative control (same J713 without patch): qweights.conv/held1/k3_c33_o79_34x45_b0_none__f16__us (uint8 -> int8, w=34, row=64 < 2*width) failed H16_LOAD with -ENOEXEC (Exec format error).
    • With f03100f28f2e: all 67/67 wrapped Apple H16 qweights cases loaded, executed (3.37 s wall, 0 LOADFAIL, 0 hangs/recoveries), and passed float64 MIL accuracy (qweights.conv 42/42, qweights.linear 11/11, qweights.multi 14/14; all 9 I/O combinations covered — 50 with 8-bit I/O, 17 with row < 2*width, 17 fp16; max relL2 6.26e-3 8-bit / 1.90e-3 fp16 vs tol=0.02).
    • fp16 regression (add128): 10/10 runs 16384/16384 exact, max 0 ulp (0.6 ms load, p50 109.7 us).
    • Baseline regression: 0 failed systemd units, 0 new dmesg -l err..emerg entries, 0 ANE/DART/PMGR errors (NixOS, no SELinux).
  3. J700 (T8140 / H17) live-head check on b8c5bb03 (identical ane_h16_hwx.c executable code to f03100f28f2e):
    • fp16 regression (add1024): 10/10 runs 1024/1024 exact, max 0 ulp (p50 120.3 us).
    • qweights (73 staged cases = 146 Apple + Lean programs; 1 refused at emit by Lean's D10 gate): 0 LOADFAIL, 0 hangs/recoveries (7 s device time), 70/73 pass C (remaining 3 l4-v2/l4-v4 match Apple bit-for-bit, bitdiff 0).
    • Baseline regression: 0 failed systemd units, 0 new dmesg -l err..emerg entries or ANE errors during the run (NixOS, no SELinux).

Updated the PR description with these results.

Assisted-by: LLM (Gemini)

@RyanTheTide

Copy link
Copy Markdown
Contributor

Re-review of f03100f. Both points are covered. The comment now describes what the observed objects do and says hwx_tensor() checks only the width * element floor, not row alignment. The J713 (H16G) row has the shape I asked for: the case with row < 2*width fails with -ENOEXEC without the patch, all 67 qweights cases load and pass accuracy with it, the fp16 add128 stays exact over 10 runs, and there are no new errors. The bound that keeps a program inside its buffers is unchanged and the KUnit additions cover the new pairs. Approving from my side.

Assisted-by: LLM (Claude:Opus 5.5)

@DjDeveloperr DjDeveloperr left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed f03100f28f2e against pre-pr-checklist.md (§1–§6). Every point from the prior review on 046f9839d76b is resolved:

  • §1–§3 (Code, bounds, safety, no model gating):
    • The only code delta from 046f9839d76b to f03100f28f2e rewords the comment in drivers/accel/ane/ane_h16_hwx.c:369-376 to accurately describe observed 64-byte row alignment while stating explicitly that hwx_tensor() enforces only the width * element floor.
    • hwx_element_size() rejects non-zero upper 32 bits of +0x68 and admits strictly (5, 2) (fp16), (2, 1) (int8), and (3, 1) (uint8). All overflow/span invariants (row <= plane <= batch <= n * batch == b->span <= b->section.size <= b->allocation) remain intact across H16 and H17 with zero chip/board/firmware gating.
    • hwx_element_format_test covers all 3 accepted pairs, 14 rejected pairs (including fp32/int16/uint16 and non-zero upper u32), mixed int8-in/uint8-out, and exact 256-byte row width boundaries.
  • §4 & §6 (Clean-room, attribution, naming): Commit f03100f28f2e carries human author/committer and human Signed-off-by: only (zero AI trailers in commit); PR disclosure is present; clean-room and naming are clean.
  • §5 (Hardware evidence & regression baseline):
    • H16G (J713 / T8132, dc771820 + f03100f28f2e): negative control confirms -ENOEXEC before the patch on row=64 < 2*width; patched run passes 67/67 wrapped Apple H16 qweights cases across all 9 I/O combos (0 LOADFAIL, 0 hangs/recoveries, max relL2 6.26e-3 vs 0.02), fp16 add128 10/10 exact (0 ulp), 0 failed systemd units, 0 new dmesg -l err..emerg entries (NixOS, no SELinux).
    • H17 (J700 / T8140, b8c5bb03 + identical ane_h16_hwx.c): fp16 add1024 10/10 exact (0 ulp), 73 staged qweights cases (0 LOADFAIL, 0 hangs/recoveries), 0 failed systemd units, 0 new dmesg -l err..emerg entries (NixOS, no SELinux).
    • Arm64 QEMU KUnit (268/268, KASAN+UBSAN), checkpatch.pl --strict, and W=1 clean.
  • CI: Host tests / host-tests green on f03100f28f2e.

LGTM — ready to merge.

Assisted-by: LLM (Gemini 4 Neon)

@niteria
niteria merged commit 17075e3 into aurora-silicon:integration/aurora Nov 22, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants