Repository navigation
Lower quantized multiply through the Espressif operator library - #24310
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/24310
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit df6bcad with merge base ea0f88f ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
meta-cla
Bot
added
the
CLA Signed
This PR needs a
|
### Summary The Espressif exporter currently lowers supported regions through the ESP-NN delegate. Add an opt-in `use_operator_library=True` route to `to_espnn_edge()` that uses `EspPassManager` and the existing `EspNNQuantizer`. This first step retains executable portable operators and Q/DQ, with normal Edge validation, direct-operator reporting, and strict fallback handling. The delegate remains the default for existing callers. Add the frontend `EspTester` and tests for both export entry points, multiple methods, constants, shared float consumers, fallback, and serialization. Multiply lowering (#24310) and runtime execution (#24311) follow in this stack. Authored with AI assistance using Codex. ### Test plan The isolated frontend change passed all 67 Espressif tests, including 11 new frontend tests. After rebasing the complete stack onto main, all 146 Python tests and all 11 optimized ESP32-S3 implementation cases passed. Scoped lint and whitespace checks also passed. ```bash python -m pytest -q backends/espressif/test --disable-warnings ```
3191f58 to
fd4123b
Compare
| rescale = input1.scale * input2.scale / output.scale | ||
| if not math.isfinite(rescale) or rescale <= 0: | ||
| return None |
There was a problem hiding this comment.
Low priority: could we retain the portable pattern when the original float32 arithmetic can overflow? This checks the combined rescale in Python double precision, but the DQ/multiply/Q path can overflow even when that ratio and the integer bounds are valid.
I reproduced this through torch.export.export → to_edge → AtenToEspPass with static contiguous int8 inputs, both input scales 1e19, output scale 1e38, zero points 0, and full range [-128, 127]. With both operands [1, 2, 10], the original output is [1, 127, 127]; after lowering it is [1, 4, 100] (modified=True). The latter two float32 products overflow, whereas the combined rescale is 1.
These are extreme, manually supplied qparams and unlikely with typical calibration, but they satisfy the current guards. Please add a float32-range guard and a regression test, or explicitly narrow the supported numerical contract.
The opt-in export path now replaces supported quantized multiply patterns with direct esp_nn::quantized_mul_int8 calls. AtenToEspPass dispatches to per-operator functions that read Q/DQ arguments and combine folding with replacement. The initial contract is static, equal-shaped, contiguous int8 tensors with constant full-range per-tensor qparams and safe ESP-NN shift and overflow bounds. Replace the output quantizer's uses so existing output dequantizers and chained operations consume the direct result. Retain the original ATen operation and input dequantizers when a direct float consumer needs the pre-quantization result. Remove only unused nodes. Unsupported patterns, including multiple output quantizers, keep executable Q/DQ and portable compute. Also retain that pattern when the full int8 range can overflow float32 dequantization or multiplication, accounting for float32 rounding of scales and dequantized bounds. Add functional and out schemas, a fake implementation, and a Python integer reference. Share the delegate's Q31 multiplier encoder under backends/espressif/utils, preserving its finite-positive-input rounding and shift range. The existing EspNNQuantizer serves both export paths. Native C++ kernels and runtime registration are the next implementation step; this change establishes Python execution and PTE serialization. Use named multiply cases and EspTester.test_dialect() for operator expectations and numerical checks. Pass-manager tests cover export and API contracts without depending on exact Q/DQ counts. Start review with passes/aten_to_esp_pass.py, then ops/operators.py and utils/quantization.py for the integer contract, followed by test/ops/test_mul.py and the pass-manager tests. All 96 focused multiply, quantization-parameter, and export tests passed. The overflow regression preserves the portable saturated result for extreme qparams; range cases cover each input, their product, zero points, and float32 rounding. Scoped lint and whitespace checks passed. Authored with AI assistance using Codex.
fd4123b to
df6bcad
Compare
Summary
Lower supported quantized multiply patterns to direct
esp_nn::quantized_mul_int8calls through the opt-in Espressif export path.AtenToEspPassreads Q/DQ parameters and combines folding with operator replacement. The initial contract covers static, equal-shaped, contiguous int8 tensors with constant full-range per-tensor qparams and supported shift/overflow bounds; other patterns retain executable portable compute.Retain the portable Q/DQ pattern when the full int8 range can overflow float32 dequantization or multiplication, accounting for zero points and float32 rounding of scales and intermediate bounds.
Existing output dequantizers and chained operators consume the direct result. Float consumers of the original pre-quantization result retain the original ATen multiplication and its input dequantizers, preserving their rounding and saturation behavior.
Add functional/out schemas, a fake implementation, and a Python integer reference. Share the delegate's existing Q31 multiplier encoder under
backends/espressif/utilswhile preserving its behavior. Native execution is supplied by #24311, the next PR in the stack.Start review with
passes/aten_to_esp_pass.py, thenops/operators.pyandutils/quantization.py, followed by the named operator cases and pass-manager tests.Authored with AI assistance using Codex.
Test plan
All 96 focused multiply, qparam, and export tests passed on this branch. Reproduced the extreme-scale mismatch before the fix; the regression now preserves the portable result
[1, 127, 127]. Bounds cases cover each dequantized input, the product, nonzero offsets, finite supported ranges, and float32 rounding.The complete rebased stack passed 153 frontend tests. Scoped lint and whitespace checks passed.