Repository navigation
Add a portable kernel for antialiased bicubic resize - #24447
Open
pytorchbot wants to merge 12 commits into
Open
pytorchbot wants to merge 12 commits into
pytorchbot wants to merge 12 commits into
Conversation
…and in exir (#24222) Fixes found while exporting models to ExecuTorch backends. ## Fixes - **exir:** `constant_prop_pass` keeps tensors that in-place ops write, also through views or in `no_grad`/`autocast` blocks. `to_backend` keeps the output node last. `to_edge` lowers `torch._check_tensor_all` and `torch.sym_ite`. `GroupBasedPartitioner` no longer makes partitions that depend on each other, unless caller node groups already do. Tensor constants get static shapes; deduplicated constants keep their data. - **AOTI (AOTInductor) and CUDA:** the missing-kernel check catches scatter and index_put fallbacks. `detach` lowers, also inside `torch.cond`. Strided outputs are copied. Transposed convolutions (1-D, 2-D, no dilation) and `torch._check_tensor_all` lower. - **XNNPACK:** the quantizer skips bfloat16 tensors, and `cat` respects scoped configs. ELU is delegated when XNNPACK can run it. - **Qualcomm:** autocast regions are inlined correctly. Unquantized bfloat16 nodes, data-dependent shapes and HTP partitions without runtime inputs stay on the CPU. Fixes for stacks, 0-d selects, RMSNorm and int64 constants. - **OpenVINO:** model data stays alive while OpenVINO 2026.4 reads it. Partitions are converted with their input types. - **Core ML:** bfloat16 ops, and ops above rank 5 outside static layout chains, stay on the CPU. - **MLX, Arm, Cortex-M, Vulkan, NXP, portable:** dtype checks for MLX tensor lists, one constant slot per weight, slice updates on a constant base; TOSA keeps unused delegate inputs for VGF; `MatmulToBmmPass` takes a constant operand; Vulkan constants above 4 dims; non-strict programs in the NXP quantizer; `cummin.out`. ## Behavior changes - A CUDA `.pte` loads only the weights it was exported with. A `.ptd` with other weights, for example after fine-tuning, needs a newly exported `.pte`. - CUDA graphs turn off when outputs do not match the ETensor layout. If only the capture call sees such a layout, that call returns an error. - `GroupBasedPartitioner` is about 2x slower on large graphs, and some graphs get a different number of partitions. `to_backend` keeps the original order of top-level nodes. - XNNPACK: weight-only quantized linears, and 8da4w or 8da8w linears (8-bit dynamic activations, 4-bit or 8-bit weights) lowered with `to_backend`, fail at export instead of giving wrong results. Delegated ELU flushes subnormal values to zero. Cats outside a scoped quantizer config are no longer quantized. - Ethos-U raises an error for a delegate with unused inputs instead of returning wrong values. We would like to cherry-pick this PR to release/1.6 after it merges. It needs #23915 there first. ## Testing Each fix has a test that fails without it. Run on an A100 (CUDA, Vulkan), an x86 CPU (OpenVINO, QNN SDK), macOS (Core ML, MLX), the TOSA reference model and VGF emulation (Arm), and the eIQ Neutron SDK (NXP). Not run: Metal, ROCm, Windows CUDA. (cherry picked from commit 054de1f)
This branch was picked before #24263 and #24295 reached release/1.6, so three files carried their content only in part. Take release/1.6 and use the versions a clean pick of 054de1f gives for backends/qualcomm/partition/qnn_partitioner.py, backends/qualcomm/tests/test_passes.py and exir/tests/test_pass_infra.py.
release/1.6 regenerated aoti_cuda_shims.lib to add aoti_torch_dtype_element_size, and this branch regenerated it to add aoti_torch_as_strided. Regenerate it per aoti_cuda_shims.lib.md with both names, so it lists exactly the shims this tree exports.
The TensorRT guide said that when a second export overwrites aoti_cuda_blob.ptd, the first model can return wrong answers with no error. CUDA weight keys now carry a hash of their contents, so the first model's weights are missing from the new file and loading fails instead. Docs only.
…r HTP On HTP, the partitioner keeps a partition that reads nothing at runtime on the CPU, because HTP refuses such a graph. That happens after decomposition, so an op that QNN asks to keep whole, such as linear or hardswish, could land on the CPU still undecomposed, and to_edge_transform_and_lower failed with "was not decomposed or delegated". The filter returned with the ops to keep whole now also leaves out any op that reads only constants, so it is decomposed like other CPU ops. An op that reads a runtime input is still kept whole. The constant-only partition test now has a linear in its constant branch, and fails without this change.
…ly op The comment said ops that read only constants are decomposed like other CPU ops. Partitions are not known when the filter is built, so the rule also decomposes such ops whose partition stays delegated. The comment and the ops_to_not_decompose docstring now say so. Comment only.
… HTP The HTP rule that decomposes ops reading only constants also covered the quantize and dequantize ops in the skip table, such as dequantize_per_channel_group on a frozen weight. QNN builds the weight encoding from those ops. With today's torch they are not decomposed anyway, so nothing changed in practice, but the filter should not ask for it. They are now kept whole, and ops after them still count as reading only constants. A new test checks the filter itself and needs no QNN SDK, so it runs in CI: a constant-only linear is decomposed, a linear on an input and a constant-only group-wise dequantize are kept whole.
After lowering, exir asks the partitioner for its filter again and reports any kept-whole op still on the CPU. On HTP the filter now let constant-only ops through, so that report went quiet. For example RecomposeRmsNorm rebuilt a constant-only rms_norm that then stayed on the CPU, and the program failed later in to_executorch with "Missing out variants" instead of the clear error from before. On the lowered (edge) program the filter is plain filter_fn again, so these cases report as they did before the HTP change. The filter test checks this too.
… Samsung passes torch.export does not record some of the names it assigns: the names of inputs, and the names of ops it renamed to make room for an input. With an input called mul, the model's own input * 2 becomes mul_1, and nothing marks mul_1 as taken. A later pass that creates a node can then pick one of those names again, and the graph fails with "Node redefined name t_1!". QNN models with autocast regions or attention hit it, and so does the Samsung DecomposeRoll pass on a model with an input named cat. reserve_node_names in backends/transforms/utils.py records the name of every node of the top-level graph before the passes run. The QNN and Samsung pass managers call it at the start of _transform. torch.fx has no public way to reserve a name: copying the graph would register the names, but it also renames inputs named after Python builtins (input becomes input_1) while the graph signature keeps the old names. So the helper adds the names to the graph's name set directly, with a TODO to drop it once torch.export records the names (pytorch/pytorch#202927). The node name test runs the QNN export passes on an autocast model and on an attention model whose inputs are named input and mul. It checks the graph lints, keeps the input names of the graph signature, and matches eager, and that the collision is real without the helper. Same change as #24316 on main.
…on CPU On HTP, a partition that reads nothing at runtime stays on the CPU, because HTP refuses a graph without runtime inputs. That is decided after decomposition. So a constant-only op that QNN keeps whole, such as linear, could land in such a partition undecomposed, and lowering failed with "was not decomposed or delegated". The HTP filter of ops_to_not_decompose now runs the same drop rule before decomposition and decomposes only the kept-whole ops it would leave on the CPU. The real partitions are not known yet, so it partitions the graph as if QNN took every op except the ones whose data types or shapes rule it out (the checks filter_fn already makes, now shared as must_stay_on_cpu). Ops in partitions that stay delegated keep their whole form, and with it their quantization encodings. The caller's skip_node_op_set and skip_node_id_set are left out of that estimate on purpose. Decomposing adds nodes, and the new nodes take names that the later nodes had before. If a skip could change what is decomposed, adding one skip could rename the node that a skip_node_id_set entry means, and the wrong node would be skipped. Ops that QNN only rejects while partitioning are not known yet either. So a kept-whole op that only a skip or such an op cuts off still fails as before. exir calls ops_to_not_decompose several times before decomposition and uses only one of the filters, so the estimate runs on the filter's first use. On a 10,240-node graph the partition search alone takes about 10 s. On the lowered program the filter stays plain filter_fn, so exir still reports kept-whole ops left on the CPU. The constant-only partition test now stubs the QNN manager and the compile, so it runs without the QNN SDK. It checks that a constant-only linear cut off by bfloat16 casts is decomposed, that a constant-only linear in a delegated partition stays whole, that the filter is plain filter_fn on the lowered program, and that a node ID skip still names the same add when an op skip and an ID-skipped feeder are present. Same change as #24429 on main, including its Partitioner.ops_to_not_decompose docstring. It replaces the earlier version carried here, which decomposed every op that reads only constants.
`to_edge` cannot lower `F.interpolate(x, mode="bicubic", antialias=True)` today. It exports to `aten._upsample_bicubic2d_aa`, which is not a core ATen op, so `to_edge` stops with: ``` SpecViolationError: Operator torch._ops.aten._upsample_bicubic2d_aa.default is not in Core ATen opset ``` ExecuTorch also has no kernel for it. ## What this change does It adds a portable kernel for `_upsample_bicubic2d_aa.out` and lists the op as an exception in the edge verifier, as the antialiased bilinear resize does. Like ATen's CPU kernel, it uses the Keys cubic filter (a = -0.5), widened when shrinking, resizes the width, then the height, and copies a dim that keeps its size. float16 and bfloat16, which ATen's CPU kernel lacks, are computed in float32 and rounded once. uint8 follows ATen's fixed-point path: 16-bit integer weights, and each pass rounds and clamps. Other integer types fail in the kernel, as they do in ATen. Between passes it keeps only the rows one output row reads, so shrinking 1080x1920 to 224x224 needs about 95 KB of temporary memory (43 KB for uint8). It rejects scales `F.interpolate` never produces: too small for the input, or (uint8 only) with weights too large for 16 bits. Where ATen's generic CPU path is wrong (contiguous input, output width 1, changing height; [PyTorch issue 200568](pytorch/pytorch#200568)), it differs from eager on purpose. The shared upsample helper now ignores a non-positive scale, as ATen does; of the other upsample kernels, only antialiased bilinear can pass one. A stale llava comment is updated, and the Arm all-ops test marks the kernel as needing a recipe. Like several portable kernels, it has no experimental CPU backend route yet. ## Testing - New kernel tests check against ATen: shrinking, `align_corners`, scales, windows past the input, kept dims, both layouts, a batch, all float types, uint8 bit for bit, a dynamic output, running out of temporary memory, and error cases. Each case catches a bug that no other case catches. - The `to_edge` test compares static and dynamic programs (both layouts) with eager in float32 and, bit for bit, in uint8. It fails without the kernel or the verifier exception. - Outside the suite, a random sweep matched ATen's uint8 results exactly on x86 and Arm, on both its vectorized and generic paths; float32 within 7e-6 for unit-scale inputs, half types within one unit in the last place. - With XNNPACK, the resize stays outside the delegate. On one laptop thread, shrinking a contiguous 1080x1920 input to 224x224 takes about 7 ms (12 ms channels last). (cherry picked from commit a5476a5)
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/24447
Note: Links to docs will display an error until the docs builds have been completed. ❌ 3 Pending, 2 Unrelated Failures, 6 Unclassified FailuresAs of commit bff92dc with merge base 4db778e ( UNCLASSIFIED FAILURES - DrCI could not classify the following jobs because the workflow did not run on the merge base. The failures may be pre-existing on trunk or introduced by this PR:
BROKEN TRUNK - The following jobs failed but were present on the merge base:👉 Rebase onto the `viable/strict` branch to avoid these failures
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
pytorchbot
requested review from
JacobSzwejbka,
digantdesai,
larryliu0820 and
rascani
as code owners
November 2, 2026 17:10
This was referenced Nov 2, 2026
shoumikhin
changed the base branch from
cherry-pick-24222-by-pytorch_bot_bot_
to
release/1.6
November 2, 2026 17:58
shoumikhin
requested review from
SS-JIA,
mergennachin,
metascroy,
psiddh,
robert-kalmar and
shoumikhin
as code owners
November 2, 2026 17:58
JacobSzwejbka
deployed
to
upload-benchmark-results
November 2, 2026 20:59 — with
GitHub Actions
Active
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Merge after #24298. This pick is stacked on it, because #24400 builds on changes that #24298 brings to release/1.6.
to_edgecannot lowerF.interpolate(x, mode="bicubic", antialias=True)today. It exports toaten._upsample_bicubic2d_aa, which is not a core ATen op, soto_edgestops with:ExecuTorch also has no kernel for it.
What this change does
It adds a portable kernel for
_upsample_bicubic2d_aa.outand lists the op as an exception in the edge verifier, as the antialiased bilinear resize does.Like ATen's CPU kernel, it uses the Keys cubic filter (a = -0.5), widened when shrinking, resizes the width, then the height, and copies a dim that keeps its size. float16 and bfloat16, which ATen's CPU kernel lacks, are computed in float32 and rounded once. uint8 follows ATen's fixed-point path: 16-bit integer weights, and each pass rounds and clamps. Other integer types fail in the kernel, as they do in ATen.
Between passes it keeps only the rows one output row reads, so shrinking 1080x1920 to 224x224 needs about 95 KB of temporary memory (43 KB for uint8). It rejects scales
F.interpolatenever produces: too small for the input, or (uint8 only) with weights too large for 16 bits. Where ATen's generic CPU path is wrong (contiguous input, output width 1, changing height; PyTorch issue 200568), it differs from eager on purpose.The shared upsample helper now ignores a non-positive scale, as ATen does; of the other upsample kernels, only antialiased bilinear can pass one. A stale llava comment is updated, and the Arm all-ops test marks the kernel as needing a recipe. Like several portable kernels, it has no experimental CPU backend route yet.
Testing
align_corners, scales, windows past the input, kept dims, both layouts, a batch, all float types, uint8 bit for bit, a dynamic output, running out of temporary memory, and error cases. Each case catches a bug that no other case catches.to_edgetest compares static and dynamic programs (both layouts) with eager in float32 and, bit for bit, in uint8. It fails without the kernel or the verifier exception.cc @digantdesai @freddan80 @per @zingo @oscarandersson8218 @mansnils @Sebastian-Larsson @robell @rascani