Skip to content

QNN: bind delegate outputs in exported program order - #24570

Draft
psiddh wants to merge 5 commits into
pytorch:mainfrom
psiddh:stack2/pr05-qnn-output-order
Draft

psiddh wants to merge 5 commits into
pytorch:mainfrom
psiddh:stack2/pr05-qnn-output-order

Conversation

@psiddh

@psiddh psiddh commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

QNN: bind delegate outputs in exported-program order

QNN preprocessing can build the operations that produce outputs in a different order from the exported graph. Equal-shaped K/V outputs can then bind to the wrong values without a shape error. Encode each ordinary graph output position in its tensor name and restore that order in both runtime allocation paths.

Mutable-buffer / legacy names retain their relative order. Context binaries imported from outside ExecuTorch keep their original order. Malformed and oversized output IDs are treated as legacy names rather than throwing from integer parsing.

Review order: output numbering in builders/utils.py and node_visitor.py, the shared runtime comparator and its two allocation call sites, then Python naming and host-linked runtime tests. This delegate-wide change is deliberately a separate PR for Qualcomm review. It is PR 5 of the Stack 2 series. Its local parent is stack2/pr04-qnn-options-allocator; its GitHub base is main, with earlier stack commits included.

Test Plan

On this PR's source tree:

python -m pytest -q backends/qualcomm/tests/test_output_order.py \
  backends/qualcomm/tests/test_partitioner_options.py
16 passed in 4.08s

cmake -S backends/qualcomm/tests -B qnn-tests -DCMAKE_BUILD_TYPE=RelWithDebInfo \
  -DPYTHON_EXECUTABLE=$(command -v python)
cmake --build qnn-tests -j8 --target qnn_output_order_test qnn_io_buffer_allocator_test
ctest --test-dir qnn-tests --output-on-failure
100% tests passed, 0 tests failed out of 7

Eight Python cases exercise actual output names / wrappers; five C++ cases link the rebuilt real QnnManager and check distinct values in same-shaped buffers, numerical ordering, equal-ID stability, legacy / mutable names, context-binary bypass and malformed IDs. The other tests cover the preceding PR's option routing / allocator.

Before the original numbering change, seven of eight new Python cases fail. Against the original runtime, four of five new C++ cases fail: equal-shaped values are reversed, the DLC allocation path does not sort, and oversized IDs throw. All cases pass after this fix. A full host backends/qualcomm/scripts/build.sh --skip_linux_android --job_number 8 --no_clean build also rebuilt QnnManager, the adaptor / backend, examples and llama_main.

Not run: full QNN device suite or actual context-retrieval execution. Qualcomm hardware review is still needed before merge. Buck parity is blocked by a missing worktree .buck2; local host tests are not a Buck-green or device-support claim.

Stack (review in order)

  1. Partitioner: run backend-contributed edge transform passes #24566 Partitioner: run backend-contributed edge transform passes
  2. Export: avoid reused lifted-constant names #24567 Export: avoid reused lifted-constant names
  3. LLM runner: select I/O contracts and accept custom buffer allocators #24568 LLM runner: select I/O contracts and accept custom buffer allocators
  4. QNN: construct partitioners from options and allocate LLM I/O in rpcmem #24569 QNN: construct partitioners from options and allocate LLM I/O in rpcmem
  5. QNN: bind delegate outputs in exported program order #24570 QNN: bind delegate outputs in exported program order
  6. Export hf_static float-cache programs and register their Python runner #24571 Export hf_static float-cache programs and register their Python runner
  7. LLM runner: validate and drive hf_static float-cache programs #24572 LLM runner: validate and drive hf_static float-cache programs
  8. Document the version-1 hf_static I/O contract #24573 Document the version-1 hf_static I/O contract

Each PR includes the commits of the ones above it; review only its own top commits.

Run each partitioner’s edge transforms after caller-supplied passes so backends can use the standard lowering entry point. QNN contributes its existing transforms through the new hook.
Reserve names for constant tensors already present in the exported program before lifting new constants, so methods with folded tables retain distinct constants.
Choose the registered IOManager and its logits, method names and step sizes, rejecting unknown contracts and invalid metadata. Load and reset the selected methods; route prompt tails to decode with a side-effect-free capacity check. Preserve module and I/O-buffer lifetimes and expose a backend-neutral allocator hook.
Route backend options through QnnPartitioner.from_options and register an allocator that supplies QNN shared-memory buffers through the generic IOManager hook. The LLM runner remains backend-neutral.
Encode graph output positions and restore their order at both runtime tensor-allocation entry points. Preserve context-binary order and legacy mutable outputs, and ignore malformed or overflowing position labels without throwing. This runtime change needs Qualcomm review.
@pytorch-bot

pytorch-bot Bot commented Nov 3, 2026 •

Copy link
Copy Markdown

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/24570

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 109bacd with merge base 5f72739 (image):
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@github-actions

github-actions Bot commented Nov 3, 2026

Copy link
Copy Markdown

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant