[billmguo]

Summary:
QNN running inside the CDSP user PD cannot load weight-shared context binaries ("Shared weights buffer isn't supported on this platform", err 1000), so BoltNN's executorch_app (QNN inside the Bolt DSP skel) cannot run a hybrid LLM export whose prefill_forward and kv_forward share one weight-shared context. QNN running from the CPU side (AP libQnnHtp.so + libQnnHtpV79Stub.so, graph construction and execution kicked from the AP) handles them fine.

//arvr/projects/bolt/bolt/nn/apps:boltnn_executorch_llm_runner is the upstream qnn_llama_runner CLI linked against the upstream QNN ExecuTorch runtime, so QnnBackend delegates build and run their graphs from the AP. It takes the same flags as the upstream runner.

To share the CLI, oss_scripts/llama gains qnn_llama_runner_main: qnn_llama_runner.cpp with its entry point renamed to qnn_llama_runner_main, on :runner_lib. qnn_llama_runner and runner_lib are unchanged; the fbcode/ and xplat/ copies stay byte-identical.

Authored with Claude.

Differential Revision: D129133307

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/24456

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 2 Pending, 1 Unrelated Failure, 2 Unclassified Failures

As of commit 6119f4a with merge base e40e64c ():

UNCLASSIFIED FAILURES - DrCI could not classify the following jobs because the workflow did not run on the merge base. The failures may be pre-existing on trunk or introduced by this PR:

• Build Aarch64 Linux Native Wheels / executorch-native / build-wheel-py3_11-cpu-aarch64 (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
  ModuleNotFoundError: No module named 'setuptools'
• Build Aarch64 Linux Native Wheels / executorch-native / upload / upload-wheel-py3_11-cpu-aarch64 (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
  Unable to download artifact(s): Artifact not found

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• pull / nxp-mcuxpresso-test / linux-job (gh) (trunk failure)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[meta-codesync]

@billmguo has exported this pull request. If you are a Meta employee, you can view the originating Diff in D129133307.

[github-actions]

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.