From 6119f4ac5fe4977a6baee2cce436716a335fbd6b Mon Sep 17 00:00:00 2001 From: Min Guo Date: Mon, 2 Nov 2026 11:15:55 -0800 Subject: [PATCH] Add `boltnn_executorch_llm_runner`: run weight-shared QNN LLM exports from the AP Summary: QNN running inside the CDSP user PD cannot load weight-shared context binaries ("Shared weights buffer isn't supported on this platform", err 1000), so BoltNN's `executorch_app` (QNN inside the Bolt DSP skel) cannot run a hybrid LLM export whose `prefill_forward` and `kv_forward` share one weight-shared context. QNN running from the CPU side (AP `libQnnHtp.so` + `libQnnHtpV79Stub.so`, graph construction and execution kicked from the AP) handles them fine. `//arvr/projects/bolt/bolt/nn/apps:boltnn_executorch_llm_runner` is the upstream `qnn_llama_runner` CLI linked against the upstream QNN ExecuTorch runtime, so `QnnBackend` delegates build and run their graphs from the AP. It takes the same flags as the upstream runner. To share the CLI, `oss_scripts/llama` gains `qnn_llama_runner_main`: `qnn_llama_runner.cpp` with its entry point renamed to `qnn_llama_runner_main`, on `:runner_lib`. `qnn_llama_runner` and `runner_lib` are unchanged; the `fbcode/` and `xplat/` copies stay byte-identical. Authored with Claude. Differential Revision: D129133307 --- .../qualcomm/oss_scripts/llama/targets.bzl | 25 +++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/examples/qualcomm/oss_scripts/llama/targets.bzl b/examples/qualcomm/oss_scripts/llama/targets.bzl index 54036076988..41770e37bc5 100644 --- a/examples/qualcomm/oss_scripts/llama/targets.bzl +++ b/examples/qualcomm/oss_scripts/llama/targets.bzl @@ -68,6 +68,31 @@ def define_common_targets(is_fbcode = False): **get_oss_build_kwargs() ) + # `qnn_llama_runner`'s CLI with its entry point renamed to + # `qnn_llama_runner_main`, for a binary that wraps it. + runtime.cxx_library( + name = "qnn_llama_runner_main", + srcs = [ + "qnn_llama_runner.cpp", + ], + compiler_flags = [ + "-Wno-global-constructors", + ], + preprocessor_flags = [ + "-Dmain=qnn_llama_runner_main", + ], + exported_deps = [ + ":runner_lib", + "//executorch/extension/threadpool:threadpool", + ], + external_deps = [ + "gflags", + ], + platforms = [ANDROID], + visibility = ["PUBLIC"], + **get_oss_build_kwargs() + ) + else: runtime.python_library( name = "static_llama",