[psiddh]

LLM runner: select I/O contracts and accept custom buffer allocators

A runner needs to know the program's cache contract, logits output, method names and fixed step lengths. Add an IOManager registry and a backend-neutral buffer allocator hook, and provide the existing kvcache_io adapter through that registry. Unknown / malformed labels and invalid context metadata fail instead of silently selecting a different contract.

The runner loads and resets the selected methods, resets its IOManager, and keeps module / buffer lifetimes ordered. A positional-capacity preflight can route a prompt tail through decode before execution; an execution error is never retried. Backend allocators can supply / bind cache buffers without QNN code in the runner.

Review order: the IOManager hooks / registry / allocator interface, the KvCacheIOManager implementation, runner construction and load / reset, prompt chunking / capacity routing, then generic registry / mock runner / prefiller tests. This is PR 3 of the Stack 2 series. Its local parent is stack2/pr02-lift-constants; its GitHub base is main, with earlier stack commits included. QNN's allocator follows in PR 4; hf_static's concrete IOManager follows separately in PR 7.

Test Plan

Fresh build of this PR's source:

cmake -S . -B /data/users/sidart/qnnflow/gh_prs/build-package -DEXECUTORCH_BUILD_TESTS=ON \
  -DEXECUTORCH_BUILD_EXTENSION_LLM=ON -DEXECUTORCH_BUILD_EXTENSION_LLM_RUNNER=ON \
  -DEXECUTORCH_BUILD_EXTENSION_MODULE=ON -DEXECUTORCH_BUILD_EXTENSION_DATA_LOADER=ON \
  -DEXECUTORCH_BUILD_EXTENSION_TENSOR=ON -DEXECUTORCH_BUILD_EXTENSION_FLAT_TENSOR=ON \
  -DEXECUTORCH_BUILD_EXTENSION_NAMED_DATA_MAP=ON \
  -DEXECUTORCH_TEST_EXPORT_LOCK=/data/users/sidart/qnnflow/gh_prs/model-export.lock
cmake --build /data/users/sidart/qnnflow/gh_prs/build-package -j8 --target test_runner
ctest --test-dir /data/users/sidart/qnnflow/gh_prs/build-package -R '^test_runner$' --output-on-failure -V
148 tests: 146 passed, 0 failed, 2 pre-existing skipped

The skips are TextDecoderRunnerTest.LoadUsesMethodName and StepWithAllModels. New generic tests distinguish registered / unknown / null factories, selected method arguments, IOManager reset, near-context decode preflight, and no retry of execution errors. hf_static's genuine generate-reset-generate test is in its concrete runtime PR.

Tests / builds use an isolated source tree; no existing checkout was moved or published. The reachable base is 5f72739cf26ed53df21a746fb5b022fb8e4266d9, because GitHub main fetch returned proxy 403. No #23563 cherry-pick is included.

Stack (review in order)

1. Partitioner: run backend-contributed edge transform passes #24566 Partitioner: run backend-contributed edge transform passes
2. Export: avoid reused lifted-constant names #24567 Export: avoid reused lifted-constant names
3. LLM runner: select I/O contracts and accept custom buffer allocators #24568 LLM runner: select I/O contracts and accept custom buffer allocators
4. QNN: construct partitioners from options and allocate LLM I/O in rpcmem #24569 QNN: construct partitioners from options and allocate LLM I/O in rpcmem
5. QNN: bind delegate outputs in exported program order #24570 QNN: bind delegate outputs in exported program order
6. Export hf_static float-cache programs and register their Python runner #24571 Export hf_static float-cache programs and register their Python runner
7. LLM runner: validate and drive hf_static float-cache programs #24572 LLM runner: validate and drive hf_static float-cache programs
8. Document the version-1 hf_static I/O contract #24573 Document the version-1 hf_static I/O contract

Each PR includes the commits of the ones above it; review only its own top commits.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/24568

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 3823faf with merge base 5f72739 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.