[psiddh]

LLM runner: validate and drive hf_static float-cache programs

The runner's logical position could reset while HF cache counters stayed stale, corrupting the next generation. This IOManager writes every counter from start_pos before a step, validates the version-1 metadata and all method buffer shapes / dtypes, and drives the fixed prefill / decode methods through shared float-cache buffers.

Window masks identify real rows and exclude padding. A prompt tail that fits the context but not a padded prefill call is routed through decode before any execution. Logits are read at 3 * n_layers; the mutation prefix is opaque. Real HF programs can also return ordered state again after logits. Each unplanned suffix output is bound to its existing cache / counter buffer, preventing the null-output crash without allocating another cache.

Review order: the version-1 contract documented in the following docs PR, hf_static_io_manager.h/.cpp validation and state handling, each IOManager's own registration, build wiring, then the tiny float32 / float16 / HF-style repeated-state fixtures and C++ regressions. This is PR 7 of the Stack 2 series. Its local parent is stack2/pr06-hf-static-export; its GitHub base is main, with earlier stack commits included. Quantized caches, sha and full-length sliding mode are excluded.

Test Plan

The isolated runtime implementation passes:

cmake --build /data/users/sidart/qnnflow/gh_prs/build-final -j8 --target test_runner
ctest --test-dir /data/users/sidart/qnnflow/gh_prs/build-final -R '^test_runner$' --output-on-failure -V
225 tests: 223 passed, 0 failed, 2 pre-existing skipped

The assembled final top was rebuilt from source and rerun with the same225/223/2 counts; no stale executable is counted.

Forty-two float-only v1 fixtures generated with run_reinplace_pass=True, without #23563, pass 77 / 77 HfStatic gtests with no skips. Original runtime against the new fixture corpus gives 41 expected failures, including stale counters, generate-reset-generate, metadata mismatches, equal-byte different shapes / dtypes, malformed mask / positions and context-tail handling. Additional output-boundary probes capture the before-fix repeated-state rejection / unbound-counter segfault and after-fix generation, reset and existing-buffer alias checks.

No #23563 memory-planning cherry-pick is carried; it is listed as related dependency work, with clean-base fixture evidence above. No device lease or sliding-window device latency run was done. CPU correctness is not a claim of device support beyond context 512.

Stack (review in order)

1. Partitioner: run backend-contributed edge transform passes #24566 Partitioner: run backend-contributed edge transform passes
2. Export: avoid reused lifted-constant names #24567 Export: avoid reused lifted-constant names
3. LLM runner: select I/O contracts and accept custom buffer allocators #24568 LLM runner: select I/O contracts and accept custom buffer allocators
4. QNN: construct partitioners from options and allocate LLM I/O in rpcmem #24569 QNN: construct partitioners from options and allocate LLM I/O in rpcmem
5. QNN: bind delegate outputs in exported program order #24570 QNN: bind delegate outputs in exported program order
6. Export hf_static float-cache programs and register their Python runner #24571 Export hf_static float-cache programs and register their Python runner
7. LLM runner: validate and drive hf_static float-cache programs #24572 LLM runner: validate and drive hf_static float-cache programs
8. Document the version-1 hf_static I/O contract #24573 Document the version-1 hf_static I/O contract

Each PR includes the commits of the ones above it; review only its own top commits.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/24572

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 0ef7a34 with merge base 5f72739 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.