feat(executor): reserved-source materialization (#376) + binding files= on all download paths (#377)#54
Merged
PaulFidika merged 1 commit intoJul 4, 2026
Conversation
…iles= on all download paths #376: producer-kind payloads carrying the reserved source struct now get a local snapshot at ctx.source_path (and ctx.source/ctx.destination metadata) before the handler runs, via the same ModelStore.ensure_local path as model bindings — identical retry/classification and ModelEvents. Empty source.ref is INVALID; absent source is a no-op. #377: Executor registers each endpoint binding by wire ref on the ModelStore; bare-ref callers (ModelOp DOWNLOAD/LOAD, lifecycle startup prefetch) now recover files=/provider instead of falling back to whole-repo downloads. Also fixes the fp32 pull root cause: select_hf_files no longer downloads root monolithic checkpoints in diffusers-layout repos (sd1.5's untagged v1-5-pruned*.safetensors won the root group despite fp16 component weights).
PaulFidika
deleted the
issue/376-377-source-materialization-binding-prefetch
branch
PaulFidika
added a commit
that referenced
this pull request
Jul 4, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes tracker issues #376 and #377 (agents/completed.md).
#376 — reserved-source materialization
Executor._run_jobextracts the reservedsource/destinationstructs from producer-kind (non-inference) payloads, populatesctx.source/ctx.destination, and materializespayload.sourceviaModelStore.ensure_local(same snapshot path, retry, RETRYABLE/INVALID classification, and ModelEvent emission as model bindings) before the handler runs.ctx._set_source_pathnow has a real caller; training-endpoints transform/quant producers get a materialized snapshot dir.source= no-op; present-but-emptyref= INVALID.#377 — binding files= ignored on prefetch/ModelOp paths
Executor.__init__registers every endpoint binding by wire ref on theModelStore(register_binding);ensure_localfalls back to the index for bare-ref callers (ModelOp DOWNLOAD/LOAD, lifecycle startup prefetch).files=/provider now apply on ALL download paths.select_hf_filestreated the untagged root monolithic checkpoints (sd1.5'sv1-5-pruned*.safetensors, 12GB) as their own weight group, and "untagged" always won for that group even though fp16 component weights exist. Diffusers-layout repos now exclude root single-file checkpoints entirely.Tests
tests/test_executor_source_materialization.py(5 tests): materialized dir + ctx.source metadata; missing-source no-op; empty ref INVALID; RETRYABLE/INVALID parity; inference kind ignoressource.tests/test_executor_prefetch_binding.py(4 tests): ModelOp + bare-ref store paths pass the binding'sallow_patterns/provider at the download-layer boundary (no network); explicit binding wins; unknown ref unpatterns.tests/test_hf_selection.py: sd1.5-shaped root-checkpoint exclusion test.Deferred: live cast-dtype run against the hub (needs hub infra + GPU; training-endpoints #37 deferral stands).
🤖 Generated with Claude Code