Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -1176,6 +1176,7 @@ jobs:
working-directory: benchmarks
run: |
PYTHONPATH=harness uv run --locked python -m unittest \
tests.test_progressive_provider_attempt \
tests.test_progressive_provider_plan \
tests.test_progressive_provider_run \
tests.test_progressive_run \
Expand Down
7 changes: 6 additions & 1 deletion benchmarks/Makefile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
.PHONY: install smoke smoke-python smoke-rust fly-adapter-static local-admission progressive-qualification-list progressive-qualification-plan progressive-qualification-run progressive-qualification-project-s20 progressive-qualification-binaries progressive-provider-plan qualification-operator
.PHONY: install smoke smoke-python smoke-rust fly-adapter-static progressive-provider-attempt-static local-admission progressive-qualification-list progressive-qualification-plan progressive-qualification-run progressive-qualification-project-s20 progressive-qualification-binaries progressive-provider-plan qualification-operator

install:
uv sync --locked
Expand All @@ -19,6 +19,11 @@ fly-adapter-static: install
PYTHONPATH=harness uv run --locked python -m unittest \
tests.test_fly_adapter tests.test_fly_tiny_qualification

# Provider-free proof of the whole-attempt state machine and recovery contracts.
progressive-provider-attempt-static: install
PYTHONPATH=harness uv run --locked python -m unittest \
tests.test_progressive_provider_attempt

local-admission: install
PYTHONPATH=$(CURDIR)/harness uv run --locked reframe -C reframe/settings.py -c reframe/checks -n '^LocalBenchExecAdmission$$' -l | grep -F LocalBenchExecAdmission
PYTHONPATH=$(CURDIR)/harness uv run --locked reframe -C reframe/settings.py -c reframe/checks -n '^LocalBenchExecAdmission$$' -r
Expand Down
21 changes: 17 additions & 4 deletions benchmarks/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -330,6 +330,19 @@ A successful rung emits exactly `sN-plan.json`, `sN-benchexec.json`,
engineering evidence. The canonical order remains S18, S19, S20, S22, S24,
S25, S26; the first failed or missing gate stops the planner.

The provider-free whole-attempt controller core is exercised with:

```bash
make -C benchmarks progressive-provider-attempt-static
```

It requires a schema-valid, commit-bound S18/S19 prefix; validates a closed
five-hour, integer-micro-USD spend authorization; binds the first admitted plan
before any provider mutation; advances one rung at a time; accepts only the
canonical five-file bundle; and persists an fsync-backed ownership ledger for
cleanup-only recovery. Its transport is injected, so this proof makes no
provider calls and spends nothing.

Provider credentials belong to Pulumi ESC rather than GitHub workflow inputs or
the caller's ambient shell. Live operator commands are rendered from
`config/gate-registry.json` and run through the Python control plane:
Expand All @@ -344,10 +357,10 @@ make -C benchmarks qualification-operator \
The operator uses the shell-free form `pulumi env run <environment> -- <argv>`;
secret values are never copied into its command line or evidence. The
`progressive-ladder` is registered, but still fails before opening ESC. The
offline image and rung runner do not implement whole-attempt Fly orchestration,
typed spend authorization, ownership-ledger recovery, or teardown inventory.
This is a deliberate live capability boundary, not a GitHub-dispatch
prerequisite.
typed whole-attempt state machine, ownership-ledger recovery, and sanitized
teardown inventory now exist offline; the real Fly transport, ESC environment
binding, and live recovery gate remain deliberately unavailable. This is a
live capability boundary, not a GitHub-dispatch prerequisite.

The controller derives bulk-ingest capability from the same run's bounded
ordinary `gf import-session commit --json` receipt: its construction evidence
Expand Down
Loading
Loading