fix: [no-ticket] harbor - ops integrity (exhaust-budget instruction lever, fail-closed ledger restore)

[shehabyasser-scale]

Stacked on #35 (harbor-8-honest-signals). Closes out the pre-paper hardening series (#34 access tiers, #35 honest signals, #36 ops).

Changes

1. instruct_exhaust_budget build-config lever (build/config.py, compiler.py, instruction.md.j2, mirrored into serve.json like instruct_multifidelity). The compiled instruction's "unspent budget is wasted: re-measure your champion or try one more variant" bullet was unconditional; it is now a flag, default on (current behavior byte-for-byte in spirit; the bullet splits from the "scores are noisy" fact, which stays unconditional). Off makes stopping-early the agent's own choice. This is the ablation arm for the persistence experiments: with the exhortation always baked in, there is no way to measure what it contributes to optimizer persistence vs what the optimizer does on its own.

2. Ledger restore fails closed (serve.py). _load_or_build_ledger restored the configured (full) budgets whenever the persisted ledger.json existed but could not be parsed. The persisted ledger exists precisely so a sidecar restart cannot refill spent budget; falling back to full on a parse failure refunds the agent everything already spent, so any crash that corrupts the flush mints budget. Now: a missing file is still a fresh boot (configured budgets), but an unreadable file restores every metered split with zero remaining, preserves the unreadable original as ledger.corrupt for the operator, and logs at error level with the recovery instruction (delete ledger.json deliberately to boot fresh). Admin and finalize paths are unaffected; only agent metering closes.

Tests

• Instruction lever: default renders the bullet; instruct_exhaust_budget=False omits it while keeping "Scores are noisy".
• Ledger: missing file boots configured; unparseable file restores zero-remaining and preserves ledger.corrupt byte-for-byte; structurally-malformed JSON entries (KeyError path) also fail closed.
• 48 passed, 1 skipped (pre-existing) on the affected files.

🤖 Generated with Claude Code

Greptile Summary

This PR closes out the pre-paper hardening series with two self-contained changes: a build-config lever (instruct_exhaust_budget) that makes the "spend your remaining budget" instruction bullet optional for ablation experiments, and a fail-closed fix to ledger restore that treats a corrupt ledger.json as fully spent rather than silently refunding the agent everything it had already used.

• instruct_exhaust_budget lever (build/config.py, compiler.py, instruction.md.j2, serve.py): the budget-exhaustion bullet is now conditional on a flag (default on, preserving current behavior); off gives the ablation arm needed to isolate what the exhortation contributes to optimizer persistence.
• Ledger fail-closed (serve.py): a missing file still boots with configured budgets; a file that exists but cannot be parsed now zeroes all metered remaining_* fields, saves the corrupt original to ledger.corrupt, and logs at error level with the deliberate-delete recovery instruction.
• Additional fixes in server.py and runner.py: free-baseline claim is now atomically set before the await (preventing concurrent double-free), volume-reuse ordinal scan resumes past surviving eval dirs on restart, score_se is renamed to mean_score_se for precision, empty transcript files skip rather than surface as empty-string feedback, and crash transcripts are attached to the no-rewards error path.

Confidence Score: 5/5

All changes are defensive fixes with no new attack surface; the most sensitive path (ledger restore) now fails closed rather than open.

Every changed path has a targeted test added alongside it. The ledger fail-closed logic is straightforward and the zeroing loop is easy to audit. The async free-baseline fix correctly covers the BaseException refund path. No pre-existing guarantees are weakened.

The score_se to mean_score_se rename in server.py is a breaking schema change to summary.json; worth confirming all downstream consumers are updated.

Important Files Changed

Filename	Overview
vero/src/vero/harbor/serve.py	Fail-closed ledger restore: corrupt ledger.json now zeroes all metered budgets instead of refunding full configured budgets; saves backup to ledger.corrupt and logs at error level. Also adds instruct_exhaust_budget field mirroring build config.
vero/src/vero/harbor/server.py	Three fixes: free-baseline claim moved before await to prevent concurrent double-free; volume-reuse ordinal scan on restart to avoid wiping prior session eval dirs; score_se renamed to mean_score_se in summary.json.
vero/src/vero/harbor/build/config.py	Adds instruct_exhaust_budget bool (default True) to _BuildConfigBase with thorough inline documentation explaining the ablation purpose.
vero/src/vero/harbor/runner.py	Two fixes: no-rewards error path now carries _failure_feedback so the optimizer can see what crashed the agent; empty transcript files are skipped rather than returned as empty string.
vero/src/vero/harbor/protocol.py	Corrects build_status default k_anonymity_floor from 1 to 5 to match EvaluationSidecar enforcement default; prevents agents from being advertised a laxer floor than the sidecar enforces.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Sidecar startup] --> B{ledger.json exists?}
    B -- No --> C[Fresh boot: configured budgets]
    B -- Yes --> D{JSON parse ok?}
    D -- Yes --> E[Restore persisted remaining values]
    D -- No --> F[Copy to ledger.corrupt]
    F --> G[Reconstruct from config with zero remaining]
    G --> H[Log ERROR + recovery instruction]
    C --> I[BudgetLedger in memory]
    E --> I
    H --> I

    J[evaluate called] --> K{free baseline?}
    K -- Yes --> L[Claim freebie BEFORE await]
    L --> M[await engine.evaluate]
    M -- exception --> N[Refund freebie then re-raise]
    M -- success --> O[Route results to versioned dir]
    K -- No --> M

    P[_route_results called] --> Q{_eval_seq == 0?}
    Q -- Yes --> R[Scan results dirs for max ordinal]
    R --> S[Resume past max ordinal]
    Q -- No --> T[Increment and write dest dir]
    S --> T

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
    A[Sidecar startup] --> B{ledger.json exists?}
    B -- No --> C[Fresh boot: configured budgets]
    B -- Yes --> D{JSON parse ok?}
    D -- Yes --> E[Restore persisted remaining values]
    D -- No --> F[Copy to ledger.corrupt]
    F --> G[Reconstruct from config with zero remaining]
    G --> H[Log ERROR + recovery instruction]
    C --> I[BudgetLedger in memory]
    E --> I
    H --> I

    J[evaluate called] --> K{free baseline?}
    K -- Yes --> L[Claim freebie BEFORE await]
    L --> M[await engine.evaluate]
    M -- exception --> N[Refund freebie then re-raise]
    M -- success --> O[Route results to versioned dir]
    K -- No --> M

    P[_route_results called] --> Q{_eval_seq == 0?}
    Q -- Yes --> R[Scan results dirs for max ordinal]
    R --> S[Resume past max ordinal]
    Q -- No --> T[Increment and write dest dir]
    S --> T

Loading

_{Reviews (2): Last reviewed commit: "fix(harbor): review follow-ups from #34/..." | Re-trigger Greptile}