fix: [no-ticket] harbor - scoring integrity (dead attempts, best-rank monotonicity, strict reward key)

[shehabyasser-scale]

What

Three scoring-integrity fixes in HarborRunner, each closing a channel a live optimizer exploited or could exploit:

1. Dead attempts count 0.0 in mean aggregation (new n_dead metric). Attempts dying before the verifier scored were silently dropped from the mean, so the score estimated P(pass | attempt survived). Measured live in the Pawn replication: a no-retry candidate won selection at an inflated 0.233 while its retry-hardened successors measured an honest ~0.19 and lost; the optimizer documented the artifact in its own notes ("retry HELPS true quality but can LOWER measured train score"). All-dead samples error loudly instead of scoring 0.0.
2. 'best' trial ranking is monotone in the reward. The rank was (clean, has_rewards, recency), so with concurrent attempts 'best' meant "last clean attempt to finish" and a later clean 0.0 could clobber an earlier clean 1.0, violating the invariant documented in _load_trials. Reward now precedes recency in the key.
3. Strict reward_key extraction. A configured reward_key missing from a rewards dict no longer silently falls back to 'pass'/'reward' (attempts within one mean could be scored on different metrics), and several unrecognized keys are refused instead of averaged (a candidate could inflate the unweighted mean by emitting easy auxiliary metrics beside the real one). Sole-key dicts stay accepted.

Tests

tests/test_harbor_runner.py: 48 pass, including new pins for zero-fill (0.5 not 1.0), all-dead-errors, rank monotonicity, strict key, and refuse-to-average.

Stack

Based on #31 (harbor-4-mode-config-split). Part of the pre-paper hardening series; found by an 18-agent adversarial code sweep + live optimizer behavior.

🤖 Generated with Claude Code

Greptile Summary

This PR closes three scoring-integrity gaps in HarborRunner that a live optimizer was actively exploiting. The changes are well-motivated by empirical evidence and thoroughly covered by 48 tests.

• Dead-attempt zero-fill: Attempts that die before the verifier runs now contribute 0.0 to the mean rather than being silently dropped, closing a P(pass | attempt survived) bias. All-dead samples surface as an error (falling through to the existing "No verifier rewards" path) rather than scoring 0.0. A new n_dead metric lets operators distinguish infra noise from scored failures.
• Monotone _trial_rank: Reward is now a rank key ahead of recency, so a later clean 0.0 can no longer silently replace an earlier clean 1.0 under best aggregation. The method was also converted from @staticmethod to an instance method so it can call self._extract_reward.
• Strict _extract_reward: A configured reward_key missing from a rewards dict now returns None (unscorable) instead of silently falling back to pass/reward; multiple unrecognized keys are refused rather than averaged, preventing score inflation via auxiliary metrics. A sole-key unambiguous dict is still accepted.

Confidence Score: 5/5

The changes close three well-documented scoring-integrity gaps with minimal risk of regression; each fix is backed by at least one pinning test.

All three changes are narrowly scoped, internally consistent, and directly tested. The _trial_rank conversion from static to instance method is safe. The _extract_reward return-type widening to float | None is handled at every call site in the diff. No risky control flow or silent failures introduced.

No files require special attention — both changed files are straightforward and well-tested.

Important Files Changed

Filename	Overview
vero/src/vero/harbor/runner.py	Three well-reasoned integrity fixes: dead-attempt zero-fill with n_dead metric, reward-monotone trial ranking, and strict reward_key extraction with None fallback and clear error surfacing.
vero/tests/test_harbor_runner.py	New tests pin zero-fill behavior (0.5 not 1.0), all-dead error surfacing, rank monotonicity, strict key contract, refuse-to-average, and key-mismatch zero-fill in mean mode.

_{Reviews (2): Last reviewed commit: "test(harbor): pin reward-key-mismatch ze..." | Re-trigger Greptile}