fix: [no-ticket] harbor - floored rewards name their cause (candidate crash vs infra outage)

[shehabyasser-scale]

Stacked on #36. Motivated by a live wave-1 incident: in the E5 transfer matrix, three of five champions scored 0/72 on claude-opus-4-8 because their optimizers had hardcoded temperature=0 (a variance trick on their home model) and litellm rejects it for opus. Separately, a budget-exhausted key produced dead attempts that nearly got booked as a champion property. In both cases the harness behaved safely but the record could not distinguish the two floored cases, and they demand opposite actions (report the portability failure vs invalidate and re-run).

Changes

1. Collation names the crash (runner.py). The no-verifier-rewards error string now carries the dead attempts' exception types: No verifier rewards for task 't'. (attempts died: UnsupportedParamsError x6). The error string is the one field that flows everywhere (DB, per-sample files, verifier), so the cause survives to every consumer.

2. Floored targets carry their cause (verifier.py). _admin_eval_score now returns (score, failure_cause); when a target is floored after retries, target_errors[key] includes a frequency summary of the dominant per-sample errors (top 3). One identical cause across every sample is the signature of a deterministic candidate crash; a mixed bag points at infra. The cause extraction is fail-safe (any surprise result shape degrades to a fixed string) so diagnostics can never fail finalize.

Tests

• Runner: error string carries UnsupportedParamsError x2 for attempts dead of the same cause.
• Verifier: an all-errored target floors to 0.0 with target_errors naming the dominant cause and its sample count; existing floor/retry/recover tests unchanged.
• 85 passed across the two affected suites.

🤖 Generated with Claude Code

Greptile Summary

This PR improves observability for floored (0-scored) evaluation results by embedding the failure cause into the error strings that flow to every durable record. Two previously indistinguishable floored outcomes — a deterministic candidate crash vs an infrastructure outage — now produce distinct, actionable messages.

• runner.py: When a trial produces no verifier rewards, the error string now tallies each dead attempt's exception type (e.g. attempts died: UnsupportedParamsError x6), giving downstreams a signal without needing to re-examine attempt-level data.
• verifier.py: _admin_eval_score is promoted to return (score, failure_cause); when a target is floored after retries, target_errors includes a frequency summary built by _dominant_sample_errors, which normalises task names via regex before grouping so a deterministic crash across a multi-task slice still reads as one dominant cause rather than N singletons.
• Tests cover both paths, including a regression test for the multi-task-name clustering that was flagged in the prior review thread.

Confidence Score: 5/5

Safe to merge — the diagnostic additions are all additive, the tuple return change is applied at every call site, and the failure-cause extraction is wrapped in a broad except so it can never break finalize.

All three callers of the refactored _admin_eval_score unpack the new tuple correctly. The _dominant_sample_errors helper degrades gracefully on any unexpected result shape. Runner-side cause building uses defensive .get() guards throughout. Two new tests (including one that directly covers the previous review concern about multi-task clustering) pass alongside the unchanged floor/retry/recover suite.

No files require special attention.

Important Files Changed

Filename	Overview
vero/src/vero/harbor/runner.py	Adds dead-attempt exception-type tallying to the no-rewards error string; logic is safe and handles missing keys with or {} guards throughout
vero/src/vero/harbor/verifier.py	_admin_eval_score promoted to return (score, cause) tuple; new fail-safe _dominant_sample_errors normalises task names via regex before grouping; all three call sites updated; _maybe_score_baseline correctly calls engine directly and is unaffected
vero/tests/test_harbor_runner.py	Adds test asserting exception type appears as "UnsupportedParamsError x2" in error string when two dead attempts share the same exception type
vero/tests/test_harbor_verifier.py	Adds two new tests: one verifying the all-errored floor path records dominant cause, and one specifically covering the multi-task-name clustering fix (the prior review thread concern)

_{Reviews (2): Last reviewed commit: "fix(harbor): dominant-cause grouping nor..." | Re-trigger Greptile}