feat(verification): type verification_spec with roles (schema foundation)

[geojaz]

Type verification_spec with roles (schema foundation)

Where this comes from

We've been working through how to implement the v1.0 scoring framework (the cat_v * c^0.5 * rec_v^0.5 outcome score). Computing those three signals means a task has to declare which checks are objectives (feed correctness c) and which are safeguards (feed rec_v/cat_v depending on severity). Today verification_spec is untyped (Any), so there is nowhere to put that distinction. This PR is the foundational piece: give the field a real schema with a role and severity, so the scoring layer has something to roll up. It is intentionally split from the machinery that consumes it so this part can land as a small, behavior-neutral change.

Vocabulary follows the task/verifier schema doc: roles are objective | safeguard, and a safeguard carries severity: recoverable | catastrophic.

Part 1 of 2.

What this does

• Adds VerificationEntry (name, role, severity, spec) and types Task.verification_spec as list[VerificationEntry] | None.
• role is one of objective | safeguard and defaults to objective, so every existing spec stays valid unchanged.
• severity (recoverable | catastrophic) is required when role: safeguard and forbidden when role: objective, enforced by a model validator.
• Declares optional mode and weight fields on BaseVerifier (and weight on VerificationResult). Additive metadata; nothing consumes them for scoring in this PR.
• The runner (VerifierAgent._run_leaf) is authoritative for a leaf's weight: it stamps the verifier's configured weight onto the result after verify() returns, so a custom verifier that builds a VerificationResult directly (bypassing _poll_to_result) can never silently drop it.
• Adapts the harness parse path to accept typed entries, preserving the existing verification_parse_errors behavior.

What this does NOT do

No new verifiers, no scoring logic, no runtime behavior change. tasks/common/optimize-scale parses and runs exactly as before (regression test included). The verifier vocabulary and the c/rec_v/cat_v rollup that consume role/severity/mode/weight land in Part 2.

[gemini-code-assist]

Code Review

This pull request introduces a typed schema for verification specifications by defining the VerificationEntry model and execution Mode literals, updating the Task schema to parse verification_spec as a list of VerificationEntry objects, and adding support for weighting verification results. The review feedback highlights several robust improvements: adding defensive checks when dumping verification_spec entries to avoid crashes with raw dictionaries, wrapping placeholder resolution in a try-except block to handle substitution failures gracefully, logging warnings for duplicate verification entry names to prevent silent overwrites, and ensuring that custom leaf verifiers properly propagate their weight to VerificationResult instances.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[gemini-code-assist]

If task.verification_spec contains raw dictionaries (e.g., if the task was loaded bypassing Task.from_dict or using a mock/stub in tests), calling e.model_dump() will raise an AttributeError and crash the harness. Use a defensive check to only call model_dump() if the object has that attribute.

Suggested change

	"verification_spec": (
	[e.model_dump() for e in task.verification_spec]
	if task.verification_spec is not None
	else None
	),
	"verification_spec": (
	[
	e.model_dump() if hasattr(e, "model_dump") else e
	for e in task.verification_spec
	]
	if task.verification_spec is not None
	else None
	),

[gemini-code-assist]

Wrap the placeholder resolution (_resolve_spec_placeholders) inside the try-except block as well. If placeholder substitution fails or raises an unexpected exception, it will currently crash the entire harness run instead of being gracefully caught and logged as a verification parse error.

Suggested change

	for entry in raw:
	spec_resolved = self._resolve_spec_placeholders(
	entry.spec, cluster_name, target_deployment, namespace
	)
	try:
	mapping[entry.name] = VerificationSpec(spec_resolved)
	except Exception as exc: # noqa: BLE001 - surface every failure
	_log.warning(
	"verification entry %r failed to validate; skipping: %s",
	entry.name,
	exc,
	)
	errors.append({"name": entry.name, "reason": str(exc)})
	for entry in raw:
	try:
	spec_resolved = self._resolve_spec_placeholders(
	entry.spec, cluster_name, target_deployment, namespace
	)
	mapping[entry.name] = VerificationSpec(spec_resolved)
	except Exception as exc: # noqa: BLE001 - surface every failure
	_log.warning(
	"verification entry %r failed to validate; skipping: %s",
	entry.name,
	exc,
	)
	errors.append({"name": entry.name, "reason": str(exc)})

[gemini-code-assist]

If multiple verification entries share the same name, the subsequent entries will silently overwrite the previous ones in mapping. This is likely an authoring error (e.g., copy-paste mistake) and can lead to verification checks being silently skipped. Consider logging a warning when a duplicate name is detected.

                    if entry.name in mapping:
                        _log.warning(
                            "Duplicate verification entry name %r detected; overwriting previous entry.",
                            entry.name,
                        )
                    mapping[entry.name] = VerificationSpec(spec_resolved)

[gemini-code-assist]

Since VerificationResult defaults weight to 1.0, any custom leaf verifier that overrides verify and instantiates VerificationResult directly (instead of using _poll_to_result) must explicitly forward self.weight to the result (i.e., VerificationResult(..., weight=self.weight)). Otherwise, the configured weight on the verifier will be lost in the result tree. Consider documenting this requirement or ensuring that the runner/agent automatically overrides the result's weight with the verifier's weight during evaluation.