fix(answer_length, inconsistent_format): check only what the task's scorers compare

[MattFisher]

Summary

answer_length and inconsistent_format ask one question: could this answer's form cost it a point under the task's scorer? Until now any verbatim scorer in the task meant "yes" for every check. inspect_ai's scorers normalise before they compare, so the answer is often "no".

Scorer (default arguments)	Case	Trailing punctuation	Length
exact	folded	stripped	counts
match	folded	stripped	counts
includes	folded	counts	counts
pattern	counts	counts	counts
f1	folded	stripped	partial credit

Two changes, kept separate in the code and the tests:

1. What each scorer compares. inconsistent_format runs its capitalisation, punctuation and length checks only for an aspect some scorer of the task is thrown by. Under exact alone, only length outliers are checked now, because case and punctuation can't change an exact score. A scorer someone adds to VERBATIM_SCORERS without an entry is assumed to compare everything, as before.
2. Partial credit. Beside f1, a long answer still scores, so answer_length and the length check don't apply. This reverses the decision in Answer rules ignore the task's scorer and choices #33 that exact beside f1 "measures as before". The case behind it is SQuAD, scored with both: on a sweep of every Inspect Evals eval, these two scanners reported 1,495 findings on SQuAD's span answers and nowhere else. None of them could change a score.

A scanner that doesn't apply now says which aspects the task's scorers ignore, for example "…this task scores with inspect_ai/f1, inspect_ai/exact, which do not depend on the answer's length (f1 gives partial credit)". Under a scorer that compares no text, such as a code-execution or choice scorer, the reason is unchanged.

Testing

• tests/test_answers.py: what each scorer compares, partial credit beside exact, and the short reason for non-text scorers.
• tests/test_inconsistent_format.py: exact checks only length; includes checks punctuation, not case; length outliers are still flagged under exact; exact with f1 doesn't apply.
• tests/test_answer_length.py and tests/test_cli_task_scorers.py: exact with f1 doesn't apply.
• The two existing "as before" tests now use exact (answer_length) and pattern (inconsistent_format) instead of exact with f1.

The new tests failed first. Suite: 650 passed, excluding the browser e2e test. basedpyright: 0 errors. pre-commit passes.

🤖 Generated with Claude Code