feat: implement spec-driven directive parser

[rlippmann]

What changed

• Made the directive grammar parser authoritative for canonical directive recognition.
• Updated validate_directive(...) to enforce the normative grammar:
  • case-insensitive keywords and fixed tokens;
  • allowed whitespace normalization;
  • canonical-only parsing;
  • no malformed-input repair;
  • compound directive rejection.
• Updated engine parsing to consume canonical directives from the grammar layer before semantic evaluation.
• Updated tests and conformance fixtures to reflect the grammar boundary:
  • invalid syntax is no longer treated as semantic clarification;
  • lexical normalization cases are accepted;
  • semantic clarification remains handled after successful parsing.
• Restored engine coverage to 100% for the updated parser integration paths.
• Renamed one outdated test to reflect the current contract after removing pending-confirmation wording.

Why

The previous parser behavior mixed syntax handling and semantic evaluation. Near-canonical or malformed inputs could enter clarification paths even though they were not valid directives.

This change establishes a clearer boundary:

• grammar determines whether input is a canonical directive;
• semantic evaluation determines whether a valid directive can be applied.

This prevents invalid syntax from being treated as an authoritative state transition while preserving existing semantic conflict handling for valid directives.

Checklist

• pre-commit run (uv run pre-commit run --all-files)
• tests pass (uv run pytest)