docs(tutor-quality): re-assess open problems and propose the next architecture step

[ttbombadil]

Purpose

Closes the prompt series (v1/v3/v4) analytically and proposes the next step. Documentation only, no code, no provider calls.

Main finding

I replayed the real Course Content (UnoSim-Examples @ 2ac716f) through the real TutorService + CurriculumTutorAdapter, with a fake provider standing in for the model.

• In 16 of 35 examples, variables-and-serial is unresolved from the first turn.
  • Cause: concept serial-output requires variable-values, whose questions need int.
  • Effect: the Topic-driven Tutor never serves a planned question and stays content-exhausted for the whole session. On it03-05-millis the same happens right after long-values is mastered.
  • The runtime behaves as the SSOT says. The defect is in the content.
  • The authoring gate misses it because it only checks topics a quality case expects.
• The production content has no extensions. It is Topic schema v1, so the EXPAND extension path that the anchor corpus measures (and that the prompt series targeted) does not occur in production.

Decision and status (updated)

• Content: prerequisite removed. fix(tutor): let serial-output be learned without an int variable ttbombadil/UnoSim-Examples#4 brings unresolved-from-start activations from 16/36 to 0/36.
• Gate: catalog-wide invariant added in fix(tutor-quality): check every Topic an Example activates for a masterable LEARN path #149. It reports exactly the 16 Examples on the old content and passes on the fixed content.
• Fixtures: no small corpus PR. Adding LEARN evidence alone does not change the follow-up, because the planner excludes used questions only through the dialog history. Consistent fixtures need two things: a decision on the planner's repetition rule, and more post-mastery content.
• Model comparison: still sensible, but lower priority. It needs a fixed model ID from the user.

🤖 Generated with Claude Code