Skip to content

fix(tutor-quality): check every Topic an Example activates for a masterable LEARN path - #149

Merged
ttbombadil merged 1 commit into
mainfrom
fix/tutor-authoring-gate-example-activations
Oct 3, 2026
Merged

ttbombadil merged 1 commit into
mainfrom
fix/tutor-authoring-gate-example-activations

Conversation

@ttbombadil

Copy link
Copy Markdown
Collaborator

Purpose

The authoring gate missed the most severe deterministic Tutor defect in the production catalog. This PR extends it with a generic invariant, so that the anchor corpus and the real Course Content cannot drift apart unnoticed again.

Finding

tutor-quality-validator checked prerequisites, mastery capacity and the executable LEARN path only for Topics that a quality case expects. A Topic that a real Example merely activates was never checked.

In ttbombadil/unosim-examples (2ac716f, the default Course Content source in server/config.ts), variables-and-serial is activated on 16 of 35 Examples and unresolved from the first turn: its Concept serial-output required an int-only prerequisite. On those Examples the Topic-driven Tutor never serves a planned question and stays content-exhausted for the whole session. The content fix is in ttbombadil/UnoSim-Examples#4.

Invariant

For every Example in the manifest and every Topic the production matcher activates on its main sketch, a learner who answers every planned question successfully must reach Topic mastery. The path uses the production planner, Topic classification and the Example's effective LEARN strategy.

  • It needs no quality case, no list of known Examples or Topics, and no provider.
  • It finds Topics that are unresolved from the first turn: unprobeable prerequisites, missing applicable questions, insufficient mastery capacity.
  • Legitimate later exhaustion is not forbidden: neither exhaustion after weak answers (the strong path is the best case) nor post-mastery (DEEPEN) capacity is checked here. DEEPEN capacity stays checked for expected Topics only, as before.
  • New issue code unmasterable-example-activation carries exampleId (also printed by the CLI).
  • The strict LEARN loop is extracted into strictLearnPath and shared with the existing per-case check; that check's behavior is unchanged.
  • TutorQuality SSOT §4 documents the invariant.

RED / GREEN

result
6 new validator tests (unprobeable prerequisite; same Topic without it; probeable prerequisite; Topic not activated; later exhaustion allowed; Example strategy used) RED (function missing) → GREEN
directory test: Example without quality case activates an unmasterable Topic RED with the old directory validator → GREEN
validate:tutor-course-content on real content 2ac716f exit 1, 16 × unmasterable-example-activation (exactly the 16 known Examples, nothing else)
same on the content fix (UnoSim-Examples#4) exit 0

Rollout order

  1. Merge UnoSim-Examples#4 first. Its CI still uses the validator pinned at aaa52fad and passes.
  2. Merge this PR.
  3. Then bump .unosim-compatible-commit in UnoSim-Examples so that its CI enforces the invariant.

Merging this PR before #4 makes the local validate:tutor-course-content against the current content red. UnoSim's own CI is unaffected, because it uses synthetic fixtures.

Gates

Gate main (db272a84) this PR
npm run check 0 errors 0 errors
npm run test:tutor-quality 380 387 (1.7 s / 15 s)
npm run test:unit 2642 + 1 skipped 2649 + 1 skipped
npx eslint . --ext .ts,.tsx 0 errors / 10 warnings in tracked files identical
npm run check:docs, git diff --check pass pass
pre-push (unit + Sonar) – pass, quality gate PASSED

No provider calls; no prompt, Judge, criteria or verdict change.

🤖 Generated with Claude Code

…erable LEARN path

The authoring gate checked prerequisites and the executable LEARN path only
for Topics that a quality case expects. A Topic that a real Example merely
activates was never checked. In the production catalog, variables-and-serial
was activated on 16 of 35 Examples and unresolved from the first turn (its
Concept serial-output required an int-only prerequisite), which blocked the
Topic-driven Tutor with content-exhausted for the whole session.

New catalog-wide invariant, independent of quality cases: for every Example
in the manifest and every Topic the production matcher activates on its main
sketch, a learner who answers every planned question successfully (production
planner, the Example's effective LEARN strategy) must reach Topic mastery.
No provider is involved; weak-answer exhaustion and DEEPEN capacity stay out
of scope. Issue code unmasterable-example-activation, with the Example ID.

The strict LEARN loop moves into strictLearnPath and is shared with the
existing per-case executable-path check (behavior unchanged).

RED/GREEN: 6 validator tests and 1 directory test (RED without the wiring).
Against the real Course Content: 16 issues at 2ac716f, none with the
serial-output prerequisite removed. TutorQuality SSOT §4 states the check.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ttbombadil added a commit that referenced this pull request Oct 3, 2026
…refined fixture finding

Prerequisite removal (UnoSim-Examples#4) and the catalog-wide authoring gate
(#149) are decided and implemented; the catalog has no Topic left that is
unresolved from the first turn. The fixture finding is refined: missing
LEARN evidence alone does not cause the recurring follow-up, because the
planner excludes used questions only through the dialog history; consistent
fixtures need a planner repetition decision and more post-mastery content.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ttbombadil
ttbombadil merged commit 5042bd0 into main Oct 3, 2026
5 checks passed
@ttbombadil
ttbombadil deleted the fix/tutor-authoring-gate-example-activations branch October 3, 2026 16:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant