Skip to content

#17 refactor: self-updating model catalog, then trim agent-notes to its mission - #31

Merged
verkligheten merged 16 commits into
developfrom
refactor/model-catalog-and-mission-trim
Jul 29, 2026
Merged

#17 refactor: self-updating model catalog, then trim agent-notes to its mission#31
verkligheten merged 16 commits into
developfrom
refactor/model-catalog-and-mission-trim

Conversation

@verkligheten

Copy link
Copy Markdown
Member

Implements epic #17 — make the model catalog self-updating, then trim agent-notes back to its mission: a shared template of skills, agents, and a role→model→effort map.

144 files changed, +3,852 / −5,816 (net −1,964 LOC). Suite green: 1780 passed, 15 deselected.

Three live bugs found and fixed

These were not in the original scope. They surfaced while verifying the plan's assumptions.

  1. Claude Opus 5 was billed at 3×$15/$75 instead of $5/$25. Two defects compounded: normalize_model only rewrote dashed pairs, so claude-opus-5 passed through untouched and then hit the *opus* catch-all in the legacy pricing row. No warning fired, because something did match. Verified against real session files, where claude-opus-5 is the exact string reaching the pricer.
  2. claude-opus-4-10 and 4-11 priced as legacy — surfaced only once model IDs were stored in a single canonical form. Fixed by anchoring the globs so *opus-4-1 cannot swallow 4-10.
  3. claude-fable-5 was unreachable as a default — it declared model_class: fable, but no role declares typical_class: fable. Now classified into the frontier class and marked never_default, so it stays selectable by pin but is never auto-assigned. Fable is a frontier model, but too expensive to hand an agent by default.

What changed

Catalog (#18, #20, #21, #22)data/models/*.yaml (14 hand-maintained files) is replaced by three layers: a committed seed.json snapshot of what providers publish, a rules.yaml judgment layer (classes, families, deprecation, capabilities), and an optional user override. agent-notes models refresh fetches the two provider endpoints and writes a cache; models freeze promotes that cache to the seed for committing. Refresh → review diff → freeze → commit is now the whole model-update workflow — no code change per provider release.

build never touches the network. It reads ~/.cache/agent-notes/catalog.json, falling back to the committed seed. Only refresh fetches, via a lazily-imported stdlib urllib.requestno new dependency.

The catalog swap was gated: test_catalog_equivalence.py asserts the generated catalog matches the previous 14 models field-for-field. The YAMLs were deleted only after it was green.

Scope narrowed to anthropic + openai — github-copilot aliases dropped. #29 (adopt models.dev) was closed as not-planned on the same grounds: every install in state.json is claude only, so the Bedrock/Vertex/Copilot alias tables it uniquely provided are not reachable today.

Mission trim (#19, #23, #24, #25, #26) — deleted the wiki memory backend (~2,500 LOC across 8 modules, plus the wiki-compiler agent), dead code (tiers, Model.pricing, strip_memory_section, doctor_checks.py), and three skills that Superpowers ships longer versions of (tdd, brainstorming, writing-great-skills). Memory and cost-report were decoupled from rendering.py, hook.py and installer.py and colocated under agent_notes/memory/ and agent_notes/cost/.

A state.json with memory.backend: "wiki" now raises a clear migration error — never a silent switch to another backend.

Credential guard (#27) — fixed false positives that were blocking work on this repo: template files (credentials.example.yml, secrets.sample.json), source files (services/credentials.py), and keyword-only commands. A template marker must be its own dotted component, and hard-secret extensions (.enc, .key, .pem, …) are never exempt, so foo.example.key stays blocked. A security audit of the first implementation found six bypasses — bash -c, dd if=, python -c, example.env, secret/config.py, and --flag=path — all confirmed empirically and closed. data/rules/safety.md was updated so the shipped doc matches the guard.

Dependencies (#28) — dropped the tomli conditional, the project's only version-conditional dependency, and raised the Python floor to 3.11 so tomllib covers the single TOML read path. tomli-w stays: it is the Codex writer, and hand-rolling TOML escaping on a path that carries agent prompts and secret material is not worth removing a 10 KB pure-Python dependency.

Verification

  • uv run pytest tests/ -v → 1780 passed, 15 deselected
  • agent-notes build compared under a pinned XDG_CONFIG_HOME against a git worktree built at the branch point — dist changes are limited to the deliberate ones (wiki agent and three skills removed, safety.md rules text)
  • Cache isolation regression-tested: a developer's ~/.cache/agent-notes/catalog.json can no longer silently drive the test suite

Follow-up

#30 remains open — reinstall from this branch to clear the stale pre-#27 guard, then remove the two workarounds it forced (an orphaned test module with its collect_ignore entry, and a dead import shim in the credentials service).

Closes #18, #19, #20, #21, #22, #23, #24, #25, #26, #27, #28

Note: this PR targets develop, so GitHub will not auto-close those issues on merge — they close when develop reaches main.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant