Skip to content

feat(context-budget): keep plugin skill descriptions visible when the skill listing overflows #6458

Open
5 of 6 issues completed

Description

@kyle-sexton

Problem

The skill listing loaded at session start has a character budget: skillListingBudgetFraction, default 0.01 of the context window, with a per-skill cap set by skillListingMaxDescChars, default 1536 (https://code.claude.com/docs/en/settings-reference). When the listing goes over budget, descriptions are dropped. A dropped description means the model cannot tell when to invoke that skill.

Measured on 2026-10-05 with Claude Code 2.1.289, 30 headless runs on claude-haiku-4-5-20251001:

  • The listing overflowed in every run, even with every plugin disabled.
  • With a full melodic install, 317 of 329 skills lost their descriptions.
  • Turning plugins off saves almost no tokens, because overflowing descriptions are dropped anyway. The real per-session costs are agent listings, SessionStart hook text and MCP tool names.

The budget fraction is set inconsistently: 0.05 in this repo's .claude/settings.json, 0.03 in medley, and the default elsewhere.

Decide

  1. A fleet-wide skillListingBudgetFraction value, measured on Opus 5.5 and Fable 5.1 as well as Haiku. The budget scales with context size, so the overflow on a 1M-context model is unmeasured.
  2. A description cleanup across the melodic plugins:
    • shorter, sharper trigger text for model-invocable skills
    • disable-model-invocation on skills that only a person runs, so they leave the listing
  3. Whether /context-budget:audit should report listing overflow: which skills are dropped at a given budget.

Related: #6093 (measure-only audit mode). The plugin profiles design is the other lever: fewer active plugins per task.

🤖 Generated with Claude Code

Activity

  1. added
    needs-triageNot yet classified. Floor until a type and one priority tier are set.
    on Oct 5, 2026
  2. github-actions commented on Oct 5, 2026

    @github-actions
    Contributor

    Proposed work class: none

    Size: Large
    Complexity: Medium
    Uncertainty: Knowable
    Flags: intent
    Blocked by: none

    Decision recorded.

    Question (re-asked 2026-10-22): who may start repo-sweep, sync and tidy-work on their own? The earlier answer was given after the session wrongly said the flip "changes nothing"; all three were still disable-model-invocation: true.

    Answer: owner, Issue Triage session: "Go with recommendations".

    Decision: (b) only playbooks:repo-sweep becomes model-invocable; it works on its own branch behind a draft PR. repo-fleet-hygiene:sync (acts across every repo; --yes skips the script prompt, sync-fleet.sh:317-321) and session-flow:tidy-work stay human-started. #9134 carries the repo-sweep flip only. The grade table rows for sync and tidy-work (docs/conventions/invocation-mode/README.md:355-356) are corrected in a follow-up.

    #6458 (comment)

    The decision and the slice list settle item 2: the disable-model-invocation grading is split into slices, and description trimming is carried by #6846 and #6476. They don't settle items 1 and 3, so these stay open:

    1. Which fleet-wide skillListingBudgetFraction should be used, and must it be measured on Opus 5.5 and Fable 5.1 first? RECOMMENDED: measure first, then pick one value. Basis: the issue says the 1M-context overflow is unmeasured.
    2. Should /context-budget:audit report which skills are dropped at a given budget? RECOMMENDED: yes, as a separate slice after feat(context-budget): add a measure-only audit mode for consumer skills #6093. Basis: the issue names feat(context-budget): add a measure-only audit mode for consumer skills #6093 as related work.

    Automated triage. A maintainer reviews it.

  3. added
    needs-humanHuman-in-the-loop required; autonomous sessions must not resolve items carrying this.
    on Oct 5, 2026
  4. kyle-sexton commented on Oct 5, 2026

    @kyle-sexton
    MemberAuthor

    Decision (2026-10-05): keep skillListingBudgetFraction at its shipped default (0.01) and fix this through descriptions, not a bigger budget.

    • Claude Code drops the descriptions of the least-used skills first, and a larger fraction costs tokens in every session.
    • So this issue's item 1 is now: remove the overrides, 0.05 in this repo's .claude/settings.json and 0.03 in medley.
    • The main work is item 2 (description cleanup, and taking skills that only a person runs out of the listing), plus a way for new skills to get picked up despite starting at the bottom of the drop order.

    🤖 Generated with Claude Code

  5. kyle-sexton commented on Oct 5, 2026

    @kyle-sexton
    MemberAuthor

    Invocation audit (2026-10-05, Claude Code 2.1.289; docs + changelog + 10 Haiku runs)

    What the two flags do

    Listing Human /skill (interactive and claude -p) Skill tool (main, subagent, another skill) Agent skills: preload, /loop
    disable-model-invocation: true removed (name and description) works refused refused
    user-invocable: false stays (no savings) -p: refused silently (exit 0, empty result) works works

    Classifying all 345 melodic skills

    • 153 stay model-invocable.
    • 152 human-only: 74 already are, 78 are new.
    • 4 internal.
    • 36 unsure.

    The 78 new human-only skills save 44,818 listing chars. The model-invocable rest is still about 84k chars, so the descriptions themselves have to shrink too, and profiles still matter.

    Must stay model-invocable

    • review:code-review and review:security-review. The CI lanes reach them through a mid-prompt /skill; that path is documented but not verified in CI.
    • babysit-prs and pull-request (/loop).
    • 6 skills named by workflows, 6 preloaded by agents, and 26 *:check skills named by hooks.
    • 4 recurring-schedule skills reached through /work-items:work. These are unsure.

    Risk: repo-hygiene:clean is model-invocable and has destructive tiers. Split it.

    Full per-skill tables are kept locally, in invocation-audit.md and .csv. They are not attached here.

    🤖 Generated with Claude Code

  6. kyle-sexton commented on Oct 5, 2026

    @kyle-sexton
    MemberAuthor

    Direction change (2026-10-05): default to model-invocable.

    A human-only skill runs only when its / invocation starts the prompt. A pasted prompt that says "use the X skill" mid-sentence silently does nothing. The audit found a doc claim that a mid-prompt /skill works, but in practice it does not. Convenience wins over listing savings, including for skills with destructive modes that already ask for approval (repo-hygiene:clean).

    So:

    • Drop the 78 proposed new human-only flips.
    • Re-review the 74 existing human-only skills under the new default.
    • Cut listing cost through shorter descriptions and plugin profiles.

    🤖 Generated with Claude Code

  7. removed
    needs-triageNot yet classified. Floor until a type and one priority tier are set.
    on Oct 12, 2026
  8. kyle-sexton commented on Oct 13, 2026

    @kyle-sexton
    MemberAuthor

    Decision recorded. The owner accepted the adjudicated recommendation on 2026-10-12.

    Option 1: the #6805/#6846 position stands. Item 1 of the 2026-10-05T19:06 decision (remove the overrides) is superseded; this issue is re-scoped to the 2026-10-05T19:36 ask to re-review the human-only skills under the model-invocable default.

    Why: On origin/main the fraction is 0.07 (#6805, authored and merged by the owner 2026-10-10) and the owner-filed #6846 plans to keep an override and lower it later, but neither names the 2026-10-05T19:06 'keep 0.01, remove overrides' decision, which under precedence rule 2 is evidence of a course change, not proof, and the same question put to the owner on #6846 (2026-10-11T06:29) is unanswered. Upstream (settings-reference and skills page, fetched this run) sets the default at 0.01 with names kept and least-used descriptions dropped first, so at 3 bytes per token a 1M window buys about 30,000 characters against 146,814 for this repo's eligible descriptions alone and about 177,000 installed; 0.01 would strip most of the fleet, which defeats the owner's own 19:36 model-invocable, discoverability-first default. The re-review half is not in dispute: 74 skills carried disable-model-invocation: true at the 19:36 comment and 78 do now, none flipped, and the convention's check 24 only emits a hand-verify note, so nothing has done that work.

    Precedence applied: Rule 2 (newest owner decision), decided by its own caveat: #6805 (merged 2026-10-10) and the owner-filed #6846 are newer than the 2026-10-05T19:06 'keep 0.01, remove overrides' comment but neither names it, so they are evidence of a course change, not proof, and two owner decisions stand in tension; that is the brief's owner_must_decide case. Rule 4 (current upstream docs) supplies the facts behind the recommendation: default 0.01, names kept, least-used descriptions dropped first, plugin skills immune to skillOverrides. Rule 1 is not engaged: no stop condition or security boundary is touched. The research record's Q1 wording ('#6805 supersedes') ranks under rule 5 and overstated the evidence; its verify-block correction on Q2 (keep the re-review alive) holds and both researchers agree with it.

    Sources:

    • gh issue view 6458: owner comments 2026-10-05T19:06:08Z (keep 0.01, remove overrides), 19:31:22Z (invocation audit), 19:36:08Z (default model-invocable; re-review the 74 existing human-only skills); state OPEN, label needs-human
    • gh issue view 6846: filed by kyle-sexton 2026-10-10T21:48:08Z, calls chore(config): raise skillListingBudgetFraction to 0.07 so every description fits #6805 a stopgap, plans trim then lower the override with 15% headroom; comment 2026-10-11T06:29:50Z (self-described overnight worker, posted under the owner's account) asks which position wins, recommends chore(config): raise skillListingBudgetFraction to 0.07 so every description fits #6805's; handoff 2026-10-11T16:19:46Z NEEDS-HUMAN; no owner answer; labels needs-human, needs-triage
    • gh pr view 6805: author kyle-sexton, merged by kyle-sexton 2026-10-10T23:09:10Z; body opens 'No related issue', Related names chore(config): raise skillListingBudgetFraction to 0.05 so the enabled fleet fits #3767 only; measured 38 descriptions shed at 0.05 on a 1M claude-opus-5-5 session; post-merge listing capture still outstanding
    • git show origin/main:.claude/settings.json line 61: "skillListingBudgetFraction": 0.07 (commit 85e303b)
    • git grep '^disable-model-invocation: true' -- 'plugins//skills//SKILL.md': 78 on origin/main, 74 at 27fdf01
    • gh api repos/melodic-software/medley/contents/.claude/settings.json line 3: "skillListingBudgetFraction": 0.03

    Next: Read-only this pass. Once the owner answers on #6846 with option 1: (a) comment on #6458 with a dated line that item 1 (2026-10-05T19:06, keep 0.01 and remove overrides) is superseded by #6805 and #6846, that item 3 is shipped by /harness-ops:audit-skill-visibility, and that description trimming lives in #6846 and #6476 (standard merged in #6503); (b) retitle and re-scope #6458 to 'review the 78 disable-model-invocation: true skills against the invocation-mode exception classes', or file that as a successor and close #6458 as covered; (c) note on #6846 that the fraction is lowered only after a fresh-session listing capture shows 0 shed, and that the trims need attended model-graded before/after trigger-rate runs; (d) open a medley issue to re-measure its 0.03 under the same rule. No settings change in this repo; 0.07 is already on main.

    Researched by two independent agents (local evidence and upstream docs) and adjudicated by a third.

  9. kyle-sexton commented on Oct 18, 2026

    @kyle-sexton
    MemberAuthor

    This was generated by AI during backlog shaping.

    This issue is split into 4 pieces, each sized for one lane PR. The 2026-10-13 Decision recorded re-scopes this to re-reviewing human-only skills under the model-invocable default. origin/main has 79 disable-model-invocation: true skills: 64 *:setup (human-only by the setup contract, docs/plugin-philosophy.md:549), 9 non-setup graded KEEP on 2026-08-17 (docs/conventions/invocation-mode/README.md:216-235), and 6 non-setup never graded (discipline:be-ambitious, docs-naming:generate-file-name-gate, playbooks:repo-sweep, repo-fleet-hygiene:sync, session-flow:tidy-work, testing:map-features; grep of that README finds none). Whether the 2026-10-05 'convenien

    • filed: docs(invocation-mode): grade the six ungraded human-only skills against the exception classes (docs(invocation-mode): grade the six ungraded human-only skills against the exception classes #7974)
    • not filed: feat(plugins): flip the human-only skills the 2026 grade marks FLIP to model-invocable: depends-on-research: the sibling grade decides which of the six skills flip; then at most 5 skills per PR (one SKILL.md and one changelog fragment each)
    • not filed: docs(invocation-mode): re-grade the nine 2026-08-17 KEEP skills under the model-invocable direction: owner-decision: does the 2026-10-05 direction ('convenience wins, including for skills with destructive modes that already ask for approval') retire rubric class (i) for destructive-capable skills that carry their own confirmation gate (disk-hygiene:clean, repo-fleet-hygiene:apply, harness-ops:plugins, harness-ops:lanes)? Recommendation: yes for skills whose every destructive step is gated, keep class (i) for human-internal triggers (discipline:wait-what) and (iii) maintainer-only (dometrain:sync, firecrawl:update, playbooks:update); basis: feat(context-budget): keep plugin skill descriptions visible when the skill listing overflows #6458 comment 2026-10-05 and docs/conventions/invocation-mode/README.md:216-235. Record it as a Decision recorded. comment.
    • not filed: docs(invocation-mode): record whether setup skills stay human-only under the model-invocable direction: owner-decision: do the 64 *:setup skills keep disable-model-invocation: true under the setup contract? Recommendation: keep; basis: docs/plugin-philosophy.md:549 and docs/conventions/prerequisites/README.md:183 both rely on setup being unreachable by the model, and <plugin>:check skills already give the model a read-only path (docs/plugin-philosophy.md:614).

    Filed pieces are sub-issues of this one, and the paced queue admits them to the lane. Unfiled pieces wait on the blocker shown. The owner approved this split in an attended backlog-shaping session on 2026-10-18.

  10. kyle-sexton commented on Oct 18, 2026

    @kyle-sexton
    MemberAuthor

    Decision recorded.

    Question: On 2026-10-05 you changed the default to model-invocable, "including for skills with destructive modes that already ask for approval". Nine skills were graded KEEP human-only on 2026-08-17. Four of them are class (i) destructive or mutating: disk-hygiene:clean, repo-fleet-hygiene:apply, harness-ops:plugins and harness-ops:lanes. Options: (a) yes, flip every gated destructive skill; (b) yes, but only a skill that proves every destructive step stops for a person even when it runs unattended; (c) no, class (i) stands.

    Owner's answer: (b). Grade the four skills one at a time. Keep discipline:wait-what (its trigger is private to you), education:teach (choosing to enter a mode, not destructive, so the 2026-10-05 direction does not reach it) and the three class (iii) maintainer skills. Unattended lanes read untrusted issue and PR text while holding push credentials. A model-invocable destructive skill can be reached from that context, so its gate has to hold when no person is present. Correction: the earlier proposal left education:teach out of the keep list. Basis: docs/conventions/invocation-mode/README.md:93-109 (classes), :216-238 (2026-08-17 grade); #6458 comment 2026-10-05T19:36; AGENTS.md "lanes read untrusted issue and PR text while holding push credentials".

    The owner accepted this recommendation as item D3 of a 17-decision batch in an attended backlog-shaping session on 2026-10-18. The batch question: "Accept all 17 batch 5 decisions as recommended, D1, D2, D3, D5 and D17 included? I post a quoted 'Decision recorded.' comment on each of the 11 issues." The owner answered: "Accept all 17 (Recommended)".

    Posted by AI from the owner's answer in that session.

  11. kyle-sexton commented on Oct 18, 2026

    @kyle-sexton
    MemberAuthor

    Decision recorded.

    Question: 64 of the 79 human-only skills are *:setup. The setup contract makes them human-only because apply installs and configures things. Options: (a) keep disable-model-invocation: true on setup; (b) flip setup to model-invocable under the new default.

    Owner's answer: (a). The contract and the prerequisites convention both rely on the model being unable to run apply. <plugin>:check skills already give the model a read-only path. Basis: docs/plugin-philosophy.md:549-556; docs/conventions/prerequisites/README.md:183; git grep this session: 79 human-only, 64 of them setup.

    The owner accepted this recommendation as item D4 of a 17-decision batch in an attended backlog-shaping session on 2026-10-18. The batch question: "Accept all 17 batch 5 decisions as recommended, D1, D2, D3, D5 and D17 included? I post a quoted 'Decision recorded.' comment on each of the 11 issues." The owner answered: "Accept all 17 (Recommended)".

    Posted by AI from the owner's answer in that session.

  12. 2 remaining items

  13. kyle-sexton commented on Oct 22, 2026

    @kyle-sexton
    MemberAuthor

    Decision recorded.

    Question (re-asked 2026-10-22): who may start repo-sweep, sync and tidy-work on their own? The earlier answer was given after the session wrongly said the flip "changes nothing"; all three were still disable-model-invocation: true.

    Answer: owner, Issue Triage session: "Go with recommendations".

    Decision: (b) only playbooks:repo-sweep becomes model-invocable; it works on its own branch behind a draft PR. repo-fleet-hygiene:sync (acts across every repo; --yes skips the script prompt, sync-fleet.sh:317-321) and session-flow:tidy-work stay human-started. #9134 carries the repo-sweep flip only. The grade table rows for sync and tidy-work (docs/conventions/invocation-mode/README.md:355-356) are corrected in a follow-up.

  14. removed
    needs-humanHuman-in-the-loop required; autonomous sessions must not resolve items carrying this.
    on Oct 24, 2026
  15. kyle-sexton commented on Oct 24, 2026

    @kyle-sexton
    MemberAuthor

    Split into slices by the owner's HITL session (2026-10-24, C12 and Q93): this waited only on a gate the owner removed.

    Done already: #7974 (six ungraded skills graded), #9134 (repo-sweep flip). Open: #9176 (sync and tidy-work rows to KEEP).

    The four class (i) skills, graded one at a time per the 2026-10-18 decision (D3, option b), in order:

    1. docs(invocation-mode): re-grade disk-hygiene:clean under the 2026-10-18 decision #9706 disk-hygiene:clean
    2. docs(invocation-mode): re-grade repo-fleet-hygiene:apply under the 2026-10-18 decision #9707 repo-fleet-hygiene:apply, blocked by docs(invocation-mode): re-grade disk-hygiene:clean under the 2026-10-18 decision #9706
    3. docs(invocation-mode): re-grade harness-ops:plugins under the 2026-10-18 decision #9708 harness-ops:plugins, blocked by docs(invocation-mode): re-grade repo-fleet-hygiene:apply under the 2026-10-18 decision #9707
    4. docs(invocation-mode): re-grade harness-ops:lanes under the 2026-10-18 decision #9710 harness-ops:lanes, blocked by docs(invocation-mode): re-grade harness-ops:plugins under the 2026-10-18 decision #9708

    Description trimming is carried by #6846 and #6476.

  16. github-actions commented on Oct 25, 2026

    @github-actions
    Contributor

    The triage sweep dispatched intake triage for this issue (attempt 1 of 2).

  17. added
    priority: mediumReal value, no hard deadline; normal backlog flow.
    status: needs-decisionAwaiting a human or maintainer judgment call.
    needs-triageNot yet classified. Floor until a type and one priority tier are set.
    on Oct 25, 2026
  18. melodic-automation-lanes commented on Oct 25, 2026

    @melodic-automation-lanes
    Contributor

    The triage verdict left this issue at status: needs-decision:

    • flagged: intent
    • held by status: needs-decision
    • no lanes App verdict comment shows whether an earlier verdict was flagged
  19. melodic-automation-lanes commented on Oct 25, 2026

    @melodic-automation-lanes
    Contributor

    Refinement. A read-only run explored this repository, searched its issues and pull requests, and read the allowed docs to draft what follows. On an issue a listed human filed, it counts as the issue's spec once posted, while unedited; it is still model text, so check it.

    Intended outcome: Close or re-scope this umbrella: item 2 is carried by #​7974, #​9134, #​9176, #​9706-#​9710, #​6846 and #​6476, and item 3 already shipped, leaving only the fleet-wide fraction (item 1) open.

    Related work

    Open questions

    1. Is item 3 (audit reports which skills are dropped at a given budget) done by /harness-ops:audit-skill-visibility, so it can be struck from this issue?
      RECOMMENDED: Yes: the skill exists, so record item 3 as shipped. Confirm it reports which skills are dropped at a given fraction, then strike it. Basis: plugins/harness-ops/skills/audit-skill-visibility/SKILL.md; owner-adjudicated comment 2026-10-13 says it ships item 3
    2. Does the 0.07 override in .claude/settings.json stand as the fleet-wide value, superseding the 2026-10-05 'keep 0.01, remove overrides' decision?
      RECOMMENDED: Yes: keep 0.07 for now and move the lowering decision to #​6846, which lowers it only after a fresh-session listing capture shows 0 shed. Measure on Opus 5.5 and Fable 5.1 there. Basis: .base/.claude/settings.json:61; 2026-10-13 Decision recorded comment; #​6846 is not a candidate here, so its state is unchecked
    3. Should this umbrella be closed as covered by its slices, or retitled to track only the remaining work?
      RECOMMENDED: Close it once #​9176 and #​9706-#​9710 land. Until then keep it open as a tracker only, with no lane work of its own. Basis: 2026-10-24 slice comment; judgment
    4. Should medley's 0.03 override be re-measured under the same rule?
      RECOMMENDED: Yes: file it as a separate medley issue, as the 2026-10-13 decision proposed. It is outside this repo, so a lane run here cannot do it. Basis: 2026-10-13 Decision recorded comment, step (d)

    Sources read

    • /home/runner/work/claude-code-plugins/claude-code-plugins/.base/.claude/settings.json:61: skillListingBudgetFraction is 0.07 on main
    • /home/runner/work/claude-code-plugins/claude-code-plugins/.base/plugins/harness-ops/skills/audit-skill-visibility/SKILL.md: the skill exists
    • context.json comments 2026-10-13 and 2026-10-24: item 1 and item 2 re-scoped, slices named, description trimming carried by #​6846 and #​6476

    Automated refinement by the lanes App. Answer with a comment whose first line is **Decision recorded.**.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    priority: mediumReal value, no hard deadline; normal backlog flow.status: needs-decisionAwaiting a human or maintainer judgment call.

    Fields

    No fields configured for Feature.

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions