perf(skills): trim skill descriptions until the listing fits, then lower skillListingBudgetFraction #6846

Activity

1. github-actions

   added

   needs-triageNot yet classified. Floor until a type and one priority tier are set.Not yet classified. Floor until a type and one priority tier are set.

   on Oct 10, 2026
2. github-actions commented on Oct 10, 2026

   

   github-actionsBot

   on Oct 10, 2026 – with GitHub Actions

   Last edited by melodic-automation-lanes

   Contributor

   More actions

   Proposed work class: work-class: scoped
   Basis: automated intake triage; the lanes App applied this class.

   Size: Small
   Complexity: Low
   Uncertainty: Knowable
   Flags: none
   Blocked by: #6476

   Decision recorded. The owner accepted the adjudicated recommendation on 2026-10-12.

   Option A: keep an override. After the #6476 rollout trims are measured, lower skillListingBudgetFraction from 0.07 to the smallest value that fits the installed listing on a 1M window at 3 bytes per token with 15% headroom and no descriptions shed. 0.07 stays until then.

   [quote truncated; the full decision is at the link]

   #6846 (comment)

   The decision settles the open questions. The remaining work is to re-measure the listing after the #6476 trims, set the new skillListingBudgetFraction in .claude/settings.json, and update docs/cloud-sessions.md. The fresh-session listing capture needs an attended claude run, which this lane can't do. It is not a reason to hold the rest.

   _{Automated triage, applied by the lanes App.}
3. github-actions

   added

   priority: mediumReal value, no hard deadline; normal backlog flow.Real value, no hard deadline; normal backlog flow.

   on Oct 10, 2026
4. kyle-sexton commented on Oct 11, 2026

   

   kyle-sexton

   on Oct 11, 2026 – with Claude

   MemberAuthor

   More actions

   Overnight worker: stopped before any change. One decision is needed first.

   Question: which budget target does this issue serve?

   The two recorded positions conflict:

   • feat(context-budget): keep plugin skill descriptions visible when the skill listing overflows #6458, your decision of 2026-10-05: keep skillListingBudgetFraction at the shipped default (0.01), remove the repo override, and let Claude Code drop the least-used descriptions.
   • chore(config): raise skillListingBudgetFraction to 0.07 so every description fits #6805, which you merged on 2026-10-10, and this issue: keep an override (now 0.07 at .claude/settings.json:49), then lower it to the smallest value that fits with 15% headroom and 0 descriptions shed.

   Acceptance criteria 2 and 3 here (lowered fraction, 0 shed) only make sense under the second position.

   Options:

   1. (Recommended) The chore(config): raise skillListingBudgetFraction to 0.07 so every description fits #6805 position stands. Keep an override, trim, then lower it as this issue says. Record on feat(context-budget): keep plugin skill descriptions visible when the skill listing overflows #6458 that its item 1 is superseded. Reason: it is the later of the two, and at 0.01 on a 1M window (about 30,000 characters at 3 bytes per token), no amount of trimming fits this repo's listing alone.
   2. The feat(context-budget): keep plugin skill descriptions visible when the skill listing overflows #6458 position stands. Rewrite this issue to "trim descriptions, then remove the override", and drop criteria 2 and 3.
   3. Split the work. File the trims as their own issue, since both positions want them, and decide the fraction after the trims are measured.

   Second blocker, whichever option you pick: the trims cannot ship from an unattended run tonight.

   • The skill criteria (plugins/playbooks/skills/skill-authoring/reference/skill-criteria.md, Measurement) require a model-graded before/after trigger-rate run for each trimmed description. The lexical listing-overlap floor is only a tripwire.
   • /skill-quality:check says fleet-wide description rewrites stay attended.
   • This worker's worktree isolation refuses claude plugin eval.

   Current state:

   • No branch and no commits. Nothing was changed.
   • One measurement for whoever picks this up, taken at 8fa9df0 by a frontmatter scan of plugins/*/skills/*/SKILL.md, skipping disable-model-invocation:
     • This repo has 280 listing-eligible skills, with 146,814 description characters in total.
     • The 50 longest add up to 42,971 characters, and the shortest of them is 684.
     • Halving those 50 saves about 21,500 characters. That is short of the 28,000 in criterion 1, so either more skills get trimmed or the target counts skills from outside this repo too (the 177k measure was an installed listing).

   Next step once you answer: an attended session takes the first batch, for example the 10 longest from testing:audit (1,010) down to skill-quality:check (982). For each skill it freezes a probe set plus a held-out set and runs before and after concurrently. It ships only the trims that pass non-inferiority, with Refs: #6846.

   I added needs-human to this issue.

   ---

   Generated by Claude Code
5. kyle-sexton

   added

   needs-humanHuman-in-the-loop required; autonomous sessions must not resolve items carrying this.Human-in-the-loop required; autonomous sessions must not resolve items carrying this.

   on Oct 11, 2026
6. kyle-sexton commented on Oct 11, 2026

   

   kyle-sexton

   on Oct 11, 2026 – with Claude

   MemberAuthor

   More actions

   Handoff note for whoever picks this up next. The overnight Claude run has stopped, and Kyle is moving issue work to Cursor.

   • Branch: none
   • PR: none
   • State: NEEDS-HUMAN
   • Notes: Depends on feat(context-budget): keep plugin skill descriptions visible when the skill listing overflows #6458 vs chore(config): raise skillListingBudgetFraction to 0.07 so every description fits #6805 budget conflict (question on issue, recommends chore(config): raise skillListingBudgetFraction to 0.07 so every description fits #6805's 0.07 stands). Trims also need paid model-graded before/after evals per skill, so not unattended either way
   • Next step: Wait for the owner decision asked on this issue, then continue on the branch if there is one.

   Where a branch is listed it is pushed, so it can be taken over as is. A draft PR shows a red ci-status by design until it is marked ready.

   ---

   Generated by Claude Code
7. kyle-sexton commented on Oct 13, 2026

   

   kyle-sexton

   on Oct 13, 2026

   MemberAuthor

   More actions

   Decision recorded. The owner accepted the adjudicated recommendation on 2026-10-12.

   Option A: keep an override. After the #6476 rollout trims are measured, lower skillListingBudgetFraction from 0.07 to the smallest value that fits the installed listing on a 1M window at 3 bytes per token with 15% headroom and no descriptions shed. 0.07 stays until then.

   Why: Precedence rule 2: the newest explicit owner decision on the fraction is #6458 (2026-10-05, keep 0.01 and remove the overrides); PR #6805 (merged 2026-10-10 by the owner) and #6846 (filed 2026-10-10) take the opposite position but never name that decision, so they flag a possible change of course without proving one, and current upstream facts (default 0.01 means about 30,000 chars on a 1M/3-byte window against a 146,814 to 153,232 char repo listing; skillOverrides name-only does not apply to plugin skills) make the 0.01 position a knowing acceptance of shedding rather than a fit target. The trims are wanted under both positions and already have an owner-approved, eval-gated home in #6476 (brief confirmed 2026-10-07; standard and tooling merged in #6503, #6529, #6539, #6546), so a separate trim issue would be a parallel second way. Changing the fraction now in either direction is premature: lowering it sheds about 38 descriptions again, and the issue's own criterion 2 says to lower it only after re-measuring.

   Precedence applied: Rule 2 (newest explicit owner decision): #6458 2026-10-05 is the last stated decision on the fraction; #6805 and #6846 are later owner-account actions that do not name it, so per the brief they flag a possible change of course and the conflict goes to the owner. Rule 2 also settles where the trims go: the #6476 brief (2026-10-07) is an explicit owner decision that concise-skill rewrites run fleet-wide per plugin with eval gates, and no later decision carves description trims out of it. Rule 4 (current upstream docs for 2.1.296) supplies the facts the owner weighs; no ADR applies.

   Sources:

   • origin/main 736fb71 .claude/settings.json:61 (skillListingBudgetFraction 0.07), read 2026-10-12
   • origin/main docs/cloud-sessions.md:230-262 (0.07 documented as a stopgap, 177,000-char installed listing, 150,000-char capture at 0.05 on claude-opus-5-5 1M, perf(skills): trim skill descriptions until the listing fits, then lower skillListingBudgetFraction #6846 named as the fix owner)
   • gh issue view 6458: kyle-sexton 2026-10-05T19:06Z (keep 0.01, remove overrides 0.05 here and 0.03 in medley, fix through descriptions, plus a way for new skills to get picked up) and 2026-10-05T19:36Z (direction change: default to model-invocable, cut cost through shorter descriptions and plugin profiles)
   • gh pr view 6805: author and mergedBy kyle-sexton, mergedAt 2026-10-10T23:09:10Z, body cites only chore(config): raise skillListingBudgetFraction to 0.05 so the enabled fleet fits #3767 and calls itself a stopgap; no mention of feat(context-budget): keep plugin skill descriptions visible when the skill listing overflows #6458
   • gh issue view 6846: filed 2026-10-10T21:48Z under kyle-sexton, criteria 2 and 3 require a lowered override and 0 shed; comments 2026-10-11T06:29Z (worker question, three options, trims need model-graded evals, worktree refuses claude plugin eval, 146,814 chars across 280 skills at 8fa9df0) and 2026-10-11T16:19Z (handoff, NEEDS-HUMAN); no owner answer
   • gh issue view 6476: brief locked through /planning:interview, confirmed 2026-10-07; description standard (third-person sentence plus Use when, one trigger per situation), measurement before and after each rewrite (measure-invocation probes, claude plugin eval cases), fleet rollout one PR per plugin, every fleet checkbox still unchecked; lists feat(context-budget): keep plugin skill descriptions visible when the skill listing overflows #6458 as separate complementary work

   Next: Read-only on GitHub in this pass; the owner (or a session the owner starts) does the following. (1) Post the fraction decision on #6458 and #6846. (2) Re-scope #6846: title and body become 're-measure the listing after the #6476 rollout and set skillListingBudgetFraction', keep criteria 2 to 4 under Option A or replace them with 'remove the override, retire listing_budget_configured in pr-require-checks.yml, rewrite docs/cloud-sessions.md' under Option B, keep the user-scope and medley reassessment, and add 'Refs #6476' with a note that criterion 1 (the trims) is delivered by that rollout. (3) In #6476, order the fleet rollout to start with the plugins owning the longest descriptions (testing, mutation-testing, visualization, bugs, evals, review, writing, playbooks, coupling), one draft PR per plugin, each run attended with measure-invocation probes and claude plugin eval before and after, per the #6476 measurement section. (4) Leave .claude/settings.json at 0.07 until the post-rollout re-measure; then one PR to .claude/settings.json and docs/cloud-sessions.md (and the CI step under Option B), verified by a fresh-session listing capture through /harness-ops:audit-skill-visibility --installed --context-window 1000000 --listing-capture. (5) Where check-listing-budget.sh is used for the fit verdict, pass the 3-bytes-per-token budget explicitly (CHECK_SKILL_LISTING_BUDGET_CHARS), since the script's 4 chars/token default overstates headroom for current models.

   Researched by two independent agents (local evidence and upstream docs) and adjudicated by a third.
8. kyle-sexton

   added the

   Task

   issue type on Oct 15, 2026
9. kyle-sexton

   added

   status: needs-decisionAwaiting a human or maintainer judgment call.Awaiting a human or maintainer judgment call.

   and removed

   needs-triageNot yet classified. Floor until a type and one priority tier are set.Not yet classified. Floor until a type and one priority tier are set.

   status: needs-decisionAwaiting a human or maintainer judgment call.Awaiting a human or maintainer judgment call.

   needs-humanHuman-in-the-loop required; autonomous sessions must not resolve items carrying this.Human-in-the-loop required; autonomous sessions must not resolve items carrying this.

   on Oct 15, 2026
10. kyle-sexton commented on Oct 24, 2026

    

    kyle-sexton

    on Oct 24, 2026

    MemberAuthor

    More actions

    Released by the owner's HITL session (2026-10-24, C12): it waited only on a removed gate. Intake triage re-runs on it to set its fields, then admission takes it.
11. kyle-sexton

    marked this as blocked on Oct 24, 2026

    • Concise skills: one skill-creation standard, evals, and a fleet-wide rollout #6476
12. kyle-sexton commented on Oct 24, 2026

    

    kyle-sexton

    on Oct 24, 2026

    MemberAuthor

    More actions

    Needs a run of claude (eval, probe or /usage), which the issue lane cannot do (pr-pipeline.yaml:289-314). The owner's HITL session runs it on the operator machine; no owner action needed.
13. github-actions commented on Oct 25, 2026

    

    github-actionsBot

    on Oct 25, 2026 – with GitHub Actions

    Contributor

    More actions

    The triage sweep dispatched intake triage for this issue (attempt 1 of 2).
14. melodic-automation-lanes

    added

    status: readyTriaged, unblocked, and fully specified; eligible to pick up.Triaged, unblocked, and fully specified; eligible to pick up.

    work-class: scopedA briefed fix or small feature; blast radius bounded by the brief, tests exist.A briefed fix or small feature; blast radius bounded by the brief, tests exist.

    on Oct 25, 2026
15. melodic-automation-lanes commented on Oct 25, 2026

    

    melodic-automation-lanesBot

    on Oct 25, 2026

    Contributor

    More actions

    Not admitted to the issue lane yet: blocker #6476 is not closed by a listed human or a merge.