test(developer-experience): port eval cases S1 and S4 #9992

Activity

1. kyle-sexton

   added the

   Task

   issue type on Oct 25, 2026
2. github-actions

   added

   needs-triageNot yet classified. Floor until a type and one priority tier are set.Not yet classified. Floor until a type and one priority tier are set.

   on Oct 25, 2026
3. github-actions commented on Oct 25, 2026

   

   github-actionsBot

   on Oct 25, 2026 – with GitHub Actions

   Last edited by melodic-automation-lanes

   Contributor

   More actions

   Proposed work class: work-class: scoped
   Basis: automated intake triage; the lanes App applied this class.

   Size: Medium
   Complexity: Medium
   Uncertainty: Known
   Flags: none
   Blocked by: none

   Understood: port eval scenarios S1 and S4 into plugins/developer-experience/evals/ as case directories (prompt, case.yaml, graders), keeping the S4 #10 withhold check and the rm and compound-shell checks where needed. The done-when items are clear and checkable. No likely duplicates found among the open issues.

   _{Automated triage, applied by the lanes App.}
4. github-actions

   added

   priority: mediumReal value, no hard deadline; normal backlog flow.Real value, no hard deadline; normal backlog flow.

   and removed

   needs-triageNot yet classified. Floor until a type and one priority tier are set.Not yet classified. Floor until a type and one priority tier are set.

   on Oct 25, 2026
5. melodic-automation-lanes

   added

   status: readyTriaged, unblocked, and fully specified; eligible to pick up.Triaged, unblocked, and fully specified; eligible to pick up.

   work-class: scopedA briefed fix or small feature; blast radius bounded by the brief, tests exist.A briefed fix or small feature; blast radius bounded by the brief, tests exist.

   admission-deferredReady for the issue lane; work-unblock admits it when queued work is under the queued capReady for the issue lane; work-unblock admits it when queued work is under the queued cap

   on Oct 25, 2026
6. kyle-sexton

   added a parent issue on Oct 25, 2026

   • feat(developer-experience): claude plugin eval suite in GitHub Actions #6649
7. kyle-sexton

   marked this as blocked on Oct 25, 2026

   • test(developer-experience): add shared eval fixtures for the .NET and Python repositories #9984
8. melodic-automation-lanes

   added

   agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.Fully specified and briefed; eligible for autonomous pickup from the frontier.

   and removed

   admission-deferredReady for the issue lane; work-unblock admits it when queued work is under the queued capReady for the issue lane; work-unblock admits it when queued work is under the queued cap

   on Oct 25, 2026
9. kyle-sexton commented on Oct 25, 2026

   

   kyle-sexton

   on Oct 25, 2026

   MemberAuthor

   More actions

   Duplicate split of #6649 by a second Backlog HITL writer. To reopen: reopen this issue and say what the kept child misses.

   Kept child: #9953.
10. kyle-sexton

    closed this as not plannedon Oct 25, 2026