test(developer-experience): add shared eval fixtures for the .NET and Python repositories #9943

Activity

1. kyle-sexton

   added the

   Task

   issue type on Oct 25, 2026
2. kyle-sexton

   added

   status: readyTriaged, unblocked, and fully specified; eligible to pick up.Triaged, unblocked, and fully specified; eligible to pick up.

   admission-deferredReady for the issue lane; work-unblock admits it when queued work is under the queued capReady for the issue lane; work-unblock admits it when queued work is under the queued cap

   on Oct 25, 2026
3. kyle-sexton

   added a parent issue on Oct 25, 2026

   • feat(developer-experience): claude plugin eval suite in GitHub Actions #6649
4. kyle-sexton

   marked this as blocked on Oct 25, 2026

   • chore(developer-experience): commit the hand-rolled eval scenario sources #9936
5. github-actions

   added

   needs-triageNot yet classified. Floor until a type and one priority tier are set.Not yet classified. Floor until a type and one priority tier are set.

   on Oct 25, 2026
6. github-actions commented on Oct 25, 2026

   

   github-actionsBot

   on Oct 25, 2026 – with GitHub Actions

   Last edited by melodic-automation-lanes

   Contributor

   More actions

   Proposed work class: work-class: scoped
   Basis: automated intake triage; the lanes App applied this class.

   Size: Medium
   Complexity: Medium
   Uncertainty: Known
   Flags: none
   Blocked by: #9936

   Understood: add two invented fixture repositories (tarnquill, .NET; brackenfold, Python) in four states each (bare, conventions-only, converged, legacy). They go under plugins/developer-experience/evals/fixtures/, each built by a scaffold or seed script. Done when a case naming each state loads under /evals:validate and the changelog-fragment check passes.

   The scope and approach are clear. The issue depends on #9936 (committing the eval scenario sources), which is still open, so it waits on that. I found no likely duplicates.

   _{Automated triage, applied by the lanes App.}
7. github-actions

   added

   needs-humanHuman-in-the-loop required; autonomous sessions must not resolve items carrying this.Human-in-the-loop required; autonomous sessions must not resolve items carrying this.

   on Oct 25, 2026
8. melodic-automation-lanes commented on Oct 25, 2026

   

   melodic-automation-lanesBot

   on Oct 25, 2026

   Last edited by melodic-automation-lanes

   Contributor

   More actions

   The latest triage verdict holds nothing: the issue is at status: ready.
9. melodic-automation-lanes

   added

   status: needs-decisionAwaiting a human or maintainer judgment call.Awaiting a human or maintainer judgment call.

   and removed

   status: readyTriaged, unblocked, and fully specified; eligible to pick up.Triaged, unblocked, and fully specified; eligible to pick up.

   needs-triageNot yet classified. Floor until a type and one priority tier are set.Not yet classified. Floor until a type and one priority tier are set.

   on Oct 25, 2026
10. kyle-sexton

    marked this as blocking 7 issues on Oct 25, 2026

    • test(developer-experience): port eval cases S1 and S4 #9953
    • test(developer-experience): port eval cases S2 and S5 #9958
    • test(developer-experience): port eval cases S3 and S6 #9966
    • test(developer-experience): port eval cases S7 and N1 #9979
    • test(developer-experience): port eval cases T1 and T2 #9988
    • test(developer-experience): port eval cases T3 and T4 #9993
    • test(developer-experience): port the stays-quiet eval case #9998
11. kyle-sexton

    added

    needs-triageNot yet classified. Floor until a type and one priority tier are set.Not yet classified. Floor until a type and one priority tier are set.

    and removed

    status: needs-decisionAwaiting a human or maintainer judgment call.Awaiting a human or maintainer judgment call.

    needs-humanHuman-in-the-loop required; autonomous sessions must not resolve items carrying this.Human-in-the-loop required; autonomous sessions must not resolve items carrying this.

    on Oct 25, 2026
12. kyle-sexton commented on Oct 25, 2026

    

    kyle-sexton

    on Oct 25, 2026

    MemberAuthor

    More actions

    Released for re-triage under the owner's 2026-10-24 rule: an issue stops for a person only for an intent or vision question, a security call, a host or attended run, or spend outside prepaid credit. The hold label came off and intake triage runs again now; it re-applies a hold, with a numbered question, when one of those applies.

    To hold this issue again, add needs-human (or status: needs-decision) and say why in a comment.
13. github-actions

    added

    priority: mediumReal value, no hard deadline; normal backlog flow.Real value, no hard deadline; normal backlog flow.

    and removed

    needs-triageNot yet classified. Floor until a type and one priority tier are set.Not yet classified. Floor until a type and one priority tier are set.

    on Oct 25, 2026
14. melodic-automation-lanes

    added

    status: readyTriaged, unblocked, and fully specified; eligible to pick up.Triaged, unblocked, and fully specified; eligible to pick up.

    work-class: scopedA briefed fix or small feature; blast radius bounded by the brief, tests exist.A briefed fix or small feature; blast radius bounded by the brief, tests exist.

    and removed

    admission-deferredReady for the issue lane; work-unblock admits it when queued work is under the queued capReady for the issue lane; work-unblock admits it when queued work is under the queued cap

    on Oct 25, 2026
15. melodic-automation-lanes commented on Oct 25, 2026

    

    melodic-automation-lanesBot

    on Oct 25, 2026

    Contributor

    More actions

    Not admitted to the issue lane yet: blocker #9936 is not closed by a listed human or a merge.
16. melodic-automation-lanes

    marked this as blocking on Oct 26, 2026

    • feat(developer-experience): claude plugin eval suite in GitHub Actions #6649