Repository navigation
test(source-control): serve the babysit cycle's gh commands from the gh-forge stand-in #7996
Copy link
Copy link
Closed
Task
Copy link
Labels
agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.Fully specified and briefed; eligible for autonomous pickup from the frontier.status: readyTriaged, unblocked, and fully specified; eligible to pick up.Triaged, unblocked, and fully specified; eligible to pick up.work-class: scopedA briefed fix or small feature; blast radius bounded by the brief, tests exist.A briefed fix or small feature; blast radius bounded by the brief, tests exist.
Description
Refs: #7148
Outcome
The offline gh stand-in answers the five gh commands the owner named on #7148 (gh pr merge, gh api graphql for review threads, gh api user, gh search prs, gh pr review), so babysit eval cases that use them never reach real GitHub.
Current behavior
plugins/source-control/evals/fixtures/gh-forge/gh serves one pull request for pr view, pr list, pr diff, pr ready, pr checks, repo view, auth status and one api --hostname viewer read. Every other command exits 1 and names itself.
Desired behavior
Per the owner decision on #7148 (2026-10-13), the stand-in also serves:
gh pr merge, which appends a record of the call (arguments and PR number) to a file in the forge state that a grader can read. It changes no real ref outside the fixture.gh api graphqlfor review threads, answered from fixture state thatinstall.shwrites. A resolve-thread mutation is recorded the same way as a merge.gh api user,gh search prsandgh pr review.
Every other command is still refused with exit 1, and the stand-in still reaches no network.
Scope fence
plugins/source-control/evals/fixtures/gh-forge/ghplugins/source-control/evals/fixtures/gh-forge/install.shplugins/source-control/evals/gh-forge.test.sh.changes/source-control/<slug>.md
Acceptance criteria
-
gh-forge.test.shhas one case for each new command (pr merge,api graphqlreview threads,api user,search prs,pr review). -
gh-forge.test.shhas a case showing an unlisted command still exits 1. - A
gh pr mergetest case asserts that the call record exists in the forge state and that no ref outside the fixture changed. - The PR's CI runs
plugins/source-control/evals/gh-forge.test.shand it passes. - A source-control changelog fragment exists and
scripts/check-changelog-fragments.sh --checkpasses.
Out of scope
- Eval cases (test(source-control): add a babysit-prs eval case that the safe tier never resolves or merges #7997, test(source-control): add a setup eval case that writes only what the user confirmed #7998).
- The paid with/without runs (test(source-control): add eval cases for babysit-prs, babysit-loop and setup #7148).
- Existing cases that already use the stand-in, except to keep them passing.
- Other gh commands babysit-prs calls:
gh repo list,gh api repos/...,gh api --paginate,gh auth switchandgh run download. They stay refused with exit 1.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.Fully specified and briefed; eligible for autonomous pickup from the frontier.status: readyTriaged, unblocked, and fully specified; eligible to pick up.Triaged, unblocked, and fully specified; eligible to pick up.work-class: scopedA briefed fix or small feature; blast radius bounded by the brief, tests exist.A briefed fix or small feature; blast radius bounded by the brief, tests exist.
Type
Fields
No fields configured for Task.
Activity
chore(source-control): re-run pr-verify after closing #7996