Fix/followup creator app token - #3446
Open
iamkayleb wants to merge 32 commits into
Open
Conversation
… this fork bukay was built against an older Workflows version and had already been wired to iamkayleb/Workflows by hand (38 refs, zero to stranske). Syncing the upstream-owned templates into it would have reverted every one of those refs on the first delivery, so the repoint has to land before the registration does. Consumer templates (33 functional refs, 19 files) Repoint `uses:`, `repository:` sparse-checkouts, `runner_ref:`, and the Renovate fleet preset from stranske/Workflows to iamkayleb/Workflows. The three upstream action/workflow SHAs do not exist in this fork, so they are re-pinned to e85edad, which carries all three paths. Prose and comments are untouched, which keeps the three `template_sync: exact` files byte-identical with their root copies. Registration - REGISTERED_CONSUMER_REPOS += iamkayleb/bukay (15 consumers) - pr-00-gate.yml overwrite_repos += iamkayleb/bukay. bukay's Gate carries no generated-delivery-seal job, and a create_only Gate would never receive one, so Maint 71 would never merge its delivery. Its Node CI lives in ci.yml, which stays create_only and repo-owned. - bukay added to the canary set, replacing trip-planner (the validator accepts exactly 2-3 registered repos). A non-canary needs promote plus canary evidence, which this fork cannot produce against stranske repos. Coupled inventories langsmith_fleet.MANAGED_CONSUMER_REPOS, the observability allowlist (not-applicable until bukay produces a conformant artifact), and the generated Renovate ownership preset. Health 74 Eight template fingerprints re-baselined. The checker canonicalizes the pin ref but preserves the action path, so an owner change is real drift. Only the stale fingerprints were rewritten; every hand-written "do not align" rationale is preserved. Tests Count and membership assertions updated where the fleet inventory grew. Three assertions that covered root and template together were made owner-agnostic rather than repointed, since which reusable is called is the contract, not who hosts it. The renovate-ownership test now derives its opt-out set from the manifest instead of a hardcoded repo list, and test_workflow_llm_installs discovery matches `*/Workflows` — it had silently dropped three template workflows. 4375 passed, no new failures. The 9 remaining failures are pre-existing in this sandbox (Python 3.11 vs the repo's >=3.12 pin, and missing langchain deps); verified identical on a clean tree. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Df2tQ9HcgAW8M3VNrFT3gT
A reusable runs in the CALLER's context. Repointing the consumer templates at
iamkayleb/Workflows got consumers the fork's reusable *logic*, but each reusable
then ran `actions/checkout` against `repository: stranske/Workflows` to vendor its
helper scripts — so bukay would have run fork reusables over upstream scripts, and
a script change in this fork would never have reached it.
Helper layer (27 checkouts, 13 reusables)
reusable-10-ci-python (5), reusable-20-pr-meta (5), reusable-18-autofix (3),
reusable-agents-pr-health (3), reusable-16-agents (2),
reusable-model-profile-trial (2), and one each in
reusable-19-dependency-repair-contract, reusable-agents-verifier,
reusable-backplane-conformance, reusable-{claude,codex,cursor,gemini}-run.
22 of the 27 are reachable from the reusables bukay calls.
The upstream @main-only rationale in these files ("callers ride @main, hence
main IS the pinned ref") holds when the control plane and the helper source are
the same repo. In a fork they are not.
Owner comparisons that were silently dead
reusable-18-autofix derived the owner from github.workflow_ref and compared it
to a literal `stranske/Workflows`. In this fork that branch never fired, so a
SHA-pinned self-test silently degraded to upstream main. Now matches the
repository name. Same fix in two test discovery predicates that had quietly
stopped inspecting anything -- test_reusable_ci_no_hardcoded_ref_main had
fallen to zero inspected checkouts and only its `checked >= 4` floor revealed
it; test_reusable_run_shared_base asserted the literal owner string.
Model-profile-trial is one unit
Its caller `uses:`, its `runner_sha` input, the registry `runner_ref`, and the
runner's own immutable-identity assertion (plus its remote-main freshness
probe) must all name the same repo and commit, or the identity check fails
closed against a fork checkout. Repointed together to e85edad, which carries
the runner. Its assertions stay exact rather than owner-agnostic: for a
read-only pinned trial arm, the owner is part of the guarantee.
Not changed
The 14 refs across 8 root agent workflows that call upstream reusables and
actions. That is a fork-self-CI concern and does not reach consumers; holding
it until the first bukay delivery lands. Documented in
docs/ops/CONSUMER_REPO_MAINTENANCE.md along with the corrected claim that
"root workflows were not repointed", which this commit makes partly false.
4371 passed, no new failures. Template drift 0 unallowlisted, actionlint clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Df2tQ9HcgAW8M3VNrFT3gT
Claude/repo changes review jgvimg
Agent bootstrap PRs always targeted the repository default branch: the issue bridge hardcoded `base = data.default_branch` with no override. That makes per-agent evaluation lanes (eval/claude, eval/codex) impossible — every agent PR lands on main. - reusable-agents-issue-bridge: resolve the PR base as base_branch input > `<!-- base-branch: X -->` issue marker > repo default, with an existence guard that falls back to the default when a marker names a missing branch (a typo can no longer wedge dispatch). The input is passed via env, not inline interpolation. - agents-verify-to-new-pr: carry a non-default PR base into the generated follow-up issue as a base-branch marker (both the LLM and fallback paths), so remediation PRs stay in the lane instead of escaping to main. Resolution order verified against all four cases; workflow YAML validator passes. Two pre-existing long-line lint failures in the bridge are upstream and untouched.
Support non-default base branches for agent PRs (evaluation lanes)
Cursor was integrated against the previous loop architecture: the runner (reusable-cursor-run.yml) and registry entry exist, and agents-autofix-loop had an autofix-cursor job. But both agents-keepalive-loop.yml and agents-autofix-loop.yml were retired in favour of agents-81-gate-followups, and the consolidation did not carry Cursor across — agents-81 dispatched only run-codex and run-claude. Cursor therefore had a runner that nothing called. - agents-81 template: add run-cursor (keepalive) and autofix-cursor, mirroring the Claude jobs and calling reusable-cursor-run.yml. - Add both to the needs lists of record-keepalive-completion, summary and metrics so completion accounting and reporting wait on them. - registry: capabilities.belt true (the loop now dispatches cursor) and raise the capacity placeholder off 1/day, flagged for the owner's real plan quota. Routing verified (agent:cursor -> reusable-cursor-run.yml, cursor/issue-), dependency graph checked, registry tests pass, templates in sync. The one pre-existing long-line lint failure in agents-81 is upstream and untouched.
…ed-loop /Wire Cursor into the consolidated keepalive/autofix loop
Both were hardcoded two-agent enumerations that the registry never fed, so adding Cursor to the registry was not enough to make its lane behave like the other two. - agents-capability-check gated on ["agent:codex","agent:claude","agent:auto"], so Cursor issues skipped the pre-flight capability gate entirely. Claude and Codex issues were screened for impossible tasks; Cursor's were not, which is an unequal starting condition for a three-way comparison. - agents-auto-label's skip-list omitted agent:cursor, so an issue already routed to Cursor was not recognised as already-routed and could be relabelled to a different agent. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FC8XoyssN5v6hQCTtcjoB5
…ed-loop Close two agent allowlists that silently excluded Cursor
The canary registry still carried stranske/Travel-Plan-Permission and stranske/Portable-Alpha-Extension-Model from upstream. This fork has no write access to either, so every consumer sync run fails on both and the failure issue it opens is noise that hides real sync problems. Leaves iamkayleb/bukay as the sole canary. Restore the upstream entries only if this fork is ever given write access to them. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FC8XoyssN5v6hQCTtcjoB5
Limit the canary list to repos this fork can write to
The guard counted every workflow run whose run-name ended with the issue number, with no filter on whether the run did anything. This workflow triggers on every `issues: labeled` event, but only agents:format, agents:optimize, agents:apply-suggestions and workflow_dispatch get past the trigger check — every other label spawns a run that exits immediately and still consumed one of the four slots. Applying three labels to a new issue therefore burned the whole budget before any optimizer work started, and the guard tripped on legitimate bulk seeding: "Too many optimizer runs (5 > 3) in last hour" on issues that had run the optimizer at most twice. run-name now marks a run [work] or [noop] from the triggering label, which is known at trigger time, and the guard excludes [noop]. Genuine recursion is still caught: four real runs in the window trips as before. Verified against the observed failure — the same history counts 5 (trips) under the old expression and 2 (passes) under the new one, while four work runs still trip. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FC8XoyssN5v6hQCTtcjoB5
…ounts-noops Optimizer recursion guard: stop counting no-op runs
Trimming the registry to iamkayleb/bukay alone broke every sync run: select_consumer_sync_phase rejects a config outside 2..3 canaries with canary_config_requires_two_or_three_repos, so Maint 68 failed in its first job before reaching any repository. The trim was aimed at two consistently red matrix legs for repositories this fork cannot write to. Those legs were noise, not the blocker — the run that prompted the change failed on all three legs, including bukay, for an unrelated missing read:org PAT scope. Removing them bought nothing and cost the whole sync. Relaxing the lower bound was considered and rejected: the checked-in config is also asserted to cover six distinct consumer capability shapes, which a single-consumer fork cannot satisfy without inventing capabilities. The two unowned legs stay red until this fork either gains access to them or registers a second consumer it owns. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FC8XoyssN5v6hQCTtcjoB5
Restore the full canary registry
agents-auto-pilot set no run-name, so every run appeared in `gh run list` as the bare workflow name. With no issue number in displayTitle there is no way to ask "what has auto-pilot done for issue stranske#241" — a hundred runs are indistinguishable, and per-issue history cannot be reconstructed even though auto-pilot is the workflow that owns the whole issue pipeline. agents-issue-optimizer already does this, and its recursion guard depends on it. Adopt the same shape here, covering the issue, pull_request and workflow_dispatch triggers. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FC8XoyssN5v6hQCTtcjoB5
The consumer template still carried `group: codex-belt-dispatcher`, a single fixed lane for the whole repository, while the Workflows-local copy had already moved to a per-agent group. Consumers therefore serialised every belt dispatch. Dispatching several issues at once puts them all in that one lane. GitHub keeps one running plus one waiting and cancels the rest while they are still pending, so they surface as failed runs with no jobs and no log — which reads as the dispatcher being broken rather than as contention. Six evaluation issues produced one running, one waiting and four cancelled, and every agent sat in "Waiting for agent to create branch" backoff indefinitely because no branch was ever created. Key the group by agent and forced issue. Auto-select runs (no force_issue) still share one lane per agent, so the "pick the next issue" path cannot race itself into double-assignment; explicitly forced per-issue dispatches now run in parallel. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FC8XoyssN5v6hQCTtcjoB5
Put the issue number in the auto-pilot run title
…oncurrency Belt dispatcher: give each dispatched issue its own concurrency lane
reusable-agents-issue-bridge.yml learned to resolve a base branch from an explicit input, then the issue's `<!-- base-branch: X -->` marker, then the repository default. agents-71-codex-belt-dispatcher kept `const base = repoInfo.default_branch` and was never updated. That is the path auto-pilot actually dispatches, so in practice the marker was ignored: an evaluation issue carrying `<!-- base-branch: eval/cursor -->` produced "Selected issue stranske#246 -> branch cursor/issue-246 on base main", and every agent branch was cut from the default branch regardless of the lane it was meant for. Adds the same resolution order and existence guard, plus a `base_branch` input so callers can override explicitly, and logs the chosen base and its source. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FC8XoyssN5v6hQCTtcjoB5
…e-marker Belt dispatcher: honour the base-branch marker
health-codex-auth-check runs twice daily, decodes CODEX_AUTH_JSON, and opens an `auth-expiring` issue before the token lapses. It existed only in this repository, so consumers — where the agents actually run and where the secret actually matters — had no auth monitoring at all. That gap has a cost on record: a consumer's CODEX_AUTH_JSON expired and every dispatched agent produced no commits, with no error naming the cause; the same failure is documented upstream as having gone unnoticed for 34 days. The workflow is already portable. It sparse-checks-out error_classifier.js, github-api-with-retry.js and token_load_balancer.js and uses the local setup-api-client action — all four are already present in consumer repos via this manifest — and needs only CODEX_AUTH_JSON, which consumers already set for the agent runners. Copied verbatim; no changes required. Detection only. Refreshing the token needs `codex login --device-auth` at a browser, so the most any automation can do is warn early. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FC8XoyssN5v6hQCTtcjoB5
Ship the Codex auth expiry check to consumer repos
The issue bridge and the belt dispatcher both resolve a pull request base from the issue's `<!-- base-branch: X -->` marker. Auto-pilot did not: it took `repoInfo.default_branch` at both of its PR sites and never read the marker at all. Auto-pilot is the component that actually calls pulls.create, so the marker was honoured when the branch was cut and then ignored when the PR was opened. An issue marked `eval/cursor` produced branch `cursor/issue-285` and a pull request into `main` — the per-lane separation collapsed silently, and merging any such PR would put one lane's work on the shared branch. Both sites now resolve marker over default, verify the branch exists before using it, fall back with a warning when it does not, and log the chosen base and its source. This is the third and last place that computed a PR base: the bridge, the belt dispatcher and auto-pilot are now consistent, and no `const baseBranch = repoInfo.default_branch` remains in any of them. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FC8XoyssN5v6hQCTtcjoB5
Auto-pilot: honour the base-branch marker when opening the pull request
Codex defaults to `workspace-write`, which makes it sandbox with bubblewrap.
bwrap cannot create a loopback interface on a GitHub-hosted runner:
bwrap: loopback: Failed RTM_NEWADDR: Operation not permitted
Every shell command then fails before it runs. Codex reported this exactly —
"the workspace shell fails before any command runs ... I therefore can't
inspect the existing availability API, run tests, or create the required
verified commit safely" — and its session recorded 0 commands executed and 0
files changed.
From outside that reads as the agent producing nothing: the job is green, the
ledger closes the task, and the only commit is a telemetry timestamp. Two
evaluation rounds were scored against Codex on this basis before the session
transcript was read.
The runner is an ephemeral single-tenant VM and is already the isolation
boundary, so Codex's inner sandbox buys nothing here. Pass
`danger-full-access` on both Codex paths in the consolidated loop, matching
what the runner already documents for CI use.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC8XoyssN5v6hQCTtcjoB5
…ed-loop Give Codex a usable sandbox in the consolidated loop
agents-verify-to-new-pr chose OWNER_PR_PAT, then SERVICE_BOT_PAT, then GITHUB_TOKEN. It never tried a GitHub App installation token, unlike the rest of the fleet, so a consumer with neither PAT set created every follow-up issue with GITHUB_TOKEN. Issues and labels created that way raise no events for other workflows (GitHub's loop guard), so the follow-up issue was created correctly, carried the right labels, and then sat there: auto-pilot never saw it and a human had to remove and re-apply `agents:auto-pilot` by hand to start the work. The automatic remediation loop silently required a manual step on every iteration. Mint an App token when WORKFLOWS_APP_ID and WORKFLOWS_APP_PRIVATE_KEY are present and prefer it, keeping the PAT chain as fallback. When nothing better than GITHUB_TOKEN is available the step now warns that the issue will not trigger auto-pilot, instead of failing silently. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FC8XoyssN5v6hQCTtcjoB5
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Advanced Run ID: Comment |
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
There was a problem hiding this comment.
zizmor found more than 20 potential problems in the proposed changes. Check the Files changed tab for more details.
| needs.evaluate.outputs.action == 'conflict') | ||
| uses: stranske/Workflows/.github/workflows/reusable-claude-run.yml@main | ||
| uses: iamkayleb/Workflows/.github/workflows/reusable-claude-run.yml@main | ||
| secrets: inherit |
| steps.check-merged.outputs.merged == 'true' && | ||
| env.WORKFLOWS_APP_ID != '' && env.WORKFLOWS_APP_PRIVATE_KEY != '' | ||
| continue-on-error: true | ||
| uses: actions/create-github-app-token@v3 |
| steps.check-merged.outputs.merged == 'true' && | ||
| env.WORKFLOWS_APP_ID != '' && env.WORKFLOWS_APP_PRIVATE_KEY != '' | ||
| continue-on-error: true | ||
| uses: actions/create-github-app-token@v3 |
| runs-on: ubuntu-latest | ||
| steps: | ||
| - name: Checkout retry helpers | ||
| uses: actions/checkout@v7 |
|
|
||
| - name: Check for existing open issue | ||
| id: existing | ||
| uses: actions/github-script@v9 |
| steps.existing.outputs.result != 'true' && | ||
| (steps.check.outputs.status == 'expiring-soon' || | ||
| steps.check.outputs.status == 'expired') | ||
| uses: actions/github-script@v9 |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c2d357868f
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
Comment on lines
+759
to
762
| - run-cursor | ||
| if: >- | ||
| always() && | ||
| needs.evaluate.outputs.dispatch_should_run == 'true' && |
There was a problem hiding this comment.
When agent_type == 'cursor', run-cursor is the only active runner, but this condition still checks only Codex and Claude, so the completion job is skipped and the dispatch remains pending until the 30-minute stale timeout. The summary section likewise aggregates only Codex/Claude results and outputs, causing a completed Cursor run to be reported as skipped and omitting its commits and task progress; include Cursor everywhere runner results are selected.
AGENTS.md reference: AGENTS.md:L54-L54
Useful? React with 👍 / 👎.
Comment on lines
+362
to
+366
| } catch (error) { | ||
| core.warning( | ||
| `Base branch '${base}' (from ${baseSource}) not found; ` + | ||
| `falling back to '${defaultBranch}'.`, | ||
| ); |
There was a problem hiding this comment.
If the non-default branch lookup exhausts retries because of a 403, rate limit, or transient API failure, this catch treats that failure as proof the branch is absent and silently falls back to the default branch. An evaluation-lane issue can consequently produce a PR against the default branch—the exact escape this feature is intended to prevent. Fall back only for a confirmed 404 and fail the dispatch for other errors; the same broad catch is duplicated in the dispatcher and auto-pilot paths.
Useful? React with 👍 / 👎.
Comment on lines
+321
to
+322
| let base = String(process.env.INPUT_BASE_BRANCH || '').trim(); | ||
| let baseSource = base ? 'input' : ''; |
There was a problem hiding this comment.
For either workflow_call or workflow_dispatch with base_branch set, INPUT_BASE_BRANCH is never defined in the Resolve candidate issue step's environment—the step exports only FORCE_ISSUE and AGENT_KEY. Therefore this expression is always empty and the advertised explicit override is ignored in favor of the issue marker or default branch; pass ${{ inputs.base_branch }} through the step environment as the reusable issue bridge does.
Useful? React with 👍 / 👎.
| - prepare | ||
| - autofix | ||
| - autofix-claude | ||
| - autofix-cursor |
There was a problem hiding this comment.
Adding autofix-cursor to needs does not add its result to the metrics aggregation below, which still selects only needs.autofix.result or needs.autofix-claude.result. On every Cursor-routed autofix, both of those jobs are skipped, so the artifact records autofix_result: skipped regardless of whether Cursor succeeded or failed, corrupting provider outcome metrics; include needs.autofix-cursor.result in the selection.
AGENTS.md reference: AGENTS.md:L54-L54
Useful? React with 👍 / 👎.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Workflow Source
Started from:
Automation intent:
Notes:
Summary
Testing