Repository navigation
Add OrcaReplay to coding (record and replay provider exchanges) - #64
Conversation
autogame-17
left a comment
There was a problem hiding this comment.
Reviewed current head f0808deb762d2f8bf4cc0eeaffbf5b9adae4d797, including the canonical repository, license and replay implementation. The coding-tool scope is appropriate; please tighten one factual claim before inclusion.
data/projects.json:1543 and generated README.md:156: “replay can fork from any step” should say that it forks from available derived checkpoints. At pinned upstream 6af80889:
deriveCheckpointsrequires both a complete conversation prefix and a filesystem snapshot.- The fork command rejects runs without checkpoints and snaps a non-checkpoint
--fromsequence back to the nearest prior checkpoint, emittingfork.snapped.
A reader comparing models from a particular event can therefore start from an earlier state, or find that the run cannot fork. “Replay can fork from available checkpoints onto a different model” preserves the useful capability without promising arbitrary-event boundaries. Please update the JSON description, regenerate README, and qualify the equivalent “any step” wording in the PR body.
Actual Node 24.19.0 checks on this head: 120 projects / 10 categories validated, two generator passes were byte-identical to the committed README, the reviewed diff passed whitespace checking, and the initial full link pass was 120 ok / 0 failures / 0 warnings. A later full check of current upstream main found 118 ok / 1 failure / 0 warnings due to HTTP 404 on existing Fzkuji/OpenProgram; that separate catalog availability issue also prevents using the earlier green result as a current all-links-clear result. OrcaReplay's URL passed, its Apache-2.0 license was present, and the normal weekly star snapshot drift is not a requested change.
After updating the description:
node scripts/generate-readme.js
node scripts/validate.js
git diff --check
node scripts/check-links.jsBoth Actions runs on this PR are currently action_required with 0 jobs (validate, links); Cursor Bugbot success is not a replacement for these checks. Maintainer workflow approval and successful execution are still required. No merge has been performed.
Review was right. deriveCheckpoints requires both a complete conversation prefix and a filesystem snapshot, and `replay --fork` rejects a run with no checkpoints and snaps a non-checkpoint --from back to the nearest prior one (fork.snapped). The entry now says it forks from available checkpoints. Also dropped "offline" from the same sentence -- replay withholds the model provider, it does not take the run off the network -- and said so explicitly. Regenerated README; validate, diff --check and check-links all clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
f0808de to
2eca16c
Compare
Bugbot is paused — on-demand spend limit reachedBugbot uses usage-based billing for this team and has hit its on-demand spend limit. A team admin can raise the spend limit in the Cursor dashboard, or wait for the next billing cycle to continue. |
|
Thank you for reading the implementation rather than the description — the finding is correct and I have changed it.
Two things changed, not one:
I also qualified the equivalent wording in the PR body, including the Checks on this head, Node 22.23.2:
The branch is rebuilt on current Disclosure: I maintain OrcaReplay, so this is a self-submission. |
|
Rechecked the revised head Actual local verification using the current trusted main scripts on this head's data and README, with Node 24:
The remaining blocker is CI execution, not the old wording: Validate curated data and Check Links are both No further description edit is requested by this follow-up. Keeping the PR open/blocked; no approval, merge, or workflow execution was issued by this maintenance run. |
Refresh the canonical star snapshot while preserving the reviewed replay description. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Resolve against current main and regenerate the catalog README. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
autogame-17
left a comment
There was a problem hiding this comment.
Verified the refreshed catalog-only diff against current main. The previously corrected available-checkpoint boundary is retained without additional wording changes; the title now accurately describes provider-exchange replay. Generated README, validation, and link checks passed locally and in the two current GitHub Actions jobs.
Adds OrcaReplay to
coding, following the four steps inCONTRIBUTING.md.Disclosure: I maintain OrcaReplay. Apache-2.0, Node 20+.
Why it belongs in
codingrather than a new categoryThe
codingcategory already holds the agents themselves — Claude Code, Codex, Cline, goose, Pi — plus tooling around them (agent-skills,Taste-Skill). This is tooling of the second kind: a way to run a past session again.It records the agent's exchange with its model provider at the HTTP boundary, so the recording is the bytes the agent actually sent and received rather than a summary it wrote about itself.
orca replay lastruns that exchange again with no provider contacted and nothing spent.orca replay last --from <n> --model <other>replays the prefix from the recording and then continues on a different model, so the model is the only variable. This is bounded by checkpoints, not by arbitrary events:deriveCheckpointsonly produces a checkpoint where there is both a complete conversation prefix and a filesystem snapshot,--forkrefuses a run that has none, and a--fromthat does not land on a checkpoint is snapped back to the nearest prior one withfork.snapped. So a reader comparing models from a particular event starts from an earlier state, or finds the run cannot fork.Against the inclusion criteria
codingentries (Claude Code, Codex, Cline, goose) are agents it records directly; goose and Claude Code have adapters and Claude Code is validated against a real bug fix.spec/orca-trace-v0.md), so a recording is readable without the CLI.orca quickstartwrites a small buggy project plus a recording and replays it, so the claim is checkable in one command.orcareplay. (Star count left toupdate-stars.jsrather than written into the PR text.)Not a cross-list entry — it is not a multi-agent framework.
Steps followed
Run on Node 22.23.2. The branch is rebuilt on current
main, so the earlier conflict is gone; the diff is+13 −0indata/projects.jsonand+1 −0in the generated README.What I am not claiming
🤖 Generated with Claude Code