Repository navigation
test(e2e): replay recorded weather app project end to end in CI - #81
Merged
TheGreatAxios merged 1 commit intoNov 2, 2026
Conversation
TheGreatAxios
force-pushed
the
cl-11938-replay-a-whole-builder-project-from-recorded-model-turns-in
branch
3 times, most recently
from
November 2, 2026 05:40
cc9d3a6 to
36a1530
Compare
TheGreatAxios
marked this pull request as ready for review
November 2, 2026 05:40
TheGreatAxios
force-pushed
the
cl-11938-replay-a-whole-builder-project-from-recorded-model-turns-in
branch
2 times, most recently
from
November 2, 2026 05:54
a6d0ed6 to
4b1f787
Compare
There was a problem hiding this comment.
Triage requests changes:
- Conventions: added code matches a code pattern
apps/hub/src/builds/posix-sandbox.ts:435 empty-list-fallback: Invented fallback `?? []`; handle the missing value instead.apps/hub/src/builds/posix-sandbox.ts:527-541 nested-ternary: Nested ternary; use if statements or a lookup.apps/hub/src/builds/posix-sandbox.ts:784-845 iife: Immediately invoked function; name the function.apps/hub/src/builds/posix-sandbox.ts:784-845 promise-chain: Promise chain; use async/await with try/catch.apps/hub/src/builds/sidecar-tools.ts:225-300 promise-chain: Promise chain; use async/await with try/catch.test/e2e/support/extract.ts:118 empty-list-fallback: Invented fallback `?? []`; handle the missing value instead.workflows/coder/check/check-app.ts:576-581 promise-chain: Promise chain; use async/await with try/catch.
TheGreatAxios
force-pushed
the
cl-11938-replay-a-whole-builder-project-from-recorded-model-turns-in
branch
from
November 2, 2026 06:32
4b1f787 to
bcd5608
Compare
There was a problem hiding this comment.
Triage requests changes:
- Conventions: added code matches a code pattern
apps/hub/src/builds/posix-sandbox.ts:567-586 nested-ternary: Nested ternary; use if statements or a lookup.apps/hub/src/builds/posix-sandbox.ts:829-890 iife: Immediately invoked function; name the function.apps/hub/src/builds/posix-sandbox.ts:829-890 promise-chain: Promise chain; use async/await with try/catch.apps/hub/src/builds/sidecar-tools.ts:225-300 promise-chain: Promise chain; use async/await with try/catch.test/e2e/support/extract.ts:118 empty-list-fallback: Invented fallback `?? []`; handle the missing value instead.workflows/coder/check/check-app.ts:576-581 promise-chain: Promise chain; use async/await with try/catch.
TheGreatAxios
force-pushed
the
cl-11938-replay-a-whole-builder-project-from-recorded-model-turns-in
branch
from
November 2, 2026 06:53
bcd5608 to
ecb10db
Compare
|
Triage: In progress Triage is waiting for CI to finish before it decides. This comment shows the verdict once it has. |
TheGreatAxios
enabled auto-merge
November 2, 2026 07:02
TheGreatAxios
force-pushed
the
cl-11938-replay-a-whole-builder-project-from-recorded-model-turns-in
branch
from
November 2, 2026 09:19
ecb10db to
10aad2e
Compare
TheGreatAxios
force-pushed
the
cl-11938-replay-a-whole-builder-project-from-recorded-model-turns-in
branch
from
November 2, 2026 09:57
10aad2e to
51ecc24
Compare
TheGreatAxios
disabled auto-merge
November 2, 2026 10:04
This was referenced Nov 2, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replays a full Builder project end to end against a real Hub, Postgres, sidecar and workspace sandbox without calling a live model provider.
test/e2e/support/recorded-model.tsserves an OpenAI-compatible/v1/chat/completionsSSE endpoint backed by recorded exchanges undertest/fixtures/pipeline/. Each request matches itsexpect.jsonstructurally by run kind, ordered tool declarations, user/tool turn shapes and required tool-result substrings, captures runtime values (approvalId,appUrl) from tool results and substitutes{{bind:<key>}}placeholders in streamed tool call arguments. Any divergence fails with the scenario, stage, run, exchange index and re-record instructions.test/e2e/pipeline.test.tsdrives the animated weather app through the planning documents (problem_brief,constraints,approachesv1, reviewer revision toapproachesv2,product_requirements), the three-screen HTMLdesign(check_designin headless Chrome),build_plan, the coder (app_status, Playwrightbrowser_*checks,bun test,git commit) and the builtapp(v1from starter setup andv2after the coder commits, fetched viaGET /api/projects/:id/app, extracted and verified withbun test), asserting each stage's Hub state and the orderedrunsandartifactssequences againstmanifest.json.apps/hub/src/builds/posix-sandbox.ts,apps/hub/src/builds/sidecar-tools.tsandapps/hub/src/live-app/host.tspass--conditions=intx-srcto the sandboxed tool child, keepTMPDIRpointed at a writable directory inside wrapped commands while denying writes to/tmp/claude, allow the Linux proxy bridge Unix sockets inallowRead, relay the dev app over a Unix socket across Linux sandbox network namespaces (createAppRelaywithbwrapmount/PID isolation), enableallowAllUnixSocketson Linux for sidecar-owned browser/relay helpers after wrapping the agent tool child, and prefix Linux sandbox wrappers withexecsokillterminates the bubblewrap namespace owner directly.apps/hub/src/middleware/correlation.tsattachesrequestContextascontextLocalStoragewhen LogTape was initialized with the default sink beforesetupHubLogging()..github/workflows/ci.ymlkeepse2eon the fast tests (custom-provider,oauth-model-login,specialist-end-notices,recorded-model) and runstest/e2e/pipeline.test.tsin its owne2e-pipelinejob (continue-on-error: true).test/AGENTS.mddocuments the test layout, how recorded replay works and how to update or re-record fixtures when prompts, tools or starter files change.Verified with
bun run format:check,bun run lint,bun run typecheck,bun run test,bun --conditions=intx-src test ./test/e2e/support/recorded-model.test.ts,bun --conditions=intx-src test ./test/e2e/pipeline.test.ts,bun run pins:verifyandbun run vendor:verify.