Repository navigation
Chat: wake resolves the freshly invited run's binding; seed writers share one asset ensure (CL-7486) - #675
Merged
Conversation
…t-against-interchange-main-before-the' into cl-7486-wake-binding # Conflicts: # packages/agent-directory/src/one-shot-prompt.ts # packages/chat/src/platform-adapter.ts # packages/chat/src/send-run-mail.ts # packages/webhook-triggers/src/launch.ts # packages/webhook-triggers/test/launch.test.ts # packages/workflows/src/index.ts # packages/workflows/src/launch/agent-session.ts # packages/workflows/test/agent-session.test.ts
…uplicate key Merging cl-7107's repin/re-vendor audit into this branch left packages/workflows/src/index.ts re-exporting the pre-CL-7481 agent-session API (recordAgentSessionForRun) that no longer exists, its matching test file testing that same stale API, and a duplicated eventCollectors key in apps/hub/src/index.ts's webhook-trigger launch wiring. Restores the current exports (recordAgentSessionAtProvision, ensureRunSession) and their test, and drops the duplicate object key.
sendRunMailWithReclaimRetry retried every attempt against the address it started with. Proves the reproduction of the production failure: a participant already one relaunch generation in that dies again mid-retry must have its next retry chase the run that replaced it, not keep hammering the address that just died.
sendRunMailWithReclaimRetry retried every delivery attempt, and woke, against the address it captured at the top of the loop. A wake in between retries can relaunch the run (its dead-run branch repoints the room's stable participant at a fresh address without telling this loop), and a participant already one relaunch generation in that dies again mid-retry leaves this loop holding an address that is no longer either the stable participant id or the current run id — invisible to readBindingByAddressAnyTenant. The next wake for it then threw "No workbench_launch binding" instead of finding the run that replaced it, which is what chat e2e's mention-fanout and invite-drives-a-turn cases were hitting. Re-resolves the live address (and its session) off the stable participant id after each wake, so the retry loop always chases whichever run is actually current. Also hardens publishCorbitsToolsRegistry's create-then-list-on-conflict ensure: two overlapping seed runs racing the same tenant's package registry asset can still throw a raw asset_tenant_kind_name violation if the substrate's own 409 mapping is bypassed; the ensure now also recognizes that violation by name and resolves the winner's row instead of throwing.
…486-wake-binding # Conflicts: # packages/webhook-triggers/test/launch.test.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Resolves CL-7486.
Stacks on #671 (cl-7481-eager-session).
Root causes
Wake retry never re-resolved the live address.
sendRunMailWithReclaimRetry(packages/chat/src/platform-adapter.ts) retried every delivery attempt, and every wake in between, against the address it captured at the top of the loop. A wake can itself relaunch the run —wakeByAddress's dead-run branch repoints the room's stable participant at a fresh address without telling the retry loop about it. A participant already one relaunch generation in (workbench_launch.instanceIdstable,currentRunIdalready repointed once) that dies again mid-retry leaves the loop holding an address that is now neither the stable participant id nor the current run id — invisible toreadBindingByAddressAnyTenant, which only matches those two columns. The next wake for that address then threwNo workbench_launch binding for address "..."; instances launched before launch-body persistence existed cannot be woken, instead of finding the run that replaced it. This is exactly the "mention fan-out drives the mentioned run", "inviting the echo agent … receives @mentions", and "auto-invites the echo agent and delivers un-mentioned messages" failures from PR Record a run's session at provision; re-key it when Interchange anchors the run (CL-7481) #671 CI (run 34058688883): the hub log shows the wake failing for a run id that was never independently deployed — the idsendRunMailWithReclaimRetrystarted with, already superseded by a relaunch. Fixed by re-resolving the live address (and its session) from the stable participant id after every wake, so the loop always chases whichever run is actually current (packages/chat/src/platform-adapter.ts:1005-1060, call site atpackages/chat/src/platform-adapter.ts:~1420).Asset-ensure didn't tolerate a raw constraint violation.
publishCorbitsToolsRegistry'sensureRegistryAsset(packages/tool-registry-publish/src/publish.ts) already did create-then-list-on-409 for the tenant's package-registry asset, but only recognized an HTTP 409 as the race signal. Two overlapping seed runs for the same tenant (the root tenant's own boot-timerunSystemSeedracing a second caller seeding the same tenant) can both pass the initiallistRegistryAssetcheck before either has created the row, and the loser can still see the rawasset_tenant_kind_nameunique-constraint violation surface as a generic failure rather than a clean 409.ensureRegistryAssetnow also recognizes that violation by name and resolves the winner's row instead of throwing (packages/tool-registry-publish/src/publish.ts:96-140).Also folded in, as fallout from re-merging cl-7107's repin/re-vendor branch onto this stack:
packages/workflows/src/index.tswas re-exporting the pre-CL-7481recordAgentSessionForRunAPI that no longer exists (restored torecordAgentSessionAtProvision/ensureRunSession), its matching test predated the same rename (restored), andapps/hub/src/index.ts's webhook-trigger launch wiring had picked up a duplicateeventCollectorsobject key from the merge.Tests
packages/chat/test/platform-adapter.test.ts: newsendRunMailWithReclaimRetry re-resolves the live addresstest, stubbed db, reproducing the wake-binding failure — confirmed red against the pre-fix loop (retries exhaust and throwAgentUnreachableError) and green after.Local verification
bunx tsc --noEmitclean forpackages/chat,packages/seeding,apps/hub,packages/webhook-triggers.bun test packages/chat/src packages/seeding apps/hub/src: 442 pass, 0 fail.bun test packages/chat/test/platform-adapter.test.ts: 54 pass, 0 fail (new test included, confirmed red/green).scripts/e2e/chat.test.tsagainst a local Postgres: the "No workbench_launch binding" crash is gone after the fix — the three previously-crashing cases now fail cleanly withAgentUnreachableError(exhausted reclaim retries) rather than the wake-binding throw. Local sidecar processes were heavily contended on this machine (load average 8-12, ~80 concurrent node/bun processes from other agents' worktrees) and never completed a deploy inside the retry budget for any of the three; this reads as environment resource contention rather than a code defect, given the unit-level reproduction is deterministic and green. CI's isolated runners are the authoritative check for this suite.