Repository navigation
Conversation
AdaAibaby
requested review from
ValentaTomas,
dobrac and
jakubno
as code owners
October 16, 2026 07:31
A client disconnect (or its SDK giving up) cancels the request context, which is threaded end-to-end through placement. Two create paths treated the resulting context.Canceled as a server-side failure: they logged at ERROR and, in placement, counted the node's createFails and excluded it. - placement: a create cut by its own context now Skips (not Fails) the node, logs at Warn, and does not exclude it — the cut says nothing about the node's health. The attempt is still counted so deadline() classifies the timeout, and the loop still ends with the caller. - create_instance reserve: a reservation cut by context.Canceled / DeadlineExceeded now logs at Warn instead of ERROR. The access-log/metrics middleware already maps these requests to 499/408 by cancel cause, so client-facing behaviour is unchanged; this only stops the duplicate ERROR logs and inflated createFails driven by client behaviour. Fixes e2b-dev#3683
AdaAibaby
force-pushed
the
fix/api-client-cancel-logging
branch
from
October 16, 2026 08:14
36959cb to
041fc22
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Fixes #3683.
When a client disconnects (or its SDK gives up) before the API finishes creating a sandbox, the request context is cancelled. Because the context is threaded end-to-end through placement, the in-flight create/reserve is cut and surfaces as a generic failure. Two hot paths in
packages/apitreated this expectedcontext.Canceledas a server-side failure — they logged at ERROR and (in placement) incremented the node'screateFailscounter and excluded it for the rest of the attempt.The access-log/metrics middleware already rewrites the HTTP status to 499 by cancel cause, yet the handler's own ERROR logs and telemetry spans still recorded these as 500, so a single client cancellation was recorded twice with two different codes. This is the same class of issue already addressed for envd (#3647) and NBD dispatch (#3274, #3348), on the API create path.
Changes
placement/placement.go): a create cut by its own context nowSkips the node instead ofFailing it, logs at Warn instead of ERROR, and does not exclude the node — a client disconnect says nothing about the node's health. The attempt is still counted sodeadline()classifies the timeout correctly, and the loop still ends with the caller (returnsdeadline()). Genuine live-context failures are unchanged: they still Fail, exclude, and log at ERROR.create_instance.go): a reservation cut bycontext.Canceled/context.DeadlineExceedednow logs at Warn instead of ERROR. Mirrors the existing guards inoutbox/river.go,orchestrator/autoresume.go,handlers/proxy_grpc.go, andfeatureflags/flags.go.Client-facing behaviour is unchanged (still 499/408 via the middleware); this only stops the duplicate ERROR logs and the
createFailsinflation driven by client behaviour. Scheduling was already unaffected (best-of-K scores onInProgress(), which is cleaned up by both Skip and Fail).Tests
TestPlaceSandbox_CutCreateNotCountedAgainstNode: a create that cancels the request context mid-flight must report a timeout, keepFailsCountat zero, and drop its in-progress reservation.TestPlaceSandbox_HardFailureNotWrappedto assert a live-context failure still counts against the node (FailsCount == 1), guarding the two branches against regressing into each other.go test ./internal/orchestrator/placement/...passes.go vetandgolangci-lintclean on the changed packages. (The DB-backed orchestrator tests requirePG_TEST_DATABASE_URL, unavailable in my env; the paths changed here have no DB dependency.)