Skip to content

fix(server): stop short network failures from failing sessions with 502 - #149

Merged
mbroton merged 1 commit into
mainfrom
fix/server-dial-retry
Nov 12, 2026
Merged

mbroton merged 1 commit into
mainfrom
fix/server-dial-retry

Conversation

@mbroton

@mbroton mbroton commented Nov 12, 2026

Copy link
Copy Markdown
Owner

A DNS or network failure shorter than one second made the relay fail new sessions with 502 Bad Gateway. The relay also marked the healthy worker stalled until its next heartbeat. Under load no other worker was free, so the client got the error.

The relay now retries a failed connection to the same worker with a short backoff (about 1.5 s in total, inside WORKER_DIAL_TIMEOUT). A temporary DNS error no longer stalls the worker. A worker that refuses connections or answers with an HTTP error is still stalled, and the session moves to a different worker as before. No configuration changes. The WORKER_DIAL_TIMEOUT docs now say that the limit includes retries.

Verification: the new relay tests fail on main and pass here. In a local Compose load test with 0.5 s worker network drops, main gave 4 × 502 in 17 drops, and this branch gave 0 in 24.

@mbroton
mbroton force-pushed the fix/server-dial-retry branch from 010e0e1 to 1a02d6d Compare November 12, 2026 19:27
@mbroton
mbroton merged commit 14ab88e into main Nov 12, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant