Skip to content

fix(worker): keep taking sessions while a recycled browser finishes slow ones - #152

Open
mbroton wants to merge 2 commits into
mainfrom
fix/worker-overlap-recycle
Open

mbroton wants to merge 2 commits into
mainfrom
fix/worker-overlap-recycle

Conversation

@mbroton

@mbroton mbroton commented Nov 13, 2026

Copy link
Copy Markdown
Owner

When a worker reached MAX_LIFETIME_SESSIONS, it stopped taking new sessions until every open session ended or DRAIN_TIMEOUT expired. One slow session could stop the worker's capacity for the full drain timeout at each recycle (300 s by default), and with one worker, all capacity stopped.

Now a recycle starts the new browser at once and keeps the worker available. Open sessions finish on the old browser, which closes after its last session ends or when DRAIN_TIMEOUT expires. The session budget counts accepted sessions, so the recycle starts as soon as the budget is spent. A worker runs at most two browsers: if the budget runs out again while the old browser is still open, the current browser keeps serving and recycles at a later limit. SIGTERM still drains and exits, also when it arrives during a recycle. No configuration changes.

Verified with one worker (MAX_SLOTS=3, MAX_LIFETIME_SESSIONS=10, DRAIN_TIMEOUT=30), one 45 s session, and short sessions for 60 s: main served 499 sessions with 2 failed after a 30 s wait; this branch served 1320 with 0 failures, a 137 ms maximum, and at most two browsers. Worker, server, and website checks pass, and the local Compose smoke test recycled Chromium, Firefox, and WebKit workers.

A recycle drained the worker first: it took no new sessions until all open
sessions ended or DRAIN_TIMEOUT expired, so one slow session stopped the
worker's capacity at each recycle.

The worker now starts the new browser at once, sends new sessions to it,
and closes the old browser after its last session ends or DRAIN_TIMEOUT
expires. The session budget counts accepted sessions, so the recycle starts
when the budget is spent. If the budget runs out again while the old
browser is still open, the worker renews the budget and keeps the current
browser, so it never runs more than two browsers. SIGTERM still drains and
exits.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant