Skip to content

pg: the timeout sweep removes the connection that timed out, not whatever its index holds afterwards - #313

Merged
MDA2AV merged 4 commits into
mainfrom
fix/pool-sweep-reentry
Oct 12, 2026
Merged

MDA2AV merged 4 commits into
mainfrom
fix/pool-sweep-reentry

Conversation

@MDA2AV

@MDA2AV MDA2AV commented Oct 7, 2026 •

Copy link
Copy Markdown
Owner

Now pg-only. The Redis half went with ioxide.redis, which #337 drops; main is merged in, and the Pg suite passes (7).

When a timed-out pg query was handled by issuing another one, the pool's timeout sweep disposed the live connection that the new command had just gone to, and that command failed with send failed: errno 9.

Sweep walks _connections by index. For a connection whose oldest command is overdue, CheckTimeout marks it broken and fails its waiters. The waiters resume inline, inside the sweep. A handler that catches the timeout and queries again calls Pick. Pick evicts the broken connection from _connections, which shifts the list, and returns the live one. Sweep then removed and disposed _connections[i], which by then was the live connection carrying those new commands. When the timed-out connection was the last entry, _connections[i] was past the end instead: the ticker faulted with Index was out of range, and the rest of that sweep, including replenishment, was skipped.

Sweep now keeps the connection it checked, removes it by reference (a no-op if a Pick already removed it), and clamps the index to the list's current length, since one Pick can evict several broken connections. No new state.

Tests

New:

  • pg: queries made by a timed-out query's handler run on the live connection the sweep keeps (PgSweepTests)

It uses a pool of 2 with a 1000 ms command timeout. The handler waits for both connections, then sends pg_sleep(30), which lands on index 0. When the timeout fails it, the handler issues two more commands. The handler also records that the timed-out connection is still listed when its catch runs. That only holds inline in the sweep, so the test can't pass on a continuation that ran after the sweep had finished. A last request checks that the pool still answers.

main this PR
two commands issued from the timeout handler both fail: send failed: errno 9 1, 2
same, with the timed-out connection last in the list (probe, not committed) ticker faulted: Index was out of range, sweep cut short no fault
the two commands, on #292 alone both fail: send failed: errno 32 (with this PR on top of #292) 1, 2

Suites: Pg 7 with main merged in (2026-10-12); Unit 63 (1 pending, as on main) when first run.

Not a hot path: the change is in the 250 ms ticker, and the extra list search runs only for a connection that timed out.

#292 does not fix this: on #292 alone, Sweep shuts the live connection down instead of closing it, and its commands fail with EPIPE. The two conflict in Sweep, because #292 rewrites the comment on the c.Dispose() line that sits between the lines changed here. To resolve, keep this structure with #292's comment. Pg passes with both merged.

…ot whatever its index holds afterwards

Sweep failed a timed-out connection's waiters through CheckTimeout and then removed and disposed
_connections[i]. The waiters resume inline, so a handler that queries again picks from the pool
inside the sweep, and Pick evicts the timed-out connection, which is broken, shifting the list.
Sweep then disposed whatever had moved into index i: a live connection holding the queries just
placed on it, which failed with "send failed: errno 9". When the timed-out connection was the
last entry, the index was past the end and the ticker faulted, skipping the rest of that sweep.

Sweep now holds the connection it checked, removes it by reference (a no-op if a Pick already
did), and clamps the index to the list's current length. No new state.

New tests, PgSweepTests and RedisSweepTests: a pool of 2 with a 1000 ms command timeout, a
pg_sleep(30) or BLPOP for 30 s on index 0, and a handler that issues two more commands when the
timeout fails it. The handler records that the timed-out connection is still listed when it runs,
which shows it ran inline in the sweep. Confirmed both ways: on f298af0 both commands fail with
errno 9 (pg and redis), and pass with the fix. With the sleep moved to index 1, f298af0 logs
"ticker faulted: Index was out of range" and the fix does not.
@MDA2AV MDA2AV mentioned this pull request Oct 12, 2026
@MDA2AV MDA2AV changed the title pg, redis: the timeout sweep removes the connection that timed out, not whatever its index holds afterwards pg: the timeout sweep removes the connection that timed out, not whatever its index holds afterwards Oct 12, 2026
# Conflicts:
#	src/clients/ioxide.pg/PgPool.cs
#	tests/Ioxide.Tests.Pg/Program.cs
@MDA2AV
MDA2AV merged commit 7f2b211 into main Oct 12, 2026
1 check passed
@MDA2AV MDA2AV mentioned this pull request Oct 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant