Repository navigation
fix(sandbox): keep serving through transient accept errors - #5211
Conversation
|
/ok to test c08b0ae |
c08b0ae to
ca48c21
Compare
|
/ok to test 7d00562 |
mrunalp
added
area:sandbox
|
Label |
|
/ok to test 7d00562 |
|
Label |
The boundary control listener treated any accept error beyond the benign kinds as fatal, so a transient EMFILE or ENFILE ended the boundary and the sandbox with it. The DNS TCP relay broke out of its accept loop on any error at all, including a connection that reset before accept, which silently disabled TCP DNS for the rest of the sandbox's life. Classify accept errors in a shared accept_errors module: EMFILE, ENFILE, ENOBUFS and ENOMEM are transient resource exhaustion, and ConnectionReset, ConnectionAborted and NotConnected mark a peer that disappeared before its connection could be served. The control listener now retries resource errors with a short backoff and a warning, and only unrecoverable errors end the boundary. The DNS TCP relay skips reset peers, retries resource errors with the same backoff, and logs before stopping on an unrecoverable error instead of stopping silently. Unit tests inject these errors into both classification paths, and a loop-level test drives the DNS TCP relay through an EMFILE and a reset-before-accept before serving a real query, then stops it on an unrecoverable error. Closes NVIDIA#5183 Signed-off-by: Emilien Macchi <emacchi@redhat.com>
Signed-off-by: divesh <dgude@nvidia.com>
Head branch was pushed to by a user without write access
7d00562 to
16d64f7
Compare
|
/ok to test 16d64f7 |
Summary
Two accept loops in the sandbox treated any unexpected accept error as fatal. The boundary control listener ended the whole boundary on EMFILE-class errors, and the DNS TCP relay stopped serving TCP DNS on any error, including a peer that reset before its accept completed. Both loops now classify accept errors and keep serving through the transient ones.
Related Issue
Closes #5183
Changes
accept_errorsmodule inopenshell-sandboxclassifying accept errors: transient resource exhaustion (EMFILE, ENFILE, ENOBUFS, ENOMEM), peers that reset before they could be served, and the already-pending network errors Linux reports fromaccept(2)(ENETDOWN, EPROTO, EHOSTUNREACH, and friends) that the man page says to treat like EAGAINRelease Notes
A sandbox no longer exits when its control listener hits a transient accept error such as EMFILE, and DNS over TCP keeps working after a client resets its connection before it is accepted.
Testing
cargo clippy --locked -p openshell-sandbox --all-targets -- -D warningscargo fmt --all -- --checkcargo nextest run --locked -p openshell-sandbox(303 tests pass)servethrough descriptor exhaustion with a queued control connection and asserts it serves again after recovery, and DNS relay tests that inject EMFILE and a reset-before-peer_addrand assert a query still round-tripsChecklist