Skip to content

bug: a transient accept error ends the boundary control listener or the DNS TCP relay #5183

Closed

Description

@EmilienM

User Story

I run OpenShell sandboxes with the Podman and Kubernetes drivers, and I read through the isolation core (openshell-sandbox, openshell-binary-identity, openshell-isolation-interface) at 00528d24a. This item comes from reading the code. I have not hit it in production. It is one line of the umbrella issue #5179.

Problem Statement

Two accept loops treat any error as fatal:

  • crates/openshell-sandbox/src/boundary_server.rs:663: the control listener returns Err("accept boundary control connection: ...") for any accept error other than the handled benign kinds, which ends the boundary. EMFILE (and ENFILE) are transient and recoverable.
  • crates/openshell-sandbox/src/network_broker.rs:479: the openshell-dns-tcp thread breaks out of for accepted in tcp.incoming() on any accept or peer_addr error, including a connection reset before accept (ENOTCONN) and EMFILE. The relay then stops serving TCP DNS for the rest of the sandbox's life.

Impact / Why This Matters

A transient resource error takes down the control endpoint (the sandbox exits) or silently disables TCP DNS for the remaining lifetime of the sandbox, instead of being retried.

Acceptance Criteria

  • The control listener retries EMFILE/ENFILE/ENOBUFS/ENOMEM with a short backoff and logs, and only unrecoverable errors end the boundary.
  • The DNS TCP loop skips a connection that resets before peer_addr and retries on resource errors, and keeps serving.
  • Unit tests inject these errors.

Reproduction Steps

Found by reading the code; not reproduced live. A unit test can inject EMFILE and a reset-before-accept into the two accept loops and assert they keep serving.

Environment

  • OpenShell: main at 00528d24a
  • OS: Fedora 44, Linux 7.2.14
  • Deployment: Podman driver (local gateway); code read for the Kubernetes driver too

Activity

  1. added
    state:acceptedA maintainer decided OpenShell should pursue this issue
    and removed
    state:triage-neededOpened without agent diagnostics and needs triage
    on Nov 3, 2026
  2. mrunalp commented on Nov 3, 2026

    @mrunalp
    Collaborator

    Accepted for implementation in #5211.

    Source inspection at 32dcc4a67: The boundary control listener returns on unclassified accept errors, and the TCP DNS relay exits on accept/peer-address errors. Retrying transient resource errors with bounded backoff and retaining service after reset peers addresses the reported failure paths.

    This assessment is based on the source and the linked report/PR; I have not independently run deployment reproductions or the PR test suite. CI and code review still need to validate the fix.

  3. added 2 commits that reference this issue on Nov 4, 2026
    ca48c21
    e1e7c40
  4. added a commit that references this issue on Nov 5, 2026
    10ace4f
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    state:acceptedA maintainer decided OpenShell should pursue this issue

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions