bug(docker): sandbox stays in Error after its container restarts, and stop/start cannot recover it #4828

User Story

I use OpenShell through NVIDIA NemoClaw, which runs agent sandboxes on the Docker compute driver.
I directly encountered this when the Docker container that backs a Ready sandbox was restarted (docker restart, as a Docker daemon restart or host maintenance would do). The sandbox stays in Error permanently, even though the container and its supervisor come back.
This affects every sandbox on the gateway, whatever workload runs in it. It was first reported for NemoClaw sandboxes in NVIDIA/NemoClaw#13165.

Problem Statement

After the backing container of a Ready Docker-driver sandbox restarts:

• The supervisor reports ReportMainProcessExit.
• The restarted supervisor reconnects with ConnectSupervisor.
• The sandbox phase still changes to Error and never returns to Ready.

Error then blocks every lifecycle operation that could recover the sandbox:

• exec is rejected because the sandbox is not Ready.
• stop requires Ready.
• start requires Stopped.

Restarting the gateway does not clear the persisted Error either.

Impact / Why This Matters

A routine container restart turns a healthy sandbox into one that no supported command can recover. The only remaining path is openshell sandbox delete and a new create, which discards the sandbox workspace.

Tools built on OpenShell are affected as well. NemoClaw's start, stop, recover and rebuild fail on such a sandbox: rebuild cannot take its pre-recreate backup because exec is rejected.

Acceptance Criteria

• After the backing container of a Ready sandbox restarts and its supervisor reconnects, the sandbox returns to Ready without user action, and exec works again.
• If the sandbox does go to Error in this case, the supported recovery works: stop is accepted from Error, then start returns the same sandbox ID to Ready with its workspace intact.
• A gateway restart does not keep a sandbox in Error when its container is running and its supervisor is connected.

Reproduction Steps

1. openshell sandbox create --name plain-rs --detach --no-tty -- sleep infinity
2. Wait for openshell sandbox list to show plain-rs as Ready. Then confirm that openshell sandbox exec --name plain-rs -- echo ok prints ok.
3. Restart the backing container: docker restart <container of openshell-default--plain-rs-...>.
4. Poll openshell sandbox list for 2 minutes, then run:

$ openshell sandbox list            # after 30 s, 60 s and 120 s
plain-rs  ...  Error                 # container: "Up 2 minutes"
$ openshell sandbox exec --name plain-rs -- echo ok
Error: × sandbox 'plain-rs' is not ready (phase: Error); wait for it to reach Ready state
$ openshell sandbox stop plain-rs
Error: × code: 'The system is not in a state required for the operation's execution', message: "sandbox must be Ready to stop (current phase: Error)"
$ openshell sandbox start plain-rs
Error: × code: 'The system is not in a state required for the operation's execution', message: "sandbox must be Stopped to start (current phase: Error)"

The gateway log for the restart shows the sequence:

INFO request{... path="/openshell.v1.OpenShell/ConnectSupervisor" ...}
INFO request{... path="/openshell.v1.OpenShell/ReportMainProcessExit" ...}
INFO request{... path="/openshell.v1.OpenShell/ConnectSupervisor" ...}

After that, the phase stays Error. We saw the same result with two NemoClaw sandbox images (Hermes Agent and OpenClaw), and also after a gateway restart.

Suggested UX (if applicable)

None. The existing stop / start commands are enough if they accept this state.

Environment

OS:            Ubuntu 24.04 x86_64
Docker:        28.2.2
OpenShell:     0.0.116 (CLI and gateway, Docker compute driver)
Gateway:       local, started by NemoClaw

Possibly related: #4727 (recovery stop vs. supervisor-failure watch restoring Error) and #4115 (lifecycle coordination across drivers). Neither covers a container restart with a reconnected supervisor.