Repository navigation
Description
User Story
I work on OpenShell's native Windows support and run the gateway with the built-in MXC driver and real sandboxes on Windows 11 x64. I directly encountered this when I hard-killed a gateway that had three running sandboxes: the sandboxes' processes kept running and a new gateway could not start on the same port. This affects anyone who restarts or loses the gateway while sandboxes are running.
Problem Statement
With the MXC driver built into the gateway, the gateway starts wxc-exec, openshell-supervisor and openshell-windows-sandbox for each sandbox. If the gateway process is killed (crash, taskkill, logoff), those processes are not ended. They keep running with no owner, and they keep the gateway's listening socket open, so the port stays bound. A new gateway then fails to start with "failed to bind ... Only one usage of each socket address (os error 10048)".
Two things combine to cause it:
- Nothing ties the lifetime of the sandbox processes to the gateway process.
- The gateway's listening sockets are created with plain
socket()(mio 1.2.0 on Windows), which leaves the handle inheritable. Child processes are started with handle inheritance enabled, so each sandbox process holds its own copy of the socket and keeps the port bound after the gateway dies.
Impact / Why This Matters
After a gateway crash or forced stop, the operator has to find and kill the leftover sandbox processes by hand before a new gateway can bind its port. Until then the sandboxes run ungoverned: no gateway enforces their policy or can delete them, and the host cannot reconcile them.
Acceptance Criteria
- After the gateway is hard-killed, none of the sandbox processes it started (
wxc-exec,openshell-supervisor,openshell-windows-sandbox) are still running. - Immediately after the kill, the gateway port can be bound again and a new gateway starts on the same port.
- The gateway's main, health and metrics listeners are not inheritable by child processes.
- Creating, running and deleting sandboxes behaves as before.
Reproduction Steps
- On Windows with the MXC driver, start the gateway on port 17680 with the built-in MXC driver (
OPENSHELL_COMPUTE_DRIVER=mxc). - Create three long-running sandboxes with
openshell sandbox create --detachand confirm all three areReady. - Hard-kill the gateway:
Stop-Process -Id <gateway pid> -Force. - After 5 and again after 25 seconds, list
openshell-supervisor,wxc-execandopenshell-windows-sandboxprocesses, and try to bind port 17680. - Start the gateway again on port 17680.
Actual: nine sandbox processes (three of each) are still running after 25 seconds, the port cannot be bound, and the new gateway exits with the bind error. Killing all openshell-windows-sandbox processes releases the port. Observed in 5 of 5 runs.
Expected: no sandbox process survives, the port can be bound, and the new gateway starts.
Environment
- OpenShell:
openshell 0.1.5-dev.40+g5b9e5b91a(windowsbranch at 5b9e5b9) - OS: Windows 11 x64, native
- Deployment: gateway with the built-in MXC driver,
backend = "process_container", plaintext gateway on loopback
Logs
Error: x transport error
`-> failed to bind to 127.0.0.1:17680: Only one usage of each socket address (protocol/network address/port) is normally permitted. (os error 10048)
Activity