feat: make gateway compute-driver diagnostics driver-owned #4827

User Story

While maintaining OpenShell and reviewing its diagnostic workflow, I need openshell doctor to report whether the selected gateway's configured compute driver is healthy. I want diagnostics that describe the environment OpenShell actually uses, including remote gateways, without adding each driver's runtime detection and configuration rules to the CLI.

This proposal follows the direction raised in #3694 (comment). It is a request for community feedback on the diagnostic contract and workflow before implementation.

Problem Statement

openshell doctor check currently runs docker info on the CLI host and reports DOCKER_HOST. It does not query the selected gateway or assess its configured compute driver. A successful local Docker check can therefore describe a different endpoint or machine from the runtime that OpenShell uses.

Extending the CLI with separate Docker, Podman, Kubernetes, VM, and future driver checks would duplicate driver-specific knowledge and require continued synchronization with driver configuration and startup behavior.

Impact / Why This Matters

Users can receive “All checks passed” while sandbox operations fail. They must then inspect driver configuration, runtime tools, and gateway or supervisor logs to understand what was actually checked and which prerequisite is missing.

Examples include Docker context/socket disagreement (#4155), missing Docker Desktop prerequisites (#4133), and Docker-only checks for Podman users (#3694, #1834). These are distinct failures with a common diagnostic ownership problem.

Proposed Design

Make compute drivers responsible for assessing their runtime health and prerequisites. Have the gateway expose those results through a common diagnostic interface, and have openshell doctor query the selected gateway and render them.

• Diagnose the configured driver using the effective endpoint and configuration used by that gateway. Local runtime installations on the CLI host should not determine a remote gateway's result.
• Return named checks with status, explanation, actionable remediation where available, and observation time. Distinguish failures, unsupported checks, unknown state, and stale results from success.
• State what each check establishes. Runtime connectivity alone must not claim that sandbox creation, isolation enforcement, or every workload will succeed.
• Support human-readable output and structured JSON with documented exit behavior for automation.
• Keep driver validation useful during gateway startup. If the gateway or driver cannot initialize, preserve actionable startup diagnostics; a gateway query cannot be the only way to diagnose that failure.
• Treat the disposition of today's local Docker-only command, and any explicit offline/pre-start workflow, as design decisions. Avoid silently presenting a local preflight as a gateway assessment.

This proposal defines ownership and observable behavior, not a particular Rust trait, RPC shape, or polling mechanism. Checks should be bounded and should not create workloads by default; any deeper sandbox probe should be an explicit operation.

Notional UX

Command spelling and output are illustrative:

openshell doctor check
openshell doctor check --output json

Gateway: production
Compute driver: podman

Runtime connectivity .... ok
Required runtime access . FAILED
  <driver-provided reason and recommended action>
Sandbox isolation probe . not checked

1 check failed. Sandbox creation was not tested.

If the gateway is unreachable, report that explicitly and direct the operator to startup diagnostics. If its version cannot provide driver diagnostics, report unsupported rather than healthy.

Questions for community feedback

1. Is driver-owned assessment, exposed by the gateway, the right long-term boundary?
2. Which minimum checks should every driver provide, and which should be driver-specific?
3. What should users run before a gateway can start, and how should the existing Docker-only command transition?
4. Should these results appear under doctor, gateway status, or both? How should results describe individual replicas when a gateway has multiple replicas?

Acceptance Criteria

• Diagnostics target the selected gateway and its configured compute driver, without driver-specific runtime detection in the CLI.
• Results reflect the driver's effective runtime connection; an unrelated healthy local runtime cannot mask failure.
• Checks explain their scope, failure reason, and remediation where available; unsupported, unknown, timed-out, and stale results cannot pass silently.
• Human-readable and JSON results provide consistent, documented exit behavior.
• An unreachable gateway and an older gateway without diagnostic support produce distinct, actionable outcomes.
• Driver initialization failures remain diagnosable before the gateway can serve requests.
• The transition from the current Docker-only check is documented, including any retained pre-start workflow.
• At least Docker and Podman demonstrate the common interface; other drivers explicitly report their supported coverage.

Alternatives Considered

• Add driver branches to the CLI. Improves individual cases but duplicates driver knowledge and does not resolve remote-gateway or endpoint disagreement.
• Probe every locally installed runtime. Useful as a host inventory, but a healthy unused runtime does not establish that the configured driver works.
• Reuse existing gateway health unchanged. Current gRPC health returns a healthy status without checking compute-driver health; the HTTP readiness path monitors database health. Neither establishes driver prerequisites.
• Use middleware, interceptors, or providers. These extension points do not own compute-runtime connectivity and prerequisites. Compute drivers are the applicable extension boundary for this assessment.

Related Issues

Direct diagnostic gaps:

• feat(cli): support Podman in openshell doctor check #3694 — Podman support in doctor check; origin of this proposal. Contributor implementation work is already discussed there and should be considered when agreeing on a transition.
• bug: openshell doctor check doesn't verify Docker Desktop prerequisites (Landlock kernel, host networking) #4133 — Docker Desktop Landlock and host-networking prerequisites are not checked.
• VM driver doesn't find Docker Desktop's socket, so locally built images fail with Docker Hub "Not authorized" #4155 — Doctor checks the Docker CLI context while the VM driver uses a different socket.
• [Bug] Gateway auto-detection fails to discover Podman on macOS #1834 — Earlier Podman auto-detection report also calls out Docker-only diagnostics (closed).
• fix(gateway): kubelet fails with permission denied on /dev/kmsg under rootless Podman #457 — Rootless Podman prerequisites were missed in the historical k3s-based deployment (closed; architectural context, not a current k3s requirement).

Related visibility requests, which should inform scope without implying that driver health covers sandbox security:

• Landlock denies reads on bind mounts on Docker Desktop for Mac with VirtioFS file sharing #4733 — Surface inaccessible bind mounts under applied Landlock policy.
• feat(cli): surface active security layer status per sandbox #745 — Expose active security layers per sandbox (closed).
• sec(sandbox): graceful degradation to no network namespace isolation #586 — Surface isolation degradation (closed).
• feat(cli): openshell doctor security posture check for gateways #4477 — Gateway security posture assessment. Gateway configuration and effective-policy findings remain a separate responsibility from compute-driver readiness, even if presented through a shared diagnostic workflow.

Reports illustrating the limits of a passing doctor check, rather than failures this proposal automatically fixes:

• OpenShell 0.1.4 — Docker sandbox supervisor fails to fetch startup configuration on macOS #4683 — Supervisor startup configuration fetch fails on macOS despite doctor passing.
• sandbox start fails after initial command exits: 'supervisor authentication bundle does not match runtime session' #4505 — Sandbox restart authentication-bundle mismatch despite doctor passing.

Agent Investigation

Source inspection at 56dc3daff08c6409a449a96fbca996d1b3200471:

• crates/openshell-cli/src/run.rs, doctor_check(): runs docker info --format '{{.ServerVersion}}' and prints DOCKER_HOST; no gateway query or driver selection.
• crates/openshell-server/src/grpc/mod.rs, health(): returns ServiceStatus::Healthy without a compute-driver health check.
• crates/openshell-server/src/readiness.rs: background database health monitoring is separate from compute-driver assessment.
• GetGatewayInfo exposes initialized driver identity/capabilities, not an ongoing diagnostic assessment.

This is a design proposal based on source review and linked reports, not a claim that the reported environments were reproduced.