Skip to content

feat(#452): doctor reports sandbox tier + opt-in auto-start — completes The Scientist (PR 3/3)#464

Merged
richard-devbot merged 3 commits into
mainfrom
feat/452-doctor-sandbox-tier
Jul 22, 2026
Merged

feat(#452): doctor reports sandbox tier + opt-in auto-start — completes The Scientist (PR 3/3)#464
richard-devbot merged 3 commits into
mainfrom
feat/452-doctor-sandbox-tier

Conversation

@richard-devbot

Copy link
Copy Markdown
Owner

#452 PR 3/3 — doctor reports the active sandbox tier + opt-in bounded auto-start (closes #452)

The final piece. PR 1 built the sandbox; PR 2 wired it into the gate; PR 3 makes the enforcement strength visible to operators and answers the "what if Docker isn't running?" question you raised.

What changed

  • doctor now reports the sandbox execution tier — the honest answer to "are my test results real or self-reported?":
    • PASScontainer-verified (docker/podman): a runtime is ready and an authoritative command is configured → sdlc_validate runs it and authors evidence from the real exit code.
    • WARN — ready but no command configured / installed-but-engine stopped (with the exact start hint) / no runtime at all / disabled in config. Each degrades to unverifiednever a false green.
  • Opt-in bounded auto-start (doctor --start-runtime, or RSTACK_SANDBOX_AUTOSTART=1) — if the engine is installed but stopped, launch it (open -a Docker / Docker Desktop / podman machine start) and poll readiness with a hard timeout. It never launches an engine otherwise — no hidden GUI launch mid-gate (exactly the concern you flagged).
  • Correctness fix (spotted building this): detectContainerRuntime now probes docker/podman info (daemon reachable), not --version (binary installed). Previously a present binary with a stopped daemon made runInSandbox spawn docker run, hit "Cannot connect to the Docker daemon," and mis-record that connection error as a test FAILURE. Now it correctly degrades to unverified. detectInstalledRuntime keeps the --version check so auto-start can tell stopped (startable) from not installed.

Posture (unchanged, your sign-off)

Confined-only executes · no-runtime → honest unverified · network OFF · auto-upgrade to container when present. Auto-start is opt-in and bounded.

Verification

  • 1594/0 full suite; typecheck 0; lint 0 errors; validate 196/196; security OK; git diff --check clean.
  • New tests/sandbox-doctor-452.test.js: tier verdicts, readiness-vs-installed detection, platform-aware start commands, and the injectable bounded auto-start (not-installed / already-running / becomes-ready / times-out / no-start-path) — all without a real daemon.

This closes #452 and the sandbox trilogy

PR 1 (core) + PR 2 (validate wiring) + PR 3 (tier + auto-start) = The Scientist is complete: untrusted code runs confined, the gate uses the real exit code, the Critic gets the real logs, and operators can see exactly how strong verification is on their machine.

Next in epic #450: #453 — Quality & Risk Index, riding on this real execution signal. That closes the autonomy epic.

Closes #452

richardsongunde and others added 3 commits July 22, 2026 13:31
…tart

- detectContainerRuntime now probes `docker/podman info` (daemon REACHABLE),
  not `--version` (binary installed). A present binary with a stopped daemon
  used to make runInSandbox spawn `docker run`, hit a connection error, and
  mis-record it as a test FAILURE — now it correctly degrades to 'unverified'.
- detectInstalledRuntime keeps the `--version` check so auto-start can tell
  'installed but stopped' (startable) from 'not installed' (nothing to start).
- runtimeStartCommand: platform-aware engine boot (open -a Docker / Docker
  Desktop / podman machine start; null on linux system daemons).
- startContainerRuntime: OPT-IN, bounded auto-start — launches the engine then
  polls readiness with a HARD timeout so a gate never hangs on a booting GUI.
  Fully injectable.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- sandboxTierCheck (pure, exported) answers 'are my test results real or
  self-reported?': container-verified (PASS) when a runtime is ready and a
  command is configured; WARN for ready-but-no-command, installed-but-stopped
  (with the exact start hint), no-runtime, or disabled — never a false green.
- checkSandboxTier wires it into runDoctor after the config checks.
- doctor --start-runtime (or RSTACK_SANDBOX_AUTOSTART=1) opts into the bounded
  auto-start; never launches an engine otherwise.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
sandboxTierCheck verdicts (PASS/WARN across ready/no-command/stopped/absent/
disabled + auto-start note), detectInstalledRuntime, platform-aware
runtimeStartCommand, and the injectable bounded startContainerRuntime
(not-installed / already-running / becomes-ready / times-out / no-start-path).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@qodo-code-review

Copy link
Copy Markdown

Qodo reviews are paused for this user.

Troubleshooting steps vary by plan Learn more →

On a Teams plan?
Reviews resume once this user has a paid seat and their Git account is linked in Qodo.
Link Git account →

Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center?
These require an Enterprise plan - Contact us
Contact us →

@coderabbitai

coderabbitai Bot commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

@richard-devbot, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 4 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 4a07a8a8-aab9-45e6-a3a5-88a8301f4437

📥 Commits

Reviewing files that changed from the base of the PR and between edb18db and 18f597f.

📒 Files selected for processing (4)
  • bin/rstack-agents.js
  • src/commands/doctor.js
  • src/core/harness/sandbox.js
  • tests/sandbox-doctor-452.test.js
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/452-doctor-sandbox-tier

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@richard-devbot
richard-devbot merged commit 724041f into main Jul 22, 2026
9 checks passed
@richard-devbot
richard-devbot deleted the feat/452-doctor-sandbox-tier branch July 22, 2026 08:06
richard-devbot pushed a commit that referenced this pull request Jul 22, 2026
#464)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Transient Sandbox Execution — 'The Scientist' (execute-and-observe, real evidence)

2 participants