Skip to content

feat(deploy): start agents through a separate provisioner service - #48

Closed
TheGreatAxios wants to merge 1 commit into
mainfrom
cl-11875-make-agent-tools-work-when-builder-runs-in-docker
Closed

TheGreatAxios wants to merge 1 commit into
mainfrom
cl-11875-make-agent-tools-work-when-builder-runs-in-docker

Conversation

@TheGreatAxios

@TheGreatAxios TheGreatAxios commented Nov 1, 2026 •

Copy link
Copy Markdown
Contributor

First cut of CL-11875 (steps 1–2 of the attached plan). Draft: the rest of the plan (tools inside agent containers, agent network and egress proxy, placement declarations, Landlock) lands on this PR before it is ready.

What changed

  • New apps/sidecar-provisioner: the compose service sidecar-provisioner, the only container with the Docker socket. It serves ensure/destroy behind a shared secret and reuses Builder's Docker provisioner (moved from the hub, hardening unchanged).
  • The hub gets a forwarding provisioner (Interchange contract, apiVersion 2). With SIDECAR_PROVISIONER_URL set, it is the only provisioner: no agents run in the hub. An unreachable service rejects placement with sidecar_provisioner_unreachable and wording the person sees.
  • Compose: the hub has no socket mount or group_add; compose.docker-sidecars.yml is removed; SIDECAR_PROVISIONER_TOKEN is a new required secret.
  • The hub reads the service's Docker binding fingerprint and capabilities from an authenticated GET /binding at startup and registers with them, so Interchange notices a changed Docker backend; a service restarted with a different setup refuses new agents with sidecar_provisioner_binding_changed. Compose starts the hub only once the provisioner is healthy.
  • A wrong or short token (under 32 characters) and malformed requests get clear, configured errors.

Verified

  • lint, format:check, typecheck, test (73 pass), vendor:verify, pins:verify.
  • Independent review; its findings are fixed (binding fingerprint, error mapping, shared Docker setup, token length).
  • A throwaway compose stack built from source: docker inspect shows the hub with no socket and the provisioner holding it; creating a project starts one hardened agent container (read-only root, all caps dropped, no-new-privileges, memory/PID/CPU limits, own volume) that connects to the hub; with the provisioner stopped, project creation returns sidecar_provisioner_unreachable with the person-facing message; after restart, creation succeeds; a wrong token and changed limits each give their configured message and start no second container.

Known gaps (later steps)

  • Tool calls inside agent containers still fail (sandbox runtime under dropped caps).
  • Agent containers reach the hub via host.docker.internal; Linux needs the agent network.
  • Design preview drafts read the local sidecar's disk, which the Docker install no longer has.

@linear-code

linear-code Bot commented Nov 1, 2026

Copy link
Copy Markdown

CL-11875

@TheGreatAxios
TheGreatAxios force-pushed the cl-11875-make-agent-tools-work-when-builder-runs-in-docker branch 2 times, most recently from 990c5aa to abb61e7 Compare November 1, 2026 04:10
@TheGreatAxios
TheGreatAxios force-pushed the cl-11875-make-agent-tools-work-when-builder-runs-in-docker branch from abb61e7 to b172e90 Compare November 1, 2026 04:26
@TheGreatAxios

Copy link
Copy Markdown
Contributor Author

Blocking: the provisioner starts the container, and coder and designer tools still cannot run inside it.

apps/sidecar-provisioner/src/docker.ts creates the sidecar with CapDrop: ["ALL"] and SecurityOpt: ["no-new-privileges"], and dockerSidecarCapabilities reports runtime:posix as available. Compose sets SIDECAR_PROVISIONER_URL, so the hub no longer starts these processes. Bubblewrap then fails with bwrap: No permissions to create a new namespace or bwrap: setting up uid map: Permission denied.

docs/SELF_HOST.md already says that container needs seccomp=unconfined, apparmor=unconfined, and systempaths=unconfined. Those exemptions are still only on the hub service in deploy/compose.yml, which is not the process running the tools. A container that starts is not the tools working.

@TheGreatAxios

Copy link
Copy Markdown
Contributor Author

Closing Temporarily: This is one potential architecture to explore for Provisioning

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant