Skip to content

Migrate Docker enclaves to the shared host-owned execution lifecycle - #10353

Merged
lpcox merged 10 commits into
mainfrom
copilot/migrate-docker-enclaves
Oct 20, 2026
Merged

lpcox merged 10 commits into
mainfrom
copilot/migrate-docker-enclaves

Conversation

Copilot AI commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Move Docker enclaves onto the shared host-owned execution lifecycle while preserving their isolation and bounded-disclosure guarantees.

  • Keep Docker launch authority, trusted repository admission, resource limits, cancellation, and recovery on the host; route broker requests through authenticated executor IPC.
  • Bound writable storage and verify container removal and storage cleanup before releasing results or settling delegated authority.
  • Integrate readiness, admission closure, draining, and shutdown; preserve static repository scope and recoverable workload/admission failures.
  • Add regression coverage and real-Docker acceptance, including script-only, agent-only, and combined-role broker containers without Docker socket or private seed/work-tree custody.
  • Document compatibility, resource accounting, recovery limits, and the exact scope of acceptance evidence.

Validation

  • Build, configured lint, and type checks pass.
  • 9,402 unit/regression tests and all 16 real-Docker acceptance cases pass; the three broker-role cases also pass a final timing-budget recheck.
  • Secret scans pass; independent code review found no remaining significant issues.
  • Supplemental local CodeQL analyzed 843 source and 307 container JavaScript/TypeScript files. Container analysis reported no findings; source findings were triaged as two unchanged production paths, one test-only race warning, and one unused initializer.
  • Automated review was unavailable, and the automated JavaScript CodeQL analysis was skipped because its database exceeded the size limit. Direct integration-file lint retains 11 pre-existing errors; configured repository lint passes.

Remaining acceptance and hand-off

  • Live Copilot/provider execution and compiler-issued GitHub delegation remain unvalidated.
  • Full primary-runtime-plus-enclave combinations, compiler gateway alias resolution, and production dependency startup remain separate acceptance gates; prestarted fixture services do not establish those guarantees.
  • Accounting restart durability is not established by the confinement tests.
  • CI on the pushed HEAD remains unverified and requires maintainer approval/re-trigger before merge.

Fixes #10340

Copilot AI and others added 2 commits October 20, 2026 20:15
Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>
Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>
Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>
@lpcox
lpcox marked this pull request as ready for review October 20, 2026 21:20
Copilot AI max review requested due to automatic review settings October 20, 2026 21:20
@lpcox lpcox changed the title [WIP] Migrate Docker enclaves to shared host-owned execution lifecycle Migrate Docker enclaves to the shared host-owned execution lifecycle Oct 20, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Security-sensitive lifecycle changes retain unresolved correctness issues and incomplete live acceptance gates.

7 open issues
What changed in this PR

Migrates Docker enclaves to AWF’s shared host-owned execution lifecycle while retaining Compose for the primary agent and sidecars.

Changes:

  • Adds host-owned Docker launch, bounded storage, and verified recovery.
  • Routes broker execution and dynamic delegation through authenticated host transport.
  • Integrates readiness and shutdown ordering, with regression tests and compatibility documentation.
File Description
src/​services/​enclave-mcp-service.ts Separates host launch settings from broker authority.
src/​services/​enclave-mcp-service.test.ts Tests broker isolation and Docker configuration.
src/​services/​enclave-dynamic-service.test.ts Tests dynamic host-channel topology.
src/​services/​enclave-agent-service.test.ts Checks removal of broker identity mounts.
src/​enclave/​tool-call-budget.test.ts Characterizes persistence failure behavior.
src/​enclave/​startup-diagnostics.test.ts Tests runtime-specific diagnostic probes.
src/​enclave/​runtime-preflight.ts Requires Docker lifecycle readiness.
src/​enclave/​runtime-preflight.test.ts Tests trusted readiness enforcement.
src/​enclave/​manager.ts Starts and stops Docker host execution.
src/​enclave/​manager.test.ts Tests staging and lifecycle integration.
src/​enclave/​information-budget.test.ts Characterizes ledger restart gaps.
src/​enclave/​host-executor-server.ts Distinguishes recoverable workload failures.
src/​enclave/​host-executor-run-state.ts Derives trusted dynamic admission authority.
src/​enclave/​host-executor-run-state.test.ts Tests static and dynamic authority separation.
src/​enclave/​host-executor-routing.test.ts Tests Docker transport and accounting.
src/​enclave/​host-executor-protocol.test.ts Tests recoverable failure settlement.
src/​enclave/​dynamic-executor.test.ts Preserves legacy gVisor coverage.
src/​enclave/​dynamic-delegation.ts Keeps Docker delegation host-owned.
src/​enclave/​dynamic-delegation.test.ts Tests custody and shutdown failures.
src/​enclave/​dynamic-delegation-service.ts Blocks admission after controller failures.
src/​enclave/​dynamic-delegation-service.test.ts Tests denial and infrastructure-failure distinctions.
src/​enclave/​dynamic-delegation-protocol.ts Clarifies legacy transport documentation.
src/​enclave/​dynamic-delegation-channel.ts Identifies the legacy gVisor channel.
src/​enclave/​docker-lifecycle.ts Orchestrates Docker host execution and settlement.
src/​enclave/​docker-lifecycle.test.ts Tests readiness, custody, and recovery behavior.
src/​enclave/​docker-host-storage.ts Implements bounded host tmpfs storage.
src/​enclave/​docker-host-storage.test.ts Tests capacity and mount verification.
src/​enclave/​docker-host-recovery.ts Journals and verifies Docker resource identities.
src/​enclave/​docker-host-executor.ts Launches, confines, and reaps Docker workloads.
src/​enclave/​docker-host-executor.test.ts Tests execution, cancellation, and recovery.
src/​enclave/​cloud-hypervisor-lifecycle.ts Limits teardown to selected VM runs.
src/​commands/​main-action.ts Orders draining before infrastructure teardown.
src/​commands/​main-action.test.ts Tests shutdown ordering and failure preservation.
scripts/​ci/​filesystem-sharing-contract.ts Records the reduced broker mount surface.
scripts/​ci/​filesystem-sharing-contract.test.ts Tests broker transport and metadata mounts.
docs/​filesystem-sharing-contract.md Documents runtime identities and mounts.
docs/​enclaves-architecture.md Describes migration, restrictions, and acceptance gaps.
docs/​awf-config-spec.md Clarifies delegation and accounting guarantees.
containers/​enclave/​script-executor/​executor-handler.js Reserves disclosure before host admission.
containers/​enclave/​mcp-server/​server.js Removes broker-owned Docker delegation.
containers/​enclave/​mcp-server/​host-executor-runner.js Routes Docker through host execution.
containers/​enclave/​mcp-server/​environment-probes.js Selects probes for host-owned execution.
containers/​enclave/​mcp-server/​config.js Loads Docker host-transport configuration.

Review effort: Max

Comment on lines +35 to +36
return config.executorTransport === 'host'
|| config.executorBackend === 'docker' || config.executorBackend === 'cloud-hypervisor';
Comment thread src/commands/main-action.ts Outdated
if (containersStarted) {
// Keep delegated identities and sidecars alive through host cancellation.
// Failure retains networks, seeds and recovery state, but still saves audit.
await stopDockerEnclaveLifecycle(config);
Comment thread src/enclave/docker-host-recovery.ts Outdated
Comment thread src/enclave/docker-lifecycle.ts Outdated
Comment on lines +289 to +291
if (entry.executorKind === 'script') {
const source = `${plan.invocationHostDir}/repo:/awf/seed:ro`;
args[args.indexOf(source)] = `${plan.seedHostPath}:/awf/seed:ro`;
Comment on lines +46 to +48
if (config.dockerHostPathPrefix && config.dockerHostPathPrefix !== '/') {
throw new Error('Docker host enclaves do not support dockerHostPathPrefix or split-filesystem daemons; mount policy is never bypassed');
}
Comment on lines +253 to +256
// No in-memory ownership means neither labels nor the absence of a journal
// proves that a legacy or interrupted invocation has released its mounts.
if (runId) {
throw new Error('Docker enclave recovery requires trusted reconciliation before cleanup');
@lpcox

lpcox commented Oct 20, 2026

Copy link
Copy Markdown
Collaborator

@copilot address review feedback

Copilot AI commented Oct 20, 2026

Copy link
Copy Markdown
Contributor Author

@copilot address review feedback

Addressed the review feedback in 751307a. The 288 focused regression tests, build, type-check, and lint pass.

@lpcox

lpcox commented Oct 20, 2026

Copy link
Copy Markdown
Collaborator

@copilot merge main and resolve conflicts

@github-actions

github-actions Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

🛡️ Smoke Copilot Network Isolation confirmed the egress allowlist is enforced. ✅

🛡️ Egress verdict from Smoke Copilot Network Isolation

@github-actions

github-actions Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

✅ Smoke Copilot BYOK AOAI (Entra) completed. Copilot AOAI BYOK (Entra) mode operational. 🔓

🪪 BYOK (AOAI Entra) report filed by Smoke Copilot BYOK AOAI (Entra)

@github-actions

github-actions Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Chroot tests passed! Smoke Chroot - All security and functionality tests succeeded.

Tested by Smoke Chroot

@github-actions

github-actions Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

✅ Smoke Copilot BYOK AOAI (api-key) completed. Copilot AOAI BYOK (api-key) mode operational. 🔓

🔑 BYOK (AOAI api-key) report filed by Smoke Copilot BYOK AOAI (api-key)

@github-actions

github-actions Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Smoke Cloud Hypervisor completed. Cloud Hypervisor + Copilot passed.

Cloud Hypervisor + Copilot smoke test by Smoke Cloud Hypervisor

@github-actions

Copy link
Copy Markdown
Contributor

Smoke Copilot: ✅ GitHub MCP (latest PR: "feat(router): add opt-in OpenTelemetry traces and metrics") · ✅ github.com 200 · ✅ File write/read
Overall: PASS — @Copilot (author), assignees @lpcox @Copilot

📰 BREAKING: Report filed by Smoke Copilot
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

Smoke Test: Copilot BYOK (Direct) Mode ✅

Test Results:

  • ✅ GitHub MCP connectivity (PR data retrieved)
  • ✅ github.com connectivity (HTTP 200)
  • ✅ File write/read test (smoke test file verified)
  • ✅ BYOK inference working (response generated via api-proxy → api.githubcopilot.com)

Mode: Direct BYOK (COPILOT_PROVIDER_API_KEY) via api-proxy sidecar

Overall Status: PASS

🔑 BYOK report filed by Smoke Copilot BYOK
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

Smoke services: ✅ Redis PONG, ✅ pg_isready, ✅ SELECT 1 — PASS

🔌 Service connectivity validated by Smoke Services
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

Smoke Test: Claude Engine Validation

  • API status: ✅ PASS
  • GitHub check: ✅ PASS
  • File status: ✅ PASS

Overall result: PASS

Generated by Smoke Claude for #10353 · claude · haiku45 · 34.1 AIC · ⊞ 6.1K · ◷
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

EGRESS_RESULT allow=pass deny=pass

✅ Allowed domain (api.github.com → 200)
✅ Blocked domain (example.com blocked)

Overall: PASS

🛡️ Egress verdict from Smoke Copilot Network Isolation
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

@Copilot @lpcox
Smoke Test BYOK Direct Mode — Azure OpenAI (Foundry, api-key)

  • GitHub MCP Testing: ❌
  • GitHub.com Connectivity: ✅
  • File Write/Read Test: ✅
  • BYOK Inference Test: ✅
    Running in direct BYOK mode (COPILOT_PROVIDER_API_KEY + COPILOT_PROVIDER_BASE_URL) via api-proxy → Azure OpenAI (Foundry, o4-mini-aw)
    Overall: FAIL

🔑 BYOK (AOAI api-key) report filed by Smoke Copilot BYOK AOAI (api-key)
Add label ready-for-aw to run again

@github-actions github-actions Bot added the smoke-copilot-network-isolation Copilot network-isolation egress smoke test label Oct 20, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Cloud Hypervisor + Copilot smoke test

  1. Merged PR lookup: PASS
  2. github.com connectivity: PASS (HTTP 200)
  3. File write/read: PASS
  4. example.com blocked: PASS (HTTP 000)
    Overall: PASS

Cloud Hypervisor + Copilot smoke test by Smoke Cloud Hypervisor
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

Smoke test:

  • ci: replace retired gpt-5.4-mini with gpt-5.6-luna in test-model-routing ✅
  • feat(router): ship the Go router image and remove the Python implementation ✅
  • Playwright title check ❌
  • File write/read ✅
  • Build (npm ci && npm run build) ✅
    Overall: FAIL

🔮 The oracle has spoken through Smoke Codex
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

OTEL smoke test

  • ✅ S1 Module load: otel.js exports startRequestSpan, setTokenAttributes, setBudgetAttributes, endSpan, endSpanError, shutdown, isEnabled (+ internals)
  • ✅ S2 Tests: 3 suites, 68/68 passed
  • ✅ S3 Env forwarding: trace/parent span IDs in env-passthrough.ts; OTEL endpoint/headers/trace vars in api-proxy-env-config.ts
  • ✅ S4 Token tracker: onUsage callback present in token-tracker-http.js
  • ✅ S5 Diagnostics: /tmp/gh-aw/otel.jsonl exists (1 line); no errors

📡 OTel tracing validated by Smoke OTel Tracing
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

Chroot Version Comparison

Runtime Host Version Chroot Version Match?
Python Python 3.12.15 Python 3.12.15 ✅
Node.js v24.22.0 v2.98.0 ❌
Go go1.22.12 go1.22.12 ✅

Not all tests passed: the Node.js version differs between host and chroot, so the smoke-chroot label was not added.

Tested by Smoke Chroot
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

🏗️ Build Test Suite Results

Ecosystem Project Build/Install Tests Status
Bun elysia ❌ 0/0 ❌ FAIL
Bun hono ❌ 0/0 ❌ FAIL
C++ fmt ✅ N/A ✅ PASS
C++ json ✅ N/A ✅ PASS
Deno oak N/A 0/0 ❌ FAIL
Deno std N/A 0/0 ❌ FAIL
.NET hello-world ✅ N/A ✅ PASS
.NET json-parse ✅ N/A ✅ PASS
Go color ✅ pass ✅ PASS
Go env ✅ pass ✅ PASS
Go uuid ✅ pass ✅ PASS
Java gson ❌ 0/0 ❌ FAIL
Java caffeine ❌ 0/0 ❌ FAIL
Node.js clsx ✅ all passed ✅ PASS
Node.js execa ✅ all passed ✅ PASS
Node.js p-limit ✅ all passed ✅ PASS
Rust fd ✅ 1/1 passed ✅ PASS
Rust zoxide ✅ 1/1 passed ✅ PASS

Overall: 5/8 ecosystems passed — FAIL

Failure details

  • Bun / Deno: The install scripts (curl ... | bash / | sh) were denied by the sandbox permission policy, so Bun and Deno were never installed and no tests ran.
  • Java: Maven failed with LocalRepositoryNotAccessibleException for the default ~/.m2 repo. After I redirected the local repo to a writable path, it failed with PluginResolutionException. The proxy configured in settings.xml was squid-proxy:3128, and plugins could not be downloaded through it.
  • .NET: The first runs crashed on a missing /run/user/<uid> directory. They passed once I set XDG_RUNTIME_DIR and DOTNET_CLI_HOME to writable paths. This is an environment quirk, not a project failure.

The build-test label was not added because some ecosystems failed.

Generated by Build Test Suite for #10353 · copilot · auto · 24.7 AIC · ⊞ 12K · ◷
Add label ready-for-aw to run again

@lpcox
lpcox deployed to aoai-model October 20, 2026 23:40 — with GitHub Actions Active
@github-actions

Copy link
Copy Markdown
Contributor

ci: replace retired gpt-5.4-mini with gpt-5.6-luna in test-model-routing - ✅
feat(router): ship the Go router image and remove the Python implementation - ✅
GitHub.com connectivity (HTTP 200) - ✅
File write/read test - ✅
Direct BYOK inference test - ✅
Running in direct BYOK mode (AWF_AUTH_TYPE=github-oidc + AWF_AUTH_AZURE_* + COPILOT_PROVIDER_BASE_URL) via api-proxy → Azure OpenAI (Foundry, o4-mini-aw) authenticated via Microsoft Entra
Overall status: PASS
cc @Copilot @lpcox

🪪 BYOK (AOAI Entra) report filed by Smoke Copilot BYOK AOAI (Entra)
Add label ready-for-aw to run again

@lpcox
lpcox deployed to aoai-model October 20, 2026 23:41 — with GitHub Actions Active
@lpcox
lpcox disabled auto-merge October 20, 2026 23:53
@lpcox
lpcox merged commit f1857c3 into main Oct 20, 2026
150 of 153 checks passed
@lpcox
lpcox deleted the copilot/migrate-docker-enclaves branch October 20, 2026 23:53

This branch was successfully deployed

1 active deployment
aoai-model — 9ac63e38 Deployed Oct 20, 2026 by lpcox via conclusion #2089
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Migrate Docker enclaves to the shared host-owned execution lifecycle

4 participants