Skip to content

sec(windows): tighten per-job dir ACLs, fail-closed egress, restrict control-pipe/socket SDs#112

Merged
luthermonson merged 3 commits into
mainfrom
sec/windows-hardening
Jul 13, 2026
Merged

sec(windows): tighten per-job dir ACLs, fail-closed egress, restrict control-pipe/socket SDs#112
luthermonson merged 3 commits into
mainfrom
sec/windows-hardening

Conversation

@ephpm-claude

@ephpm-claude ephpm-claude Bot commented Jul 12, 2026

Copy link
Copy Markdown
Contributor

⚠️ UNVERIFIED — no live Windows host; every change needs testing on a real Windows runner before merge.

This PR was authored on a macOS machine with no Windows runtime available. Every change is:

  • cross-compile-clean: GOOS=windows GOARCH=amd64 go build ./...
  • GOOS=windows go vet clean on all touched packages (and the full tree)
  • unit-tested for the pure string/SDDL/SID/policy construction

…but none of the runtime behaviour was exercised: icacls effects, whether the Hyper-V utility VM can still open the VSMB-mounted runner dir under the new principal, HCN egress-ACL enforcement, winio pipe SD acceptance, and the AF_UNIX-socket ACL on Windows all require a live Windows runner to confirm. Do not merge or deploy without validating on real Windows hardware. Worst-case failure mode of the tighter grants is a job that fails to start (fail-closed), not one that silently runs unprotected.

Closes the High/Medium/Low Windows job-runtime findings from the security audit. Each finding was re-verified against current main (4c050f3) before changing; the audit's WIN-3 dind claim was already fixed on main and is not touched here.


WIN-1 (High) — per-job runner dir ACLs — pkg/runtime/grant_windows.go

Finding: per-job runner dirs were granted Everyone:(OI)(CI)Modify and the runners parent Everyone:(RX) — scoped by path, not principal, so any local user or sibling job could read every job's checkout, artifacts, and (transiently) JIT runner config.

Fix: grant the Hyper-V VM group SID S-1-5-83-0 (NT VIRTUAL MACHINE\Virtual Machines) directly on each per-job dir instead of Everyone. This is the exact principal hcsshim's own internal/security.GrantVmGroupAccess uses to prepare host paths for VSMB (verified in the vendored hcsshim@v0.14.0-rc.1: sidVMGroup = "S-1-5-83-0", granted directly on the target — it never relies on inheritance). The old comment's premise ("the VM group SID doesn't reliably inherit, so we must use Everyone") conflated inheritance with direct grant: a direct per-path ACE for the VM group is precisely how hcsshim solves it. The runners parent is now locked to SYSTEM + Administrators + VM-group with inheritance broken (icacls /inheritance:r) so the world-traversable C:\ProgramData ACL no longer leaks read access.

Principal granted & confidence: S-1-5-83-0. Reasonably confident it's correct — it matches hcsshim's documented technique for the identical VSMB scenario, and the existing pkg/vm/linuxvm_windows.go:grantVmFileAccess comment already names this SID as the intended one. Cannot verify without a Windows host that the utility VM's virtual account actually resolves as a VM-group member for this container config; if a live test shows the VM cannot open the dir, the documented fallback is to grant the exact per-VM virtual-account SID derived from the container GUID. The change is a strict reduction from Everyone in all cases.

Residual risk: a concurrent job's utility VM (a distinct virtual account, still a VM-group member) could in principle open another job's dir — the same trust boundary hcsshim itself uses for VSMB. Eliminating that would require per-container virtual-account SIDs, which need a live host to derive and test.

WIN-4 (Med) — container-egress firewall fail-closed — pkg/networking/network_windows.go, pkg/runtime/runtime.go

Finding: per-endpoint egress ACLs (RFC1918 + link-local block) were applied best-effort and Warn-and-continued on failure (fail-open), with no global backstop, so any ACL hiccup started a container with unrestricted egress to the host LAN / other RFC1918 services / 169.254.169.254 metadata.

Fix: fail closed. setup() now deletes the endpoint and returns an error if ACL application fails; applyACLPoliciesbuildEgressBlockPolicies returns an error on any marshal failure or an empty rule set (instead of silently continue-ing to a weaker set). Critically, runtime.Create now aborts the job on Network.Setup error rather than the previous warn-and-continue — otherwise the fail-closed signal was swallowed and the container would still start. Worst case is now a job that fails to start, never one that runs unfirewalled.

Residual risk: still per-endpoint, no global WFP/HCN default-deny backstop (the HCN NAT API here doesn't expose a network-wide default-deny that I could add without live experimentation). The 10.0.0.0/8 block vs. the on-subnet dind gateway 10.88.0.1 longest-prefix question from the audit is [UNVERIFIED] and unchanged by this PR — needs a live host to confirm dind reachability and that metadata is truly blocked.

WIN-5 (Med) — containerd control-pipe security descriptor — pkg/containerd/listen_windows.go

Finding: winio.ListenPipe(address, nil) → go-winio's default (historically permissive, version-dependent) SD on a SYSTEM-backed full containerd control API pipe (\\.\pipe\ephemerd-containerd).

Fix: pass an explicit protected DACL D:P(A;;GA;;;SY)(A;;GA;;;BA) (GENERIC_ALL to SYSTEM + Administrators only, no inherited ACEs) via winio.PipeConfig{SecurityDescriptor: ...}. Confirmed against vendored go-winio@v0.6.2: PipeConfig.SecurityDescriptor string (SDDL) is the correct field, and the SDDL is validated in a unit test via winio.SddlToSecurityDescriptor. Mirrors the already-correct HvSocket bind SD (WIN-7 in the audit).

Residual risk: none beyond needing a live host to confirm the pipe still accepts the daemon's own connections under the new SD (SYSTEM is granted, so it should).

WIN-6 (Low) — control socket ACL — pkg/scheduler/grpc.go (+ socketperm_{windows,other}.go)

Finding: the control gRPC socket (KillJob DoS, GetJobLogs secret exposure) is an AF_UNIX socket secured with os.Chmod(0o600), which is a near-no-op for a socket file on Windows — access falls back to the world-traversable C:\ProgramData NTFS ACL.

Fix: split secureControlSocket by platform. POSIX keeps chmod 0600. Windows applies a real NTFS ACL (icacls /inheritance:r + full control to SYSTEM + Administrators only). Startup now fails if the ACL can't be applied.

Residual risk: AF_UNIX-socket-file ACL semantics on Windows are [UNVERIFIED] here; if icacls on the socket file proves ineffective on the target build, the follow-up is to switch the Windows control transport to a named pipe with the same SDDL as WIN-5 (noted for the live-test pass).


Deferred (NOT in this PR)

  • WIN-2 — run the service under a low-privilege virtual account (NT SERVICE\ephemerd) instead of LocalSystem. This is a larger operational change: the account then needs explicit grants to the data dir, the GitHub App private key, and every path/HCS/HCN/Hyper-V operation the daemon touches. Getting it wrong bricks the daemon, and it is impossible to validate here without a Windows host. Flagged as a separate, carefully-tested follow-up.

Tests

  • pkg/runtime/grant_windows_test.go — asserts the granted SIDs (VM-group/SYSTEM/Admins) and that Everyone never creeps back.
  • pkg/containerd/listen_windows_test.go — locks the pipe SDDL and proves it parses via winio.SddlToSecurityDescriptor.
  • pkg/networking/network_windows_test.go — proves every RFC1918/link-local range becomes an outbound Block rule (fail-closed).
  • pkg/scheduler/socketperm_windows_test.go — asserts the control-socket SIDs.

All new tests are //go:build windows-tagged since the constants/helpers are Windows-only; they compile-check clean under GOOS=windows but were not run — no Windows host.

🤖 Generated with Claude Code

https://claude.ai/code

Live Windows validation — 2026-07-13 (windows.tricorder.cc)

The header's "UNVERIFIED" caveat is now discharged. Validation performed on the production Windows fleet host:

Unit tests (first-ever execution of the //go:build windows suites, on real Windows): pkg/runtime, pkg/networking, pkg/scheduler, pkg/containerd — all pass. The WIN-5 SDDL parse test ran against the real Windows API.

Live deployment (branch build v0.1.0-3-gab92c6e deployed to the production daemon):

  • WIN-1: live per-job dir shows NT VIRTUAL MACHINE\Virtual Machines:(OI)(CI)(M) + SYSTEM/Admins inherited — Everyone gone — and jobs run, so the utility VM opens the VSMB dir under the S-1-5-83-0 grant. The documented fallback (per-VM virtual-account SID) was not needed.
  • WIN-4: full ephpm E2E rerun (3 legs, Kind-in-dind — heaviest Docker networking in the fleet) green under fail-closed egress. The 10.0.0.0/8-block vs 10.88.0.1 dind-gateway question is resolved empirically: dind connectivity intact.
  • WIN-5: daemon + all jobs functioned through the containerd control pipe under the new protected SDDL.
  • WIN-6: icacls ephemerd.sock → SYSTEM + Administrators only, inheritance stripped; control operations functional.

Load: 4-platform smoke test (all green) + 3× ephpm E2E (all green) on the branch daemon.

Branch updated with a merge of current main (v0.1.0 + #114/#115/#116/#117); affected-package tests re-run post-merge.

luthermonson and others added 2 commits July 12, 2026 14:21
…control-pipe/socket SDs

UNVERIFIED — authored on macOS with no live Windows host. Every change below
is cross-compile-clean (GOOS=windows GOARCH=amd64) and unit-tested for pure
string/SDDL construction, but the runtime behaviour (icacls effects, HCN ACL
enforcement, winio pipe SD, AF_UNIX ACL on Windows) MUST be validated on a real
Windows runner before merge/deploy.

WIN-1 (per-job runner dir ACLs): grant the Hyper-V VM group (S-1-5-83-0) —
the same principal hcsshim's GrantVmGroupAccess uses — directly on each per-job
dir instead of Everyone (S-1-1-0). Lock the runners parent to SYSTEM +
Administrators + VM-group and break inheritance from world-traversable
ProgramData. Closes cross-job / local-user reads of checkouts, artifacts, JIT
config.

WIN-4 (container egress firewall): make egress-ACL application FATAL. setup()
now tears down the endpoint and returns an error on ACL failure, and Create()
aborts the job on Setup error instead of warn-and-continue. buildEgressBlockPolicies
fails closed on marshal error or an empty rule set. No more fail-open path to
the host LAN / RFC1918 / link-local metadata.

WIN-5 (containerd control pipe SD): pass an explicit protected DACL
(D:P(A;;GA;;;SY)(A;;GA;;;BA)) to winio.ListenPipe instead of nil, restricting
the SYSTEM-backed containerd control API to SYSTEM + Administrators.

WIN-6 (control socket ACL): replace the no-op os.Chmod(0600) on Windows with a
real NTFS ACL (SYSTEM + Administrators, inheritance broken) via a
platform-split secureControlSocket helper; POSIX keeps chmod 0600.

Deferred: WIN-2 (run service under NT SERVICE\ephemerd virtual account instead
of LocalSystem) — larger operational change, needs live testing.
…kdown

Review follow-up to the WIN-1 hardening. grantHyperVTraverse ran
`icacls /inheritance:r` on the runners parent and re-granted SYSTEM only
(RX). But the daemon runs as SYSTEM and creates each per-job dir
(`job-<id>`) directly under that parent via copyDirForJob at Create(),
and removes orphan `job-*` dirs there at startup. Creating a subdirectory
needs write (FILE_ADD_SUBDIRECTORY) on the parent; breaking inheritance
also dropped the inherited SYSTEM full-control ACE. Net: every Windows job
needing a runner mount would fail to provision (fail-closed, but total).

Grant SYSTEM + Administrators inheritable Full (OI)(CI)F on the parent so
the daemon can create/clean per-job dirs; the VM group stays traverse-only
(RX, non-inheritable) — each job dir still gets its own explicit VM-group
Modify ACE. Extracted the icacls arg construction into pure helpers
(traverseGrantArgs/modifyGrantArgs) and added tests that pin the policy
(SYSTEM/Admins keep inheritable write; VM group traverse-only; no Everyone)
so the RX regression can't recur without a live Windows host to catch it.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ChVRC9ZrvQarrctDwSy1S5
@luthermonson
luthermonson force-pushed the sec/windows-hardening branch from 775f921 to d558987 Compare July 12, 2026 21:23
@luthermonson
luthermonson merged commit 2b1156a into main Jul 13, 2026
4 checks passed
@luthermonson
luthermonson deleted the sec/windows-hardening branch July 13, 2026 23:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant