Skip to content

feat(runtime): one native Postgres server per run, a database and an owner per service - #196

Merged
antoinetoussaint-byte merged 7 commits into
mainfrom
feat/one-native-server-per-run
Oct 27, 2026
Merged

antoinetoussaint-byte merged 7 commits into
mainfrom
feat/one-native-server-per-run

Conversation

@antoinetoussaint-byte

@antoinetoussaint-byte antoinetoussaint-byte commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

What

On the native (Nix) runtime, every Postgres service of a run now shares one Postgres server, with a database and a migration owner per service. Before, each service started its own postmaster. A composed platform run started about 8 to 11 of them, each holding a System V shared-memory segment (macOS allows 32 on the whole machine) and a cluster directory of its own. That is the cause of repeated could not create shared memory segment: No space left on device and the disk exhaustion.

The run is the workspace plus its naming scope, which is the identity Codefly already uses to isolate concurrent runs. Services do not change: each keeps its endpoint, its connection-string keys, its migrations and its runtime logins.

Design (all in this agent; no CLI or core change)

  • One server per run (nixpg.go). The first service of the run to initialize starts it through pg_ctl, detached, so no single agent's lifetime decides the server's. It listens on no TCP port, only on a private Unix socket. Its bootstrap superuser (codefly_admin) has a password that is generated at initdb and kept in an owner-only file under the run's root. No service ever receives it.
  • A service's endpoint (forward.go). Each service's agent listens on the service's own allocated port (on loopback, as before) and forwards every connection to the socket. SCRAM authentication stays between the client and the server. Addresses in every connection string are unchanged, and so is the CLI's TCP readiness gate.
  • A database and an owner per service (runtime_native.go, provisionDatabase). The owner is the database's own role, codefly_<db>_<digest>_owner, named the same way as the existing _ro/_rw roles. Every local service ships the same POSTGRES_USER and roles are server-wide, so the configured name could not be used. The password is still POSTGRES_PASSWORD, and the owner connection string carries the role. The owner has CREATEROLE CREATEDB NOSUPERUSER. Each database has REVOKE ALL … FROM PUBLIC, and so do postgres and template1. Two services of one run that declare the same database-name are refused, and the refusal names both.
  • Isolation matches a managed deployment. Migrations and runtime-access reconciliation run as the non-superuser owner, as managed-bootstrap does. Only extensions are created by the server administrator, because pgvector is not a trusted extension.
  • Lifecycle (holders/). Each agent records itself and the codefly run that launched it. Stop and Destroy release that service, and the last release stops the server (fast shutdown, so its shared memory is freed). A run that is killed leaves holders that are dead or orphaned (their run is gone). The next agent to initialize anywhere on the machine stops that server. The cluster directory is retained per run (core#424), as each service's directory was before.
  • Only a recorded postmaster is ever signalled (postmaster.json). The postmaster's birth — PID plus start and boot ids — is written when the server starts and re-confirmed immediately before every signal. A postmaster.pid is never a reason to signal anything: it outlives the process that wrote it (SIGKILL, OOM, a machine that lost power), PIDs are reused, and collection runs over every other run root on the machine at every native start. A lock file whose live PID is provably not a postgres server is cleared, so a reused number cannot wedge a start either; one a real postmaster holds is refused by name rather than removed, because removing it would let a second postmaster onto the same data directory.
  • An owner role another service created is refused. A migration owner may create roles, because migrations create the application's own NOLOGIN roles, and PostgreSQL 16 and later grant a role created that way back to its creator WITH ADMIN OPTION. Adopting an existing codefly_<db>_<digest>_owner would therefore hand its creator the database. An owner role carrying any pg_auth_members row is refused, and a database is adopted only when its current owner is the expected role (its claim comment alone is forgeable by anything with CREATEDB).
  • Fix in ensureLoginRole. It restated NOSUPERUSER NOREPLICATION NOBYPASSRLS, which only a superuser may do. It now clears only the elevated attributes a role actually holds, the same way libs/go/controlplane.ensureGroupRole already does for managed servers.

Docker runtime: unchanged (one container per service).

What is reused and what is not

Reused: Codefly's naming scope and workspace identity, the allocated network mappings (no new port is allocated), configuration and secrets delivery, the existing role engine (runtimeRoleNames, ReconcileRuntimeAccess), core's InspectProcess for PID-reuse-safe liveness, gofrs/flock (already in the graph through core), the embedded flake, pg_ctl, and the host IPC sweep.

Not reused: #194 (local-server / shared-local-server). It shares a server only when each service declares a dependency on a server-owning service and picks a distinct POSTGRES_USER. That changes every composition, and its migration owners are superusers, so it is not isolated. This PR does the same job with no declaration and supersedes it. Recommend closing #194 after review.

Classification: fix at the owner of the native Postgres runtime.

Behaviour changes to know

  • The owner connection's user is the derived role, not POSTGRES_USER. Applications use the reader and writer keys, so their DSNs keep their roles.
  • A run's first start on this version creates its databases empty and migrates them. Old per-service clusters under ~/Library/Caches/codefly/postgres/<hash> are neither migrated nor deleted.
  • Postgres server logs go to <run root>/server.log, not to the service's log stream. A failed start includes the log's tail in the refusal.
  • The run server's postmaster is a child of launchd, not of an agent. A manual orphan sweep that judges "parent is pid 1" would wrongly treat it as leaked. Its owners are the records in holders/.
  • A run whose postmaster.json is lost while its postmaster still serves refuses to start and says which PID holds the cluster, rather than starting a second postmaster or signalling the first. Stopping that postmaster is a deliberate act, by hand.

One PR per repo

This branch also carries #170 (dev-tag release publishing), merged in under the owner's one-PR-per-repo rule rather than left open beside this one. It is disjoint from the shared server: it repins core's release workflow to v0.16.0 — which contains main's older September pin, so the SBOM-capable publisher is kept — and routes a codefly publish dev tag to a local read-only build plus an upload-only publisher, so the tag stops hitting core's default-branch reachability gate. Both merge conflicts were additive (AGENTS.md's release-flow line, workflow_test.go's job fields). Its own tests pass in the merged tree. Follow-up for whoever lands it: core's release workflow moved again on 2026-10-19, so the v0.16.0 pin is itself slightly behind.

#194 (the older local-server / shared-local-server approach) is superseded by this PR, as the reuse section above says.

Evidence

  • Unit tests: forwarder (bytes both ways; Close ends sessions and frees the port; an endpoint already in use is refused), holder pruning (dead agent, dead run, reused PID, garbage), server options (no TCP; shell metacharacters refused), owner naming, run-key separation.
  • Real Nix Postgres (macOS): TestRunServerIsSharedAndIsolated covers two services on one postmaster, a non-superuser owner, and owner and reader both denied on the other database and on postgres/template1. It also checks that releasing one service keeps the other serving, that the last release stops the postmaster, and that data survives a restart. Also: TestRunServerRefusesTwoServicesClaimingOneDatabase, TestAbandonedRunServerIsCollected, and the existing TestCreateToRunNix and TestLifecycleContractNix against the shared server. The test suite keeps its run servers under its own temp dir and fails loudly if one outlives it.
  • Signalling, with no nix needed: a live sleep whose PID sits in an abandoned root's postmaster.pid survives both stopServer and the machine-wide collectAbandonedServers (this is the reviewer's reproduction, which killed it in 101 ms before the fix); a recorded birth whose PID has been handed on is not signalled and its record is dropped; the same stale file does not wedge a start, because the lock file is cleared once the process is shown not to be a postgres server. Against a real postmaster (nix): a missing or unreadable record makes the run refuse by name, leaving the lock file in place and the postmaster unsignalled.
  • Role squatting (nix): a service that pre-creates another's owner role makes that service's Init refuse, naming it, and the refused service holds no server.
  • CI's unit command passes locally, in full, including TestBootstrapImageMatchesLocalRuntimeSchema (which the reviewer saw fail only under local Docker contention).
  • Robin playground (module-robin ba7bff28, make playground, scope shpg7, CLI built from cli main 3ba96cd, this agent at be6d0b5 installed as 0.0.146-dev.be6d0b58a21e through the workspace agent-overrides). All 8 Postgres services ran on 1 postmaster holding 1 SysV segment, where a stock robin playground run holds 8 and 8. The seed reported ready: every step holds. Dev signed in. A Grounded chat answered with citations from autumn-field-notes.md. After Ctrl-C, no run postmaster, holder, socket directory or shared-memory segment remained. Host totals went back to their earlier value (16 → 17 → 16).

Not verified: Linux native (CI has no Nix, so those tests skip there), and the platform-obin composition.

🤖 Generated with Claude Code

antoinetoussaint-byte and others added 4 commits October 8, 2026 17:47
…owner per service

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… out

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
antoinetoussaint-byte and others added 3 commits October 27, 2026 11:24
A postmaster.pid outlives the process that wrote it, PIDs are reused, and
collection sweeps every other run root on the machine at every native
start, so trusting the number in that file made an unrelated process of
this user the target of a SIGINT. The postmaster's birth is now recorded
when the server starts and re-confirmed before any signal; a lock file
naming a live process that is provably not a postgres server is cleared
instead of waited on, and one a real postmaster holds is refused by name
rather than removed.

Also refuse an owner role another service created: PostgreSQL 16 grants a
created role back to its creator WITH ADMIN OPTION, so adopting one would
hand that creator the database.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The owner's one-PR-per-repo rule. #170 is disjoint from the shared native
server — it repins core's release workflow to v0.16.0 (over main's older
September pin, which it contains) and routes dev tags to a local
read-only build plus an upload-only publisher, so a `codefly publish dev`
tag stops hitting core's default-branch reachability gate. Both conflicts
were additive: AGENTS.md's release-flow line beside the run-server rows,
and workflow_test.go's job fields beside the shared-server assertions.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant