Repository navigation
feat(runtime): one native Postgres server per run, a database and an owner per service - #196
Merged
Merged
Conversation
…owner per service Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… out Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A postmaster.pid outlives the process that wrote it, PIDs are reused, and collection sweeps every other run root on the machine at every native start, so trusting the number in that file made an unrelated process of this user the target of a SIGINT. The postmaster's birth is now recorded when the server starts and re-confirmed before any signal; a lock file naming a live process that is provably not a postgres server is cleared instead of waited on, and one a real postmaster holds is refused by name rather than removed. Also refuse an owner role another service created: PostgreSQL 16 grants a created role back to its creator WITH ADMIN OPTION, so adopting one would hand that creator the database. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The owner's one-PR-per-repo rule. #170 is disjoint from the shared native server — it repins core's release workflow to v0.16.0 (over main's older September pin, which it contains) and routes dev tags to a local read-only build plus an upload-only publisher, so a `codefly publish dev` tag stops hitting core's default-branch reachability gate. Both conflicts were additive: AGENTS.md's release-flow line beside the run-server rows, and workflow_test.go's job fields beside the shared-server assertions. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
On the native (Nix) runtime, every Postgres service of a run now shares one Postgres server, with a database and a migration owner per service. Before, each service started its own postmaster. A composed platform run started about 8 to 11 of them, each holding a System V shared-memory segment (macOS allows 32 on the whole machine) and a cluster directory of its own. That is the cause of repeated
could not create shared memory segment: No space left on deviceand the disk exhaustion.The run is the workspace plus its naming scope, which is the identity Codefly already uses to isolate concurrent runs. Services do not change: each keeps its endpoint, its connection-string keys, its migrations and its runtime logins.
Design (all in this agent; no CLI or core change)
nixpg.go). The first service of the run to initialize starts it throughpg_ctl, detached, so no single agent's lifetime decides the server's. It listens on no TCP port, only on a private Unix socket. Its bootstrap superuser (codefly_admin) has a password that is generated at initdb and kept in an owner-only file under the run's root. No service ever receives it.forward.go). Each service's agent listens on the service's own allocated port (on loopback, as before) and forwards every connection to the socket. SCRAM authentication stays between the client and the server. Addresses in every connection string are unchanged, and so is the CLI's TCP readiness gate.runtime_native.go,provisionDatabase). The owner is the database's own role,codefly_<db>_<digest>_owner, named the same way as the existing_ro/_rwroles. Every local service ships the samePOSTGRES_USERand roles are server-wide, so the configured name could not be used. The password is stillPOSTGRES_PASSWORD, and the owner connection string carries the role. The owner hasCREATEROLE CREATEDB NOSUPERUSER. Each database hasREVOKE ALL … FROM PUBLIC, and so dopostgresandtemplate1. Two services of one run that declare the samedatabase-nameare refused, and the refusal names both.managed-bootstrapdoes. Only extensions are created by the server administrator, because pgvector is not a trusted extension.holders/). Each agent records itself and thecodefly runthat launched it. Stop and Destroy release that service, and the last release stops the server (fast shutdown, so its shared memory is freed). A run that is killed leaves holders that are dead or orphaned (their run is gone). The next agent to initialize anywhere on the machine stops that server. The cluster directory is retained per run (core#424), as each service's directory was before.postmaster.json). The postmaster's birth — PID plus start and boot ids — is written when the server starts and re-confirmed immediately before every signal. Apostmaster.pidis never a reason to signal anything: it outlives the process that wrote it (SIGKILL, OOM, a machine that lost power), PIDs are reused, and collection runs over every other run root on the machine at every native start. A lock file whose live PID is provably not a postgres server is cleared, so a reused number cannot wedge a start either; one a real postmaster holds is refused by name rather than removed, because removing it would let a second postmaster onto the same data directory.codefly_<db>_<digest>_ownerwould therefore hand its creator the database. An owner role carrying anypg_auth_membersrow is refused, and a database is adopted only when its current owner is the expected role (its claim comment alone is forgeable by anything with CREATEDB).ensureLoginRole. It restatedNOSUPERUSER NOREPLICATION NOBYPASSRLS, which only a superuser may do. It now clears only the elevated attributes a role actually holds, the same waylibs/go/controlplane.ensureGroupRolealready does for managed servers.Docker runtime: unchanged (one container per service).
What is reused and what is not
Reused: Codefly's naming scope and workspace identity, the allocated network mappings (no new port is allocated), configuration and secrets delivery, the existing role engine (
runtimeRoleNames,ReconcileRuntimeAccess), core'sInspectProcessfor PID-reuse-safe liveness,gofrs/flock(already in the graph through core), the embedded flake,pg_ctl, and the host IPC sweep.Not reused: #194 (
local-server/shared-local-server). It shares a server only when each service declares a dependency on a server-owning service and picks a distinctPOSTGRES_USER. That changes every composition, and its migration owners are superusers, so it is not isolated. This PR does the same job with no declaration and supersedes it. Recommend closing #194 after review.Classification: fix at the owner of the native Postgres runtime.
Behaviour changes to know
POSTGRES_USER. Applications use the reader and writer keys, so their DSNs keep their roles.~/Library/Caches/codefly/postgres/<hash>are neither migrated nor deleted.<run root>/server.log, not to the service's log stream. A failed start includes the log's tail in the refusal.holders/.postmaster.jsonis lost while its postmaster still serves refuses to start and says which PID holds the cluster, rather than starting a second postmaster or signalling the first. Stopping that postmaster is a deliberate act, by hand.One PR per repo
This branch also carries #170 (dev-tag release publishing), merged in under the owner's one-PR-per-repo rule rather than left open beside this one. It is disjoint from the shared server: it repins core's release workflow to v0.16.0 — which contains main's older September pin, so the SBOM-capable publisher is kept — and routes a
codefly publish devtag to a local read-only build plus an upload-only publisher, so the tag stops hitting core's default-branch reachability gate. Both merge conflicts were additive (AGENTS.md's release-flow line, workflow_test.go's job fields). Its own tests pass in the merged tree. Follow-up for whoever lands it: core's release workflow moved again on 2026-10-19, so the v0.16.0 pin is itself slightly behind.#194 (the older
local-server/shared-local-serverapproach) is superseded by this PR, as the reuse section above says.Evidence
TestRunServerIsSharedAndIsolatedcovers two services on one postmaster, a non-superuser owner, and owner and reader both denied on the other database and onpostgres/template1. It also checks that releasing one service keeps the other serving, that the last release stops the postmaster, and that data survives a restart. Also:TestRunServerRefusesTwoServicesClaimingOneDatabase,TestAbandonedRunServerIsCollected, and the existingTestCreateToRunNixandTestLifecycleContractNixagainst the shared server. The test suite keeps its run servers under its own temp dir and fails loudly if one outlives it.sleepwhose PID sits in an abandoned root'spostmaster.pidsurvives bothstopServerand the machine-widecollectAbandonedServers(this is the reviewer's reproduction, which killed it in 101 ms before the fix); a recorded birth whose PID has been handed on is not signalled and its record is dropped; the same stale file does not wedge a start, because the lock file is cleared once the process is shown not to be a postgres server. Against a real postmaster (nix): a missing or unreadable record makes the run refuse by name, leaving the lock file in place and the postmaster unsignalled.TestBootstrapImageMatchesLocalRuntimeSchema(which the reviewer saw fail only under local Docker contention).ba7bff28,make playground, scopeshpg7, CLI built from cli main3ba96cd, this agent atbe6d0b5installed as0.0.146-dev.be6d0b58a21ethrough the workspaceagent-overrides). All 8 Postgres services ran on 1 postmaster holding 1 SysV segment, where a stock robin playground run holds 8 and 8. The seed reportedready: every step holds. Dev signed in. A Grounded chat answered with citations fromautumn-field-notes.md. After Ctrl-C, no run postmaster, holder, socket directory or shared-memory segment remained. Host totals went back to their earlier value (16 → 17 → 16).Not verified: Linux native (CI has no Nix, so those tests skip there), and the platform-obin composition.
🤖 Generated with Claude Code