Skip to content

feat(policy-log): the witness starts with no warehouse mirror, and locks every entry under its delivered retention (#1154) - #1155

Closed
antoinetoussaint-byte wants to merge 5 commits into
mainfrom
policy-log-no-mirror
Closed

antoinetoussaint-byte wants to merge 5 commits into
mainfrom
policy-log-no-mirror

Conversation

@antoinetoussaint-byte

@antoinetoussaint-byte antoinetoussaint-byte commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor

Closes #1154

What we are trying to do

Let the policy-log witness run on a hosted cell before any warehouse front exists, so the host can get a policy-log transport and stop refusing every narrowing of authority. And make the evidence it writes immutable in the way its README requires: "with retention and IAM, in addition to application checks".

The decision

Owner, 2026-10-18, verbatim: "YES LET'S IMPROVE ALL THIS". That was the answer to: "should the witness be allowed to start with no mirror configured, saying so at boot, the way accounts does?" It reverses the 2026-10-17 call that policy-log stays in CreateContainerConfigError until a warehouse front exists.

#1133, the mirror speaking the warehouse front, stays open. It is no longer on the start path.

Changes

  • resolveSettings. The bucket is always required. POLICY_LOG_WAREHOUSE_ENDPOINT and POLICY_LOG_WAREHOUSE_TOKEN are optional, but only together; one without the other is refused as a half-provisioned group, naming the missing key. Every missing key is still named in one refusal, and there is still no process-environment fallback and no value in any refusal.
  • startMirror. A whole mirror builds the writer and starts the bounded worker, as before; a writer refusal (for example a non-https endpoint) still refuses the boot. With no mirror configured it wires mirror.None and prints one line naming both settings: the object store is the authority, and Read is what a warehouse added later backfills from. mirror.None is a named value rather than a nil, so "no mirror" is something the wiring states. service.New still refuses a nil mirror.
  • Settings are read with WorkspaceValueIfSet. A value nothing carries is absent. A value the carrier could not read is refused by name, so a broken delivery of both mirror settings cannot boot claiming no mirror is configured.
  • Locked retention on every entry. POLICY_LOG_RETENTION_DAYS is required with the bucket: a whole number of days, bounded in days before it is multiplied, and at most ten years. Every entries/ object is created under a Locked GCS object retention until now plus that horizon. Nobody, a project owner included, can delete or replace it before then. The head carries none, because it is rewritten on every append. A bucket without object retention enabled rejects the write, so it refuses every append rather than accepting evidence it cannot protect. This is the retention half of the README's requirement; the infrastructure's half is IAM:
    • the witness identity may create under entries/ and set retention only there;
    • it may replace only head.json;
    • it holds no delete under entries/.
  • README. It covers the boot order, the configuration rules, the "Warehouse mirror" section, the IAM scope the lock needs, the fact that enabling object retention cannot be undone, the 400/403 symptoms of a misprovisioned bucket, and how an entry orphaned by a lost head commit is unstuck (retry the identical entry).
  • Local defaults. The shipped secret default leaves the optional token out instead of shipping a placeholder that would read as half a mirror, and the local policy-log.env declares POLICY_LOG_RETENTION_DAYS.

Fix or hack: a fix, at the owner of the behaviour. The witness decides what it needs to witness, and its authority never depended on the mirror.

Reused, not rebuilt: the existing mirror.Worker, mirror.NewWriter and service.Mirror interface are unchanged. mirror.None is a five-line implementation of that same interface. No second mirror path was added.

Testing / Validation

  • go build ./..., go vet, gofmt -l (clean) and go test -count=1 ./... all pass in module/services/policy-log/code.
  • node module/tools/naming-gate.mjs check passes.
  • TestEveryDeclaredGroupShipsALocalDefault passes; it caught the empty token default.
  • New tests:
    • TestABucketWithNoMirrorIsAStartableWitness: a bucket with no mirror resolves, with no mirror.
    • TestNoMirrorIsStatedByNameAtBoot: wires mirror.None and prints exactly one line naming both settings.
    • TestAConfiguredMirrorIsBuiltAndStaysSilent: a whole mirror starts the worker and prints nothing; a writer refusal still refuses the boot.
    • TestResolveSettingsNamesEveryMissingKeyAtOnce now covers "no bucket and half a mirror" in one refusal.
    • TestAnUnreadableMirrorIsRefusedNotReadAsAbsent, TestTheEntryRetentionIsAWholeBoundedNumberOfDays (including 213504 and the int64 maximum), TestEntriesAreLockedForTheDeclaredRetentionAndTheHeadIsNot (the lock is asserted on the wire), and TestTheRetentionIsRequiredAndBounded.

Independent security review

  • Round 1 (f9991ba): MERGE WITH FOLLOW-UPS. Fixed: an unreadable setting was read as "no mirror"; the README still called the token mandatory; the local token placeholder.
  • Round 2 (de0e477): MERGE WITH FOLLOW-UPS. Fixed:
    • g1, a day count that overflowed into a 25-minute lock;
    • g2, the setRetention IAM scope (documented; the infrastructure grants it only under entries/);
    • g3, the misprovisioning symptoms (documented);
    • g4, the orphaned-slot procedure (documented).
  • Owner question raised by the review: there is no retention minimum, so 1 is accepted.

Not verified: the binary has not booted against real GCS here. That needs the dedicated evidence bucket and its head.json, which the infrastructure side provisions. The 400/403 behaviour, and how the IAM condition applies to an upload that carries a retention, are read from Google's documentation and have not been exercised.

🤖 Generated with Claude Code

antoinetoussaint-byte and others added 5 commits October 18, 2026 12:48
…red, and says so at boot (#1154)

Owner decision 2026-10-18, reversing the 2026-10-17 call that the witness
stays down until a warehouse front exists: the object store is the witness's
authority and the warehouse only mirrors what it already committed, so a
missing mirror must not keep the evidence from being written.

The bucket and the signing key stay required. The mirror's endpoint and token
are optional together. If both are delivered, the bounded worker starts as
before. If neither is, the witness wires mirror.None (a named value, never a
nil a check could absorb) and prints one boot line naming both settings. If
only one is delivered, the group is refused as half-provisioned. There is
still no ambient fallback, and no value appears in a refusal.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rror"; docs and local config say the mirror is optional (#1154)

Review of f9991ba:

- resolveSettings now reads through WorkspaceValueIfSet. A value nothing
  carries is absent; a value the carrier could not read is refused by name.
  Before, a broken delivery of both mirror settings booted with
  "no warehouse mirror is configured", which was false.
- The README's trust-domain section no longer calls the warehouse credential
  mandatory: it is required only where a mirror is configured.
- The local secret file no longer ships a warehouse-token placeholder. With
  only the bucket filled in, a local run would otherwise have been refused
  for a missing endpoint, pointing at a warehouse it does not need.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ion for the delivered horizon (#1154)

The README requires the immutable prefix to be enforced "with retention and
IAM, in addition to application checks". IAM is the infrastructure's half:
the witness identity can create entries and never delete them. This commit
is the retention half. Each entries/ object is written with a LOCKED object
retention for POLICY_LOG_RETENTION_DAYS, so nobody can delete or replace it
before then, a project owner included. The head carries no retention,
because it is rewritten on every append.

POLICY_LOG_RETENTION_DAYS is required with the bucket. It must be a whole
number of days, at most ten years, and anything else is refused by name
rather than rounded, because a Locked retention can never be shortened. A
bucket without object retention enabled rejects the write, so a store that
cannot hold the lock refuses every append.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…document the setRetention scope and the lock's operating facts (#1154)

Review of de0e477:

- POLICY_LOG_RETENTION_DAYS=213504 overflowed time.Duration into a
  25-minute retention that was accepted. The day count is now bounded before
  it is multiplied, and both that value and the int64 maximum are in the
  test.
- The README names the IAM scope the lock needs: storage.objects.setRetention
  only under entries/, never bucket-wide or through objectAdmin, because a
  lock on head.json would stop the log forever. It also says enabling object
  retention on a bucket cannot be undone, that a misprovisioned bucket shows
  up as a 400 or 403 on every append rather than at boot, and how an entry
  orphaned by a lost head commit is unstuck: retry the identical entry.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e token instead of shipping it empty (#1154)

TestEveryDeclaredGroupShipsALocalDefault refuses an empty value in a shipped
secret default, because the file is published verbatim. The token is optional
and this local composition configures no mirror, so the key is left out and
the file says why.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@antoinetoussaint-byte antoinetoussaint-byte changed the title feat(policy-log): the witness starts with no warehouse mirror configured, and says so at boot (#1154) feat(policy-log): the witness starts with no warehouse mirror, and locks every entry under its delivered retention (#1154) Oct 18, 2026
@antoinetoussaint-byte

Copy link
Copy Markdown
Contributor Author

Superseded by #1257 (3c53033): the policy-log witness service is deleted and accounts' narrowings now apply directly through the shared authority core.

auto-merge was automatically disabled October 27, 2026 13:39

Pull request was closed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

policy-log starts with no warehouse mirror configured, and says so at boot

1 participant