Skip to content

build: publish multi-tenancy platform images #467

Closed

Description

@stakach

Build the coordinated multi-tenancy platform image set for a real-cluster test, as authorised by Steve on 2026-11-18 (Australia/Sydney). Parent: #465. Platform issue: PlaceOS/PlaceOS#34.

Owner: stakach (Codex coordinator). Status: Done — all13 images published and verified. Start date: 2026-11-18.

  • Refresh expected source branches, dependency installs and current green CI heads.
  • Create an isolated ai/ release manifest branch/worktree in PlaceOS/PlaceOS, based on nightly; retain all unrelated service pins.
  • Pin models716ca76, REST1507d3a, Staffa9b8a21 and init2a986f8 after confirming current upstream heads.
  • Set VERSION to multi-tenancy; reuse the canonical service discovery, Git source contexts, Buildx amd64/arm64 matrix, build args and Docker Hub image names.
  • Make the requested build images-only: skip Git tag/release rewriting, notifications, UI dispatches and normal release channels.
  • Validate workflow and resolved image/source manifest; save a draft PR with exact build/verification plan before execution.
  • Publish all requested multi-tenancy multi-platform manifests through an authorised registry path. Respect the configured release environment review and branch protections; do not change/bypass them silently.
  • Verify every image has linux/amd64 and linux/arm64, record immutable digests and source revisions, and inspect packaged init migrations/API versions.
  • Record results and the test-cluster pull configuration. Cluster migration/deployment will be a subsequent task.

Current publication constraint: local Docker has no registry login. The canonical Docker Hub release environment requires release-team review and accepts only nightly/monthly/preview branches. Prepare a concrete reviewable custom build first; resolve registry/auth or the specific release gate with Steve if required. Do not weaken production release protections.

Staff stays domain-based, driver/interface source remains unchanged, and tenancy enforcement is not activated by this image-build task. Models is a source/library/migration pin, not a standalone platform runtime service image.

Prepared build review

Draft manifest PR: PlaceOS/PlaceOS#376 at cca78ae53610853ac3550d488931736644746dc6, on ai/multi-tenancy-images. Independent read-only review found no actionable issues. actionlint and execution of the actual platform-info/version/tag scripts pass. All13 committed service gitlinks match multi-tenancy-images.json; every generated tag is exactly placeos/:multi-tenancy, with amd64/arm64 and the protected release environment retained. SHA-pinned discovery and two independent images-only guards prevent release/tag/chat/UI side effects. Feature branches/dependencies remain clean and green.

No protection or registry changes have been made. Awaiting Steve's explicit approval to temporarily allow exactly this branch in the existing release environment and approve only its images-only run; remove the temporary rule afterward. No images are claimed built or published yet.

Latest platform validation: Shard Report CI passed on cca78ae5. No environment branch rules or approvals have been changed.

Approved publication

Steve explicitly approved this test-image build in chat. Created only temporary release branch rule67207274 for ai/multi-tenancy-images, retaining the existing release-team review policy. Dispatched the approved immutable commit cca78ae5 with images_only=true: https://github.com/PlaceOS/PlaceOS/actions/runs/44216582470 . Approval and verification apply only to this run; unrelated nightly runs remain untouched. Remove rule67207274 after completion/verification.

Final verification

Published all13 Docker Hub multi-tenancy tags for amd64+arm64. Original run44216582470 built12 images; init-only recovery44225358298 succeeded with max-parallelism1 while retaining approved source cca78ae5 and original init/model pins. Both runs skipped Tag and release.

All13 tag/index digests,26 platform configs and source/version labels verified. REST/Staff version smoke checks pass on both architectures. All149 init database files match reviewed models716ca76 on both architectures; pg_dump18.7 starts. No database migration ran. Temporary branch rule67207274 was deleted, and original release protections were verified unchanged.

Receipts and immutable pull references are committed in PlaceOS/PlaceOS#376, which remains a draft test manifest. Current branch is clean/pushed; source feature PRs remain unmerged. Known pre-existing REST platform-info display issue tracked separately in #468. Cluster migration/deployment is the next task, not an unfinished image-build step.

Activity

  1. added this to the Multi-tenancy milestone on Nov 18, 2026
  2. self-assigned this
    on Nov 18, 2026
  3. stakach commented on Nov 18, 2026

    @stakach
    MemberAuthor

    Status: In progress. Steve explicitly approved the scoped test-image publication. The approved images-only run44216582470 is running at immutable cca78ae5: discovery passed and all13 expected service build jobs are active, each configured for linux/amd64 and linux/arm64.

    Added only release branch rule67207274 for ai/multi-tenancy-images and approved only this run under the existing required release-team review. Unrelated nightly runs and all other environment settings are untouched. The temporary rule will be removed after publication and verification, or cleaned up if the build is abandoned.

    No cluster migration or tenancy enforcement activation is part of this build. Root owns build handling, registry/payload verification and temporary-rule cleanup; the regression coordinator monitors the run read-only.

  4. stakach commented on Nov 18, 2026

    @stakach
    MemberAuthor

    Status: In progress. The first run built10 services successfully so far; only REST/Staff are still running. Init failed because BuildKit ran its amd64 task compiler concurrently with the arm64 main compiler, exhausting runner memory. Crystal already has --threads=1 in all init compile commands.

    Steve asked to improve memory usage and retry only the failed image after the others finish. Added execution-only controls: BuildKit worker parallelism1 for init, an images-only single-service selector, and an immutable source_ref that drives both source checkout and PLACE_COMMIT. The recovery will still build exactly approved source cca78ae5, init2a986f8 and models716ca76, with the same tag/architectures/safeguards; no service source or manifest pins change.

    Actual shell tests pass for source freezing, unchanged defaults, invalid overrides and exact init selection. actionlint, TOML parsing and independent read-only review pass. The retry will start only once the two remaining jobs finish, per Steve's experience. Existing release-team review remains in force; only the approved test-image publication is authorised.

  5. stakach commented on Nov 18, 2026

    @stakach
    MemberAuthor

    Status: In progress. Original run44216582470 completed with12 service successes; only init failed from memory exhaustion. Its Tag and release job was explicitly skipped.

    After all other images finished, started the init-only recovery44225358298, using execution workflow9d7e11dd with images_only=true, source_ref=cca78ae53610853ac3550d488931736644746dc6 and service=init. Discovery confirms only init is selected, and the sanitized runner log confirms the original approved source SHA. Both checkout and PLACE_COMMIT retain that SHA. Existing release review remains in force; no new branch rule or broader permission change is needed.

    Init's BuildKit worker parallelism is now1 in addition to its existing Crystal --threads=1. This changes execution memory scheduling, not service source or image content inputs. REST and Staff published binaries have already passed isolated --version smoke checks on amd64 and arm64 with networking disabled. Final all-image manifest/label checks, init payload verification and deletion of temporary rule67207274 follow recovery completion.

  6. stakach commented on Nov 18, 2026

    @stakach
    MemberAuthor

    Status: Done. All13 Docker Hub images are published under multi-tenancy with verified amd64 and arm64 manifests and source/version labels. Build manifest/receipts include immutable pull references.

    Original run44216582470 built12 images; init-only recovery44225358298 succeeded with one concurrent BuildKit operation, preserving approved source cca78ae5. Crystal workers were already limited to1. Both runs skipped release/tagging/notification steps.

    REST/Staff binaries run on both architectures. Both init images contain149 database files byte-matching models716ca76, and pg_dump18.7 starts. Temporary release branch permission67207274 has been removed; original protections remain intact. No cluster migration ran. Source PRs remain unmerged and drivers/interfaces are unchanged.

    Ready for the separate test-cluster migration task. Existing platform-info display-label issue is tracked in #468; Docker tags and OCI version labels are verified correct.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions