Goal
Deploy the bootc-operator to an existing arm64 Kubernetes cluster (in our case one provisioned by CAPI) using the documented install path, and have both the controller and the daemon come up.
Problem
The published images are linux/amd64 only. ghcr.io/bootc-dev/bootc-operator:latest is a single application/vnd.docker.distribution.manifest.v2+json with "architecture": "amd64", with no manifest list, so there is no arm64 instance for a node to select.
That blocks both workloads, since config/manager/manager.yaml and config/daemon/daemon.yaml both reference the same ghcr.io/bootc-dev/bootc-operator:latest:
- the controller Deployment cannot be scheduled onto an arm64 node, and
- the daemon DaemonSet cannot run on any managed arm64 node at all. This one is structural rather than incidental: the daemon executes
bootc on the host via nsenter resolved from its own filesystem (internal/bootc/executor.go), so the container has to match the node's architecture. There is no way to run an amd64 daemon against an arm64 host.
So on an all-arm64 cluster the operator is simply not installable today, and on a mixed-arch cluster the daemon would be missing from exactly the nodes it is meant to manage.
Why this should be straightforward
Nothing in the operator looks architecture specific:
cmd/controller and cmd/daemon are both pure Go with no cgo dependencies, so CGO_ENABLED=0 GOARCH=arm64 go build already works on both.
- The
quay.io/fedora/fedora-minimal:44 base is already published for linux/arm64 (as well as ppc64le and s390x), and provides nsenter there.
This looks like build and publish plumbing rather than a code change.
Also worth deciding
config/manager/manager.yaml still carries the kubebuilder-scaffolded nodeAffinity block commented out, listing amd64/arm64/ppc64le/s390x and referring to a docker-buildx target that does not exist in this repo. Once the image is a manifest list, kubelet picks the right instance on its own and no arch constraint is needed, so that block could either be dropped or narrowed to kubernetes.io/os: linux. Happy to follow whichever the maintainers prefer.
Out of scope
- e2e coverage on arm64, and arm64 node disk images in bink — worth its own issue, since it depends on runner infrastructure rather than on this repo.
- Multi-arch node images.
internal/registry/resolver.go resolves a tag with remote.Get, which returns the index digest for a manifest list, while internal/daemon/reconciler.go compares that against the digest bootc status reports for the booted image. Every image in play today is a single manifest so the two cannot disagree, but that may need platform-aware resolution later.
Goal
Deploy the bootc-operator to an existing arm64 Kubernetes cluster (in our case one provisioned by CAPI) using the documented install path, and have both the controller and the daemon come up.
Problem
The published images are
linux/amd64only.ghcr.io/bootc-dev/bootc-operator:latestis a singleapplication/vnd.docker.distribution.manifest.v2+jsonwith"architecture": "amd64", with no manifest list, so there is no arm64 instance for a node to select.That blocks both workloads, since
config/manager/manager.yamlandconfig/daemon/daemon.yamlboth reference the sameghcr.io/bootc-dev/bootc-operator:latest:bootcon the host viansenterresolved from its own filesystem (internal/bootc/executor.go), so the container has to match the node's architecture. There is no way to run an amd64 daemon against an arm64 host.So on an all-arm64 cluster the operator is simply not installable today, and on a mixed-arch cluster the daemon would be missing from exactly the nodes it is meant to manage.
Why this should be straightforward
Nothing in the operator looks architecture specific:
cmd/controllerandcmd/daemonare both pure Go with no cgo dependencies, soCGO_ENABLED=0 GOARCH=arm64 go buildalready works on both.quay.io/fedora/fedora-minimal:44base is already published forlinux/arm64(as well as ppc64le and s390x), and providesnsenterthere.This looks like build and publish plumbing rather than a code change.
Also worth deciding
config/manager/manager.yamlstill carries the kubebuilder-scaffoldednodeAffinityblock commented out, listingamd64/arm64/ppc64le/s390xand referring to adocker-buildxtarget that does not exist in this repo. Once the image is a manifest list, kubelet picks the right instance on its own and no arch constraint is needed, so that block could either be dropped or narrowed tokubernetes.io/os: linux. Happy to follow whichever the maintainers prefer.Out of scope
internal/registry/resolver.goresolves a tag withremote.Get, which returns the index digest for a manifest list, whileinternal/daemon/reconciler.gocompares that against the digestbootc statusreports for the booted image. Every image in play today is a single manifest so the two cannot disagree, but that may need platform-aware resolution later.