Skip to content

feat: select Prometheus with ControlPlane observability.prometheus.mode - #953

Draft
boddumanohar wants to merge 27 commits into
prometheus-operator-migrationfrom
feat/prometheus-mode
Draft

boddumanohar wants to merge 27 commits into
prometheus-operator-migrationfrom
feat/prometheus-mode

Conversation

@boddumanohar

@boddumanohar boddumanohar commented Nov 20, 2026 •

Copy link
Copy Markdown
Member

Stacked on #692. Replaces the prometheus.enabled Helm value with a ControlPlane field, so OperatorHub and Helm installs choose the Prometheus the same way.

API

spec.source.local.observability.prometheus:

prometheus:
  mode: Managed      # Managed | External, default Managed
  external:          # only valid with External
    labels: {release: kube-prometheus-stack}

There is no mode without a Prometheus.

Operator

Managed External
Prometheus operator's simplyblock-prometheus none
Trusted scraper (SB_K8S_METRICS_SERVICE_ACCOUNTS) simplyblock-prometheus simplyblock-metrics-scraper (operator-owned, token Secret)
Objects static scrape config ServiceMonitor ×2, ScrapeConfig, PodMonitor ×2, PrometheusRule
  • Switching modes deletes what the other mode applied, only objects this ControlPlane owns. The Prometheus PVC stays.
  • External mode without the monitoring.coreos.com CRDs fails with an error naming prometheus-operator.
  • The 11 alert rules are in controlplane/manifests/alert-rules.yaml.
  • Removes the ServiceMonitor detection (prometheusScrapesTheManagementAPI) and SB_EXTRA_METRICS_SERVICE_ACCOUNTS.

Chart

  • prometheus.enabled, prometheus.serviceAccount, prometheus.additionalLabels, prometheus.interval, and prometheus.simplyblock.* are replaced by prometheus.mode and prometheus.external.{labels,url}.
  • kube-prometheus-stack.enabled (default false) is the subchart condition. validate-prometheus.yaml rejects it outside External mode, and keeps the CRD conflict check.
  • prometheus-operator-servicemonitors.yaml is deleted. The operator creates those objects now.
  • prometheus-operator-alerting.yaml keeps only the AlertmanagerConfig and its Secret, rendered in External mode.
  • The Control Center and --prometheus-url follow the mode. metricsAPI.prometheusURL unset now means "follow the mode", and "" still means off.
  • Thanos Query gets its --store back for the managed Prometheus's sidecar in Managed mode.

Review findings on #692 this addresses

  1. Managed installs lost scrape access: simplyblock-prometheus was dropped from the trusted scrapers.
  2. The Control Center pointed at prometheus-operated with no prometheus-operator installed.
  3. ScrapeConfig used authorization.credentialsFile. The ScrapeConfig CRD in kube-prometheus-stack 92.2.0 has only credentials and type.
  4. An existing operator-managed Prometheus kept running after a prometheus-operator took over. Also, withoutPrometheus dropped simplyblock-prometheus-client-tls, which the mutual-TLS monitors need.

Testing

  • prometheus_mode_test.go: 12 tests.
    • Ten ran red against the unchanged code, for the reason each describes, then green.
    • TestControlPlaneCELPrometheusMode was not run red: the API type was added before it.
    • TestExternalScrapeConfigAuthenticatesWithASecret was not run red because my -run filter missed it. The old code built no ScrapeConfig at all.
  • go test ./internal/... (with envtest) and make lint pass.
  • helm template checked:
    • Managed mode.
    • External mode, with and without kube-prometheus-stack.enabled, labels, and a Slack receiver.
    • The two rejected combinations.
  • helm-charts/scripts/test-prometheus-values.sh (new, in the Helm lint workflow): red on the --prometheus-url case below, then green.

Live test on cluster .61 (k3s, TLS off, metrics API off), 2026-11-20

Managed → External (bundled kube-prometheus-stack, Slack receiver) → Managed, image feat-prometheus-mode:

Step Result
Managed Release deployed; SB_K8S_METRICS_SERVICE_ACCOUNTS = simplyblock-prometheus; simplyblock_metrics and fdb_metrics targets up
→ External Managed Prometheus StatefulSet, Services, ConfigMap, and RBAC deleted; scraper account and token Secret created; management API trusts simplyblock-metrics-scraper
kube-prometheus-stack Prometheus targets up: ScrapeConfig (token from the Secret), webappapi ×2, fdb-exporter, spdk-proxy ×3
PrometheusRule loaded (11 rules), real alerts firing; AlertmanagerConfig loaded as the simplyblock-alerts route
→ Managed All monitors, the scraper account, and its Secret deleted; managed Prometheus back on its existing PVC, both targets up

Found and fixed live: with metricsAPI.enabled=false the chart passed no --prometheus-url. The capacity controllers fell back to simplyblock-prometheus, which External mode had just deleted (bd1d028).

Found, not caused by this PR:

  • Every operator rollout deadlocks on .61. The webhook certificate rotator runs only on the leader, the old pod keeps the lease, and the new pod never turns ready. Deleting the old pod clears it.
  • helm upgrade does not install a subchart's CRDs, so enabling kube-prometheus-stack on an existing release needs the monitoring.coreos.com CRDs applied by hand (--server-side) and kube-prometheus-stack.crds.enabled=false.

Known gaps

  • OperatorHub installs: they get rules but no Alertmanager routing. The notification receivers are still Helm values and need a ControlPlane field.
  • Grafana in External mode: it reads only uploaded Thanos blocks, because the external Prometheus has no sidecar.
  • RootFilesystemLowSpace: it needs node-exporter, which the bundled kube-prometheus-stack keeps off.
  • Storage-node PodMonitors: they select pods in the control plane's namespace only.
  • The two rule copies: nothing checks that the Grafana alert rules and alert-rules.yaml stay in step.

wmousa and others added 27 commits November 18, 2026 15:09
…de agent (#734)

* feat(storage-node): state Talos to the node agent, with a talosctl image slot

spec.environment: Talos resolved only into the kubelet flag; the node agent
itself was never told the host is Talos and kept discovering it by sniffing
the kernel version, after which it skipped the huge-page reservation and
left the SPDK pod Pending on a short pool. sbcli #1491 makes a stated Talos
node reserve its pages through the Talos API (a talosctl Job patching
machine.sysfs and restarting the kubelet); this is the operator half.

* StorageNodesSpec.talos (*bool), set by the expansion from the Talos
  environment the way OpenShift is, reaches the agent as TALOS_CLUSTER
  beside OPENSHIFT_CLUSTER / UBUNTU_HOST.
* spec.images.talosctl on the deployment config lands on
  StorageNodesSpec.talosctlImage and reaches the agent as TALOSCTL_IMAGE
  when stated, so a deployment pins the Job image to its Talos minor (or
  a mirror) instead of taking the agent's default.

Generated CRDs follow in the next commit.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* chore: regenerate CRDs, deepcopy and installer for the Talos fields

make manifests generate build-installer + helm-charts/scripts/sync-from-operator.sh.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* style: name the true/false strings the agent environment is built from (goconst)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix(api): the talosctl slot admits any registry

ImageSpec holds every image to simplyblock's own registries, which is right
for software simplyblock publishes and wrong for talosctl: Sidero publishes
it on ghcr.io and a mirroring site names its own registry. Found on the
first Talos deployment, where the draft rejected
ghcr.io/siderolabs/talosctl:v1.12.7 against ImageSpec's pattern.
ExternalImageSpec keeps the reference-shape check and drops the registry
allow-list; the talosctl slot uses it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* chore: regenerate deepcopy, CRDs and installer for ExternalImageSpec

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* feat(rbac): let the storage-node agent read Job pod logs and check for a Secret

Two read-only rules on simplyblock-storage-node-role, both wanted by the
Talos huge-page flow and seen as HTTP 403 in the agent log on the first
Talos run:

* pods/log get: the agent copies a finished talosctl Job's pod log into
  its own log before deleting the Job, the only record of what happened
  on the host side.
* secrets get: the pre-flight check that the Talos API credentials Secret
  exists before a Job is created to mount it; without it a missing Secret
  is a Job pod stuck in ContainerCreating for the whole Job timeout.

The operator's own role already holds both, so it can grant them.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* style: gofmt and an unshadowed loop variable (lint)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* feat(talos): enable the kubelet configuration on Talos and let the agent read configz

With sbcli configuring the kubelet's CPU topology on Talos through
machine.kubelet.extraConfig (the Talos API), the Talos environment no
longer resolves to "skip the kubelet configuration": it is enabled as on
every other distribution, and the node chooses the Talos path by itself.

The storage-node ClusterRole gains nodes/proxy get: the agent reads the
kubelet's live configuration (nodes/<name>/proxy/configz) before patching,
so a kubelet that already runs with the settings is left alone -- Talos
restarts the kubelet on every machine-config change, and a restart per
SPDK restart would be a regression.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* feat(talos): state who writes the machine config, pre-provisioned unless Managed

A Talos deployment so far implied that the node may patch a worker's
machine config through the Talos API, which made an os:admin Secret in
the namespace a requirement of every Talos cluster and left an operator
who prepares the machine config themselves with the wrong error on a
short pool. The question is now stated:

* ClusterTemplate.talosMachineConfig on the deployment config, valid only
  with environment Talos (CEL), lands on
  StorageNodesSpec.talosMachineConfig: Preprovisioned (the default, left
  unstated when the document is) or Managed.
* The agent reads it as TALOS_MACHINE_CONFIG, lowercase, set whenever
  Talos is stated so the DaemonSet shows what the node will do: with
  preprovisioned it verifies the pool and the kubelet settings and fails
  the add with the patch to apply; with managed it runs the talosctl
  Jobs (sbcli #1491).

Generated CRDs, chart and installer follow in the next commit.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* chore: regenerate CRDs, chart and installer for talosMachineConfig

controller-gen, the conversion-webhook stanza, the embedded CRD copies,
the installer and the chart's CRDs, from the types as they are after the
rebase: the talosMachineConfig field on the cluster template and on
storageNodes with its enum and the environment rule, and the talosctl
image slot in its ExternalImageSpec form.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* feat(talos): a Talos deployment turns the CPU topology on, as OpenShift does

The Talos case enabled the kubelet configuration but left the CPU
topology unset, i.e. off, and a Talos document cannot state it (the
environment decides both). The kubelet patch through the Talos API is
what that configuration is for, so the topology is on as on OpenShift.
In the pre-provisioned mode a kubelet without the settings refuses the
add with the extraConfig patch to apply; in managed mode the node
applies it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* chore: regenerate CRDs, installer and chart for the stated kubelet and CPU topology flags on Talos

The flag descriptions said Talos leaves both flags off; a Talos deployment
turns both on (the kubelet half through the Talos API), and a document may
now state either one off.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* test(support-tool): a document states the Talos machine-config mode only for a Talos fleet

The support tool's admission check carries the same rule as the CRD:
spec.cluster.talosMachineConfig is refused unless spec.environment is
Talos, and accepted there.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* Live consistency-group subsystem flip on late label join

* fix(cg-join): prepare each host with its own host NQN

* fix(swap): drop the source's dead paths before resuming on the target

* feat(cg): move a volume between consistency groups on relabel

* ran make operator-build-installer

* fix(operator): fresh step message and honest colocation count

* test(operator): drop the ineffectual ops assignment in the step-message test

* fix(volstack): grow the dmLinear mapping when a volume is expanded

* feat(driver): add spec.disableDeviceMapperIndirection to SimplyblockDriver

* feat(csi-node): turn the dm-linear indirection off on hosts without device-mapper

* fix(cg-flip): hold every consumer of a moving namespace, and nothing else touches it

* fix(cg-flip): a move that happened is committed, never rolled back after the window

* fix(cg-watcher): a member outside its group's subsystem gets the join again

* updated design doc design-cg-live-subsystem-flip.md

* fix(cg-join): wait out a held plan in Migrating, and plan again before the child

* fix(csi): clear the lint the main merge introduced

* fix(driver): close the test the main merge left open
…t OpenShift's (#919)

* fix(numa): the NUMA device plugin registers with a kubelet that is not OpenShift's

The plugin registers by connecting to the kubelet's socket, and SELinux
decides that by the type the kubelet runs as. The manifest's
container_device_plugin_t may connect to OpenShift's kubelet (kubelet_t).
RKE2 runs its kubelet as container_runtime_t, which that type may not
connect to:

  avc: denied { connectto } comm="numa-resource-p"
    path="/var/lib/kubelet/device-plugins/kubelet.sock"
    scontext=...:container_device_plugin_t tcontext=...:container_runtime_t

The plugin logged "failed to connect to kubelet: context deadline
exceeded", exited after 45 s and crash-looped; the node advertised no
numa-align, so no storage pod with CPU topology was scheduled there
(RKE2 1.36 on Rocky 9.5, SELinux enforcing, 2026-11-13).

Off OpenShift the plugin now runs as spc_t. OpenShift keeps
container_device_plugin_t. Which one the cluster is comes from the API
groups it serves, read once at start; a cluster that cannot be asked stops
the operator rather than being guessed at. The plugin stays unprivileged,
with every capability dropped and a read-only root filesystem.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

* fix(csi-chart): the chart's NUMA device plugin takes the same SELinux type as the operator's

The spdk-csi chart carries its own copy of the plugin's DaemonSet with the
fixed container_device_plugin_t, which cannot register with a kubelet that
does not run as OpenShift's does. It now renders container_device_plugin_t
on a cluster that serves config.openshift.io/v1 ClusterVersion, the test
validate-tls.yaml already uses for OpenShift, and spc_t elsewhere.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

* fix(csi-chart): node-rbac.yaml renders one document separator between its objects

The template ended its ServiceAccount block with a separator and began the
next block with another, and the whitespace trimming around the two
conditions joined them into one line of six dashes. helm lint failed the
chart on it ("invalid Yaml document separator"). The blocks now carry one
separator each and keep their line breaks. The chart renders the same 29
objects as before, with either condition off as well.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

* fix(chart): the control plane's node selector value is rendered as a string

controlplane.nodeSelector.value was rendered bare in the operator
Deployment, the three observability Deployments and the log collector's
affinity, so a value that YAML reads as something else than a string
became one: with the label node-role.kubernetes.io/control-plane=true,
which k3s sets on its servers, helm upgrade failed on the operator
Deployment with "cannot unmarshal bool into Go struct field
PodSpec.spec.template.spec.nodeSelector of type string". The ControlPlane
the chart bootstraps already quoted it, so that half of the upgrade had
gone through and the release was left failed with the selector applied to
the control plane and not to the operator.

The value is quoted in all five places. A chart that sets no selector
renders as before.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

* fix(inventory): an RKE2 cluster is concluded to be Rancher's, not K3s

RKE2 is built on the K3s code and serves the k3s.cattle.io API group, for
its add-ons and its etcd snapshot records. The group was read as K3s
outright, and K3s precedes Rancher when a cluster carries both, so the
discovery wrote `environment: K3s` for an RKE2 cluster (RKE2
v1.36.6+rke2r1, 2026-11-13).

The precedence exists for a Rancher-managed K3s cluster, and its rule is
that the distribution that installed the kubelet decides. That rule is now
applied to the pair: where the kubelets carry the Rancher family's build
suffix and none carries K3s's, a K3s conclusion becomes Rancher. A cluster
with a K3s kubelet, with kubelets of both, or with no kubelet version to
read keeps the answer it had. The K3s finding stays in the evidence.

The two environments expand to the same storage node settings today, so
no deployment changes; the draft now names the distribution the cluster
is.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
…ers (#932)

* fix(ci): run the CSI unit and integration suites on Ubuntu 26.04 runners

ubuntu-latest moved to 26.04 between two runs on main (7983a98 passed,
1c4fb97 failed), which brings two changes the tests assumed away.

- coreutils comes from uutils, where sleep is a hard link to one
  multicall binary that picks the utility from its own name. The
  nfsexport tests copied sleep as rpc.idmapd and nfsdcld to fake a
  running daemon. The copy refused to run and exited, so the "already
  running" tests lost a race on every run. The fake is now a copy of the
  test binary, which sleeps when NFSEXPORT_FAKE_PROCESS is set.
- sudo is sudo-rs, which ignores -E and warns on stderr. The integration
  harness ran talosctl under sudo -E for HOME and KUBECONFIG. talosctl
  ran with root's HOME, and the warning landed in the JSON the harness
  parses, so no suite brought its cluster up. Both variables are now
  passed by name through env, in the harness and in the workflow's
  diagnostics and cleanup steps.

Verified on an ubuntu:26.04 container: TestAFakeProcessKeepsRunning is
red before and green after, and the nfsexport package passes three runs
under -race -cover. The sudo change is unit-tested on the arguments
only; the integration job on this PR is its live check.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(ci): set talosctl's HOME and KUBECONFIG as sudo arguments, not through env

Running talosctl through `sudo env` changed the command sudo authorizes
from talosctl to env, which the scoped SETENV grant in the integration
README does not cover. An unattended developer host would have been
prompted, while the GitHub runner's NOPASSWD: ALL hid it. The variables
are now sudo's own VAR=value arguments, which that grant covers.

Checked on ubuntu:26.04 with sudo-rs 0.2.13: VAR=value works under both
NOPASSWD: ALL and a scoped SETENV grant, and the scoped grant refuses
the env form. README and workflow steps updated to match.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Exported from simplyblock/sbcli@9699508.

Co-authored-by: noctarius <1142801+noctarius@users.noreply.github.com>
…esystem (#936)

* fix(devmapper): re-point a dm-linear mapping without freezing the filesystem

The volstack heal swaps a volume's dm-linear table onto the namespace the
fabric reconnected after total path loss. Swap suspended with dmsetup's
default lockfs, which freezes the filesystem on top, and that freeze has
to write to the namespace that is gone: ext4's freeze failed with EIO, so
NodePublish never restaged the volume, and XFS's blocked in
xfs_ail_push_all_sync while holding the superblock lock, so every sync(2)
on the node hung behind it.

Swap now suspends with --nolockfs, as the live subsystem flip already
does. In-flight I/O still drains before the reload.

Seen in the lblk e2e run 44463031746: both RECONNECT-FULLLOSS specs, both
RECONNECT-GUARDIAN specs, and the PNFS two-writer spec.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(e2e): read the filesystem type with df -P so a long device name stays on one line

busybox df puts a device name longer than 20 characters on a line of its
own, so `awk 'NR==2 {print $2}'` read an empty field once volumes were
staged behind /dev/mapper/sb-<uuid>. -P keeps each mount on one line, as
the size check in utils.go already does.

Seen in the lblk e2e run 44463031746: SPDKCSI-FILESYSTEM XFS and ext4.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…937)

* ci(security): accept dotted prerelease tags such as v26.4.0-beta6.1

The 26.4 line now resolves to v26.4.0-beta6.1, which the release-tag
shape check rejected, failing the scheduled scan on main.

* build(operator): bump go.opentelemetry.io/otel to v1.47.0 (GHSA-q7gw-24fr-5p69)
… the unwind (SFAM-2824) (#933)

discardMigration returned on any CancelMigration error. A failed migration
is what fails the operation, and the control plane answers its cancel with
a 400 "is not active", so the unwind stopped on the ordinary failure path:
the validation Jobs stayed, no release reached the hosts that connected the
target paths, and status.migration kept every connection. An abort, a
withdrawal, or an abandonment hit the same refusal on every pass and never
finished, holding the volume lock.

The refusal wording now lives in the atlas-lib client rather than in the
operator. CancelMigration names the "not active" answer of both a volume's
migration and a subsystem's group ErrMigrationNotActive; CreateMigration
names ErrMigrationBlocked (a 409, or one of the three self-clearing 400
wordings release 26.3 still sends, since only main and the 26.4 betas map
them to 409) and ErrMigrationAlreadyOnTarget. discardMigration treats
ErrMigrationNotActive and ErrNotFound as taken back and goes on to release;
refusedCreate reads the sentinels instead of the body.

The operator tests build their control-plane errors through the real
client against an httptest server, so they exercise the classification.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
#938)

* build(operator): embed the CRDs from config/crd/bases instead of a copy

The upgrade tool carried a committed copy of every generated CRD under
internal/upgrade/crds/manifests, because go:embed cannot reach outside the
directory of the file that declares it. Every CRD change therefore showed
up twice in a PR in the operator module alone.

A small package in config/crd/bases now declares the embed next to the
generated files, and internal/upgrade/crds imports it. The crd-embed
target, the copy, and the copy's drift check are gone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* build(operator): attach dist/install.yaml to releases instead of committing it

The consolidated installer repeated every CRD and RBAC change in each PR.
Its committed copy also pinned the placeholder image tag 0.1.0, so the
raw.githubusercontent.com URL the README pointed at never installed a
real release.

dist/ is now ignored. The release workflow renders the installer against
the release's operator image and uploads it as a release asset, and the
README links to that asset. CI still renders and validates it with
kubeconform on every PR.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* build: collapse generated copies in pull request diffs

Marks the chart's synced CRDs, roles, and webhook, the support tool's
chart copy, the zz_generated files, and the generated control-plane client
as linguist-generated. GitHub collapses them in a PR diff, so a CRD change
shows its schema once, in operator/config/crd/bases, which stays expanded.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* build: collapse the generated discovery fixtures in pull request diffs

The probe reports, nodes, and OperatorOps under testdata/discovery are
written by hack/discoveryfixtures, and a probe-report schema change
rewrites all of them (#661 touched 898 files). The builders are what a
reviewer reads. The expected files stay expanded, because they are the
behavior under review.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* ci(operator): summarize discovery expectation changes on pull requests

A change to one default rewrites the same line in hundreds of expected
files: #805 changed 396 of them, past the 300 files GitHub renders. The
new script groups identical added and removed lines, so #805 reads as
about 25 rows, with its single enableMetadataJournal: false still on a
row of its own.

The workflow writes the summary to the job summary and keeps one PR
comment updated in place. Fork PRs get the job summary only, since their
token cannot comment. It never fails a build.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…ump (#939)

#937 raised go.opentelemetry.io/otel to v1.47.0 and go-logr/logr to
v1.4.4 in operator/go.mod. fleet, gateway, and support-tool replace the
operator module with ../operator, so the bump raised their minimum
versions too, but their go.mod files were not updated. The Docker builds
run outside the workspace and failed with "updates to go.mod needed".

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…rt one that stays there (SFAM-2850) (#931)

* fix(pnfs): probe a mount whose data goes through the MDS, and report one that stays there (SFAM-2850)

A fence from the device brings no reconnect. The client marks the device
unusable, and every READ and WRITE of the mount goes through the metadata
server. The node plugin probed a staging mount only when first seen,
after a reconnect, or after a failed probe, so nothing took the layout
again from a namespace that can see the device. In SFAM-2850 the mount
also sent all of its I/O through the MDS for the rest of the run, and
the only trace was a warning per probe in the node plugin's log.

The primer now also probes a primed mount whose READs and WRITEs through
the server grew since the last pass, at most every 30 seconds, since
some I/O, such as unaligned direct I/O, goes through the server by
design. A mount whose probes have failed for 5 minutes is logged as an
error and reported once per run of failures with a PNFSDataThroughMDS
Warning event on the volume's claim, or on its PersistentVolume when it
has no claim. The node plugin's role already allows creating events and
listing PersistentVolumes.

Together with vela-os patch 0026, which holds a client's layouts again
after a fence or a probe through the server, the probe makes the first
device lookup once the device is usable again.

Tests were written first and failed on the unchanged primer: no probe
while data went through the server, no report after 5 minutes, and no
event. Test plan rows U-129 to U-133 and F-21. F-21, a failover on a live
cluster, is not run yet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(pnfs): the MDS guest names the clients whose data goes through it (SFAM-2850)

In SFAM-2850 one client sent 6.8 GB through the metadata server in 11
minutes, for the rest of a run, and nothing on the server said which
client it was. vela-os patch 0027 has the guest's nfsd count, per
client, the bytes of the READs and WRITEs it carries outside the grace
period and the layout hold, in /proc/fs/nfsd/clients/<id>/info.

The guest agent reads those counts every minute (--mds-data-interval,
0 turns it off) and logs a warning naming each client, by its NFS name
and address, that sent 1 MiB or more since the last look. The log
reaches the MDS pod log the way the nfsd watchdog's does. The first look
names nobody, a client first seen later is counted from zero, and a
guest kernel without the counts names nobody.

Tests were written first and failed on a watch that did nothing. Test
plan rows U-134 and U-135.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* ci(csi): let a manual run build the MDS image from another pnfs-os guest [skip ci]

CI built the MDS image only from the shared pnfsos tag, which only main
and pnfs-minimal move in vela-os. A guest built from any other vela-os
branch publishes only commit-pinned pnfsos-<sha> tags, so testing one
meant a local multi-architecture build. A manual run now takes the guest
image as guest_image, defaulting to pnfsos, so push and release builds
are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(pnfs): report I/O through the MDS by what is seen, back off failed probes, report a hung probe

F-21 on dev-ocp found two defects in the report this branch added. A probe
counts as failed when the mount's WRITE count grows under it, which a
pod's own writes through the metadata server cause too. A pod writing
unaligned O_DIRECT therefore failed every probe: 201 probes in 400
seconds, each with its own LAYOUTGET, and after five minutes a
PNFSDataThroughMDS event said the probes could not take the block layout,
which they had.

- The report is driven by what the node sees: the mount's READs and
  WRITEs going through the server for five minutes, whatever its probes
  say. That covers a fenced client, a kernel that cannot open the device
  (RHEL 9.5 on lab cluster 111), and a pod doing I/O the layout does not
  serve. The event names those causes and the last probe's outcome,
  without picking one.
- A failed probe is retried after 2s, doubling to 30s, instead of on
  every pass. A reconnect still retries it at once.
- A probe that has not returned for five minutes is reported once, as
  PNFSLayoutProbeHung. On cluster 111 a probe sat in D state in unlinkat
  and the mount was never probed again.

Test plan: U-131 rewritten, U-136 to U-139 and F-22 added.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(pnfs): send layout reports beside the primer loop, bounded, and retry a failed one

Two review findings on #931:

- The report ran in the primer's only loop, on a context with no
  deadline, through a client with no request timeout. An API server that
  accepted the request and never answered stopped probes, data re-probes,
  and alias refreshes for every mount on the node. Each report now runs
  in its own goroutine under a 30s deadline.
- overdue() marked a mount reported before the report was recorded, and a
  failed report was only logged, so a mount that stayed affected never
  got its event. A failed report clears that mark and is sent again after
  a minute.

The reporter returns its error to the primer instead of logging it.
Test plan: U-144 and U-145.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
#895)

* feat(controlplane): HostMaintenance window takes a FoundationDB host through a node drain

* ran make operator-build-installer

* feat(controlplane): HostMaintenance window takes a FoundationDB host through a node drain

* fix(controlplane): a HostMaintenance window leaves Holding on a budget the controller cannot count

* fix(controlplane): AwaitingHost waits for the pods the drain evicted to return

* fix(controlplane): a stale pass does not repeat a HostMaintenance step's event

* operator: hand FoundationDB roles off before a drain evicts them

* ran make operator-build-installer

* fix(fleet): tidy go.mod for the operator's OpenTelemetry bump

* fix: tidy the gateway, support-tool, and atlas-lib modules
)

* fix(csi): give the path-loss watch the namespace under the stack, not the device on top

Stage and heal registered the top of the volume stack with the path-loss
watch. The watch compares its record with the namespace devices sysfs
lists, and behind the dmLinear indirection (on by default since #835) or
the LVM layers the top is a device-mapper node it never lists. Its first
scan after a stage reported the volume as having lost every path, and the
guardian marked it broken and restarted opted-in workloads that were
healthy.

The watch is now given the bottom layer's devices for every plan: the
namespace under fabric, or each member under a stripe, as striped volumes
already did. They are recorded under watchedDevicePaths; an unstage also
reads the stripeDevicePaths key earlier versions wrote.

Seen in the lblk e2e runs 44463031746 and 44487645991 ("Device /dev/dm-N
is no longer present" right after a stage).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(volstack): remount the filesystem over a dm-linear volume that lost every path

The dmLinear indirection keeps the device across total path loss, so the
filesystem mounted on it kept what the outage did to it. In the lblk e2e
run 44487645991, ext4 got write errors while the namespace was gone,
aborted its journal, and failed every write after the heal re-pointed the
mapping. Without the indirection the device would have vanished and the
heal would have remounted on the new one, replaying the journal.

- dmLinear marks its artifact Interrupted when the device it maps was
  removed. A removed device keeps its number while the mapping holds it,
  so the check cannot be fooled by a reused minor.
- The runner heals an Interrupted layer only after every Yielder above it
  has let go, top down. The rest of the walk then remounts them over the
  re-pointed device.
- The filesystem layer's Yield unmounts the staging path. It refuses with
  ErrHeldElsewhere while another mount shares the superblock (the old
  pod's bind mount), because a later mount would pick up the damaged
  filesystem again. NodePublish returns that as Unavailable, and kubelet
  retries after the old pod's unpublish.
- FilesystemOps gains OtherMounts, which the CSI driver answers from its
  mount table by device number.

A re-point without loss, the live subsystem flip, is not Interrupted and
keeps the mount.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(volstack): yield a filesystem only through an ordinary unmount

Yield unmounted through clear, which falls back to a lazy unmount when the
ordinary one is refused. A lazy unmount detaches the path and nothing
else, so a filesystem something still had open survived it. Yield then
reported success, the heal re-pointed the mapping and with that erased
the interruption, and the next mount picked up the aborted filesystem.

Yield now unmounts the ordinary way only and answers a refusal with
ErrHeldElsewhere, so the publish retries. The lazy fallback stays in
Release, which serves a device that is actually gone.

Raised by the Copilot review of #941.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(integration): give the on-node shell filesystem OtherMounts

The filesystem layer's ops gained OtherMounts in this branch, and the
on-node suite's shellFilesystem, a fourth implementation that only builds
on Linux, did not have it. Both integration jobs failed vet on it.

It reads /proc/self/mountinfo and matches by device number, as the CSI
driver's implementation does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* fix(pnfs): verify a remote control plane by its own hostname when collecting mds-proxy ports

On a managed consumer the operator asked the remote control plane for its
storage nodes' LVS ports with the in-cluster webappapi client, which pins the
TLS server name to simplyblock-webappapi.<ns>.svc and trusts only the
in-cluster CA bundle. Every HTTPS endpoint failed verification, the
mds-proxy ports stayed empty, and pNFS exports timed out in Assembling.

* fix(webapi): verify every remote control plane by its own hostname

All HTTPS clients shared one TLS client that pinned the server name to
simplyblock-webappapi.<ns>.svc, so any control plane outside the Kubernetes
cluster failed verification, not only the mds-proxy port lookup.

NewClient now pins the Service name only for the in-cluster address and
verifies any other address against its own hostname, trusting the system
store plus the mounted CA bundle and presenting the mounted client
certificate. NewRemoteClient is removed; the port lookup uses NewClient.

* fix(webapi): rename the address variable that shadowed net/url
…942)

* test(discovery): record each case as a delta from a shared baseline

Each discovery case recorded its whole document and every note, so a change
every fleet's draft makes the same way rewrote all 198 cases (#805 touched 396
expected files for one new field). The values more than half of the documents
share are now recorded once in testdata/discovery/baseline.yaml, and each
case's expected.yaml is a JSON merge patch against it. The comparison stays
exact: the delta applied to the baseline must give the document back.

A note is recorded only when the field its first word names differs from the
baseline in that case. The wording of a note on a field a case shares with
the baseline is no longer checked by the case tree.

-update rewrites the baseline only when every case ran, so a run narrowed by
-run records its cases against the baseline as it is.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(discovery): describe the baseline the cases are recorded against

The case catalog shows baseline.yaml and what a case's expected files hold
now. The expectation summary also counts baseline.yaml, says that one of its
lines applies to every case, and no longer fails when a branch changes the
baseline alone. A misaligned DEV-20 row is aligned so the table gate passes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…og (#950)

* fix(chart): stop fluent-bit from dropping JSON log lines before Graylog

* fix(chart): restart fluent-bit when its config changes
* fix(operator): double the operator memory limit

* fix(operator): double the operator memory limit to 256Mi

* fix(operator): match the bundle's operator resources to the Helm chart
…952)

* fix(controlplane): run the fdb-exporter from a fully qualified image

* fix(controlplane): run the fdb-exporter from a fully qualified image
…us.mode

Managed keeps the operator's own Prometheus and trusts simplyblock-prometheus
as the metrics scraper. External creates ServiceMonitor, PodMonitor,
ScrapeConfig, and PrometheusRule objects for a prometheus-operator that already
runs, scraping as an operator-owned simplyblock-metrics-scraper account whose
token Secret the ScrapeConfig references. Switching modes deletes what the
other mode applied. Replaces the ServiceMonitor detection and the
SB_EXTRA_METRICS_SERVICE_ACCOUNTS variable.
The chart passes prometheus.mode and prometheus.external.labels to the
ControlPlane it bootstraps; the operator now creates the monitors and rules, so
prometheus-operator-servicemonitors.yaml goes. kube-prometheus-stack gets its
own enabled toggle, valid only in External mode. The Control Center and the
metrics API read from the Prometheus the mode selects, and Thanos Query keeps
the managed Prometheus's sidecar in Managed mode. The alerting template keeps
only the AlertmanagerConfig routing, for External mode.
The storage-node, device, pool, and supercluster capacity controllers read the
same flag, so with metricsAPI.enabled=false an External install fell back to
the operator's built-in simplyblock-prometheus URL and lost every capacity
sample. Adds helm-charts/scripts/test-prometheus-values.sh to the Helm lint
workflow.
* feat(atlas): send the metadata journal on cluster create

The data plane can now create an lvstore with or without the metadata
journal (simplyblock/spdk d7561c5c4), and sbcli takes it at cluster
create as md_journal.

CreateClusterParams.MetadataJournal is sent when on and left out when
off, like the other two integrity settings.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(api): enableMetadataJournal on StorageCluster and ClusterTemplate

Journals the lvstore metadata pages against torn writes, for devices that
do not complete a 4K write whole. Immutable and defaulting to off, like
the two integrity settings beside it, because the data plane records it
in the lvstore super block at creation.

The expansion copies it from the document, the reconciler sends it on
cluster create, v1alpha1 keeps it in a conversion annotation, and the
support tool edits it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(nodeprobe): report each device's logical block size

The atomic write units say what a device completes whole across a power
failure, but only together with the block size do they say whether a 4K
write does: a 4K-block device always does, and a 512-byte one only when
its atomic write unit reaches 4K. The block size was read by the scan and
dropped on the way into the report.

The support tool shows it beside the device size, with the atomic range
where the kernel published one. The captured-host discovery fixtures pick
up their recorded block sizes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(discovery): derive the metadata journal and atomicity from the devices

A device that does not complete a 4K write whole can tear a metadata page
on a power loss, and checksum validation on such a device works only with
a metadata format or 4K blocks. The draft now judges every device as a
storage node will have it, then derives the settings from that:

- A device writes 4K whole when its blocks are 4K, or when the atomic
  write unit it reports runs from at most 4K to at least 4K. With
  formatting on, an NVMe namespace not at 4K is judged at the plain 4K
  format the control plane reformats it to (format 0 where it has none),
  and its probed atomic write unit no longer applies. A device with no
  reading counts as one that does not.
- enableMetadataJournal is on unless every device writes 4K whole.
- Checksum validation stays the reviewer's choice. A note says what it
  would need: devices that need enableAtomicity4K because their report
  covers 4K, devices it cannot check because their report does not, and
  devices nothing says anything about. Where an installation states it
  on, atomicity is derived the same way, and a device it cannot check
  fails the run instead of failing the node add later.

Of the captured hosts, the GCP NVMe fleet (4K blocks) drafts the journal
off, and the GCP block fleet (512-byte disks, atomic only to 512) drafts it
on with a note that checksum validation could not check those disks. The
synthetic cases state no geometry, so they read as unknown and draft it on.

The detection runbook carries the rule and the new readings.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(nvme): read each namespace's power-fail atomicity from the drive

The kernel publishes atomic write units only from 6.11 on, so on older
kernels every 512-byte NVMe namespace read as unknown and drafted the
metadata journal on. The drive states the same thing in the Identify data
the probe already reads for the LBA formats:

- Identify Namespace: NSFEAT bit 1 (the namespace states its own values),
  NAWUPF, NABSPF, and NABO.
- Identify Controller: AWUPF, which applies where the namespace states no
  value of its own.

atlas-lib decodes them into PowerFailAtomicity, with AtomicAt answering
whether an aligned write of a given size completes whole: it has to fit
the unit and must not cross a boundary. The report carries the fields on
each namespace's formats, and the support tool shows the unit and which
field stated it.

Discovery believes the drive over the kernel attributes where both
answer. The 6.11+ kernel ignores AWUPF ("AWUPF ignored, only NAWUPF
accepted") and gives up on any NABO, both of which the specification
allows a drive to rely on.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(nvme): decode a real Samsung PM983's power-fail atomicity

Identify Controller and Identify Namespace dumps from a dev-ocp worker's
Samsung PM983 (MZQLB1T9HAJR-00007, firmware EDA5702Q), captured with
nvme-cli through oc debug, serial number and NGUID blanked.

The drive states its own power-fail unit (NSFEAT bit 1) as NAWUPF 0, one
block, and AWUPF is one block too. NAWUN is 127, which covers normal
operation only. At its 4K format a 4K write is atomic, which is what the
host's 6.12 kernel reported (atomic_write_unit_max_bytes 4096). Left at
512 bytes the same drive would not write 4K whole, and would need the
metadata journal.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(nvme): the PM983 at its 512-byte format, which writes 4K whole too

The same dev-ocp drive, formatted to LBA format 0 (512 bytes), read, and
formatted back to 4K. It restates every unit in blocks of the format in
use: NAWUPF and AWUPF become 7 (eight blocks) and NABSPF 7 (a boundary
every eight), so it completes an aligned 4K write whole at 512 bytes as
well. The 6.12 kernel agreed: atomic_write_unit_max_bytes 4096 and
atomic_write_boundary_bytes 4096.

This corrects the previous commit, which read the 4K-format values as if
they held at 512 bytes and concluded the drive would tear a 4K write
there. It is also the evidence for judging a reading only at the format
it was taken at. Discovery gains the captured reading as a case.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(discovery): a Google Cloud Local SSD is not 4K-atomic, whatever it reports

A controller cannot report less than one block of power-fail atomicity:
AWUPF and NAWUPF are 0's based. On a 4K namespace that floor reads as a 4K
atomic write without the device having promised anything, and the GKE
Local SSD read on 2026-10-27 reports exactly that. Google documents no
torn-write guarantee for Local SSD (only Hyperdisk has one: 128 KB within
1 MB boundaries), and simplyblock's deployment guide rules GCP out of
production for it.

atlas-lib/nvme gains AtomicityDisclaimed, a list of controllers, by PCI
vendor and device ID, whose statement is not believed. It starts with
Google's Local SSD controller (1ae0:001f). Discovery judges such a device
as not 4K-atomic whatever its block size, so the journal is on, and the
note says which devices were overruled and why.

Fallback checksums on 4K blocks assume a 4K write is whole, so a device is
checkable without enforced atomicity only with a metadata format or with
4K blocks that are 4K-atomic. A Local SSD therefore cannot be checked.

The captured GCP NVMe fleet now drafts the journal on.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(inventory): a real GKE node reads as GKE

The node of a one-node GKE cluster (n2-standard-2, COS, kernel 6.12, GKE
1.36), created for the device capture and deleted after it, trimmed to its
labels, annotations, and node info without the instance and project
identifiers. Environment detection concludes GKE from the kubelet version
and the cloud.google.com/gke- labels, which is all it has: GKE registers no
API group of its own.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(transcript): WithClassBlock keeps the root files and machine name

WithClassBlock rebuilt the Host from its sysfs maps alone, so a host
capture that seeds block/ lost its os-release and uname machine on the way
to a fixture and read as an unknown distribution. The captures the
operator's discovery cases replay predate root files, so nothing replayed
there changes; the EKS capture added next was the first to carry them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test: what a real EKS node and its AWS devices report

From a one-node EKS cluster (i4i.large, Amazon Linux 2023, kernel 6.18,
EKS 1.37) created for the capture and deleted after it, with an EBS io2
volume attached beside the gp3 root and the instance store. All three
devices are formatted at 512 bytes, and serial numbers, NGUIDs, EUI-64s,
and host names are blanked or replaced.

- The whole host, captured with hack/inventory/capture-host.py. The scan
  reads EBS (gp3 and io2) as atomic from 512 to 16384 bytes and the
  instance store as atomic to 512 only, which matches the kernel.
- The Identify dumps of an EBS volume and the instance store. EBS states
  its own unit (NSFEAT bit 1, NAWUPF 31): 16 KiB within 16 KiB
  boundaries, as AWS documents. The i4i instance store states nothing and
  its AWUPF is the one-block floor, so a 4K write is not promised whole.
  AWS documents torn-write prevention for some I-family instance stores,
  and the i4i is not among the devices that claim it.
- The node, which environment detection reads as EKS from the kubelet
  version and the eks.amazonaws.com labels.
- Discovery cases for both devices: EBS needs no journal, the instance
  store does. No vendor list is involved either way.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(atlas): always send md_journal on cluster create

sbcli now journals unless told not to (simplyblock/sbcli#1551), so
leaving md_journal out when it is off, as the other two integrity
settings are, would create a journaled cluster for a StorageCluster that
states enableMetadataJournal: false. The setting is now sent either way.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(discovery): pin the draft's SPDK and SPDK proxy images again

discovery.draft.images.spdk and .spdkProxy seeded the draft's spec.images
through the bootstrap ConfigMap until #780 removed it. Since then the
chart accepted both and dropped them, so an installation pinning an SPDK
build got the default one without a word.

The run carries them now: OperatorOps spec.discover.images, copied
unchanged into the draft it writes, and from there onto every
StorageNode. The chart renders the two values into it. Any Discover run
can state images, not only the one the chart creates.

The render check covers both values, one alone, and the other draft
values still reaching nothing. The operator test was shown failing first.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(storagecluster): journal the metadata unless the cluster states off

enableMetadataJournal defaulted to false, so a StorageCluster written
without the field created lvstores that a torn write could leave with a
half-written metadata page. It now defaults to true, matching sbcli's own
default, and the create request reads an unset field the same way.

A discovery run is unaffected: it always states the field, and writes false
only when every device is known to complete a 4K write whole.

A cluster that already exists is not changed, since the flag is sent only
at create. A cluster stored before the field existed now reads true in its
spec, while its lvstores keep the off they were created with.

Both tests were shown failing first.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011oztNZUjNFk6NYW7UXA1A2

# Conflicts:
#	.dockerignore
#	operator/api/v1alpha2/zz_generated.deepcopy.go
#	operator/dist/install.yaml
#	operator/internal/upgrade/crds/manifests/storage.simplyblock.io_controlplanes.yaml
@github-actions

Copy link
Copy Markdown
Contributor

Discovery expectations

This branch changes 399 expected files across 199 cases. Identical lines are grouped, most frequent first.

-   199 x     storage.simplyblock.io/component: nodeprobe
-   199 x   approved: false
-   199 x   namespace: simplyblock
-   199 x   step: {}
-   199 x apiVersion: storage.simplyblock.io/v1alpha2
-   199 x kind: ClusterDeploymentConfig
-   199 x status:
-   198 x     enableDriveFormat: true
-   198 x     maxSubsystemCount: 30
-   198 x   environment: K3s
-   198 x enableDriveFormat is set, so every drive listed here is formatted before a storage node takes it: a drive that carries anything is not usable otherwise. This is the line to remove if any of them shoul...
-   198 x maxSubsystemCount is 30, which is the middle of the range this API accepts and not a reading: how many subsystems a node should serve follows from the workload rather than from the hardware
-   196 x     distro: ubuntu
-   196 x     family: Debian
-   196 x   hostOS:
-   196 x hostOS is ubuntu of the Debian family, read from every worker's os-release. It is what ubuntuHost is decided by, and the one distribution that decides it is Ubuntu, whose NVMe-oF modules are in a pack...
-   195 x       spdkOnHostNetwork: true
-   195 x     network:
-   193 x     vcpuCount: 5
-   181 x       parityChunks: 1
-   179 x     minHugePagesSize: 16G
-   171 x enableJournalDevice is left unset: worker-01 hands over no single disk smaller than its others, so there is none to dedicate. Setting it would give up a disk the fleet did not set aside, and the field...
-   171 x minHugePagesSize is 16G, the huge pages free on the memory node chosen on worker-01, which is the smallest of the fleet
-   159 x vcpuCount is 5, 10% of the 32 logical CPUs of the memory node chosen on worker-01 (the smallest of the fleet) per storage node, at least 5: the control plane assumes this uniform across a cluster's no...
-   142 x network runs the SPDK pods on the host network, because most workers of the draft have a data interface. Each group names its own NICs. A cluster's network is fixed when it is created, so this is the ...
-   136 x     clusterMode: Single
-   116 x       dataChunks: 2
-   116 x     stripe:
-    91 x stripe is 2+1 as a single-node cluster (clusterMode Single): each chunk of a stripe lands on its own device of the one worker, which hands over 4 for data, so the cluster survives 1 device failure but...
-    53 x network runs the SPDK pods on the host network, on the primary NIC, because no worker has a data interface of its own. A cluster's network is fixed when it is created, so this is the line to correct n...
+    42 x     clusterMode: null
-    14 x vcpuCount is 5, 10% of the 16 logical CPUs of the memory node chosen on worker-01 (the smallest of the fleet) per storage node, at least 5: the control plane assumes this uniform across a cluster's no...
+    12 x     minHugePagesSize: null
-     7 x stripe is 2+1, which needs 4 storage nodes and this fleet of 32 has them: it is the widest stripe to propose without knowing the workload. This fleet could also carry 1+1, 4+1, 1+2, 2+2, 4+2, which tr...
-     6 x enableJournalDevice is left unset: worker-02 hands over no single disk smaller than its others, so there is none to dedicate. Setting it would give up a disk the fleet did not set aside, and the field...
-     6 x minHugePagesSize is 16G, the huge pages free on the memory node chosen on worker-02, which is the smallest of the fleet
-     6 x stripe is 2+1 as a single-node cluster (clusterMode Single): each chunk of a stripe lands on its own device of the one worker, which hands over 3 for data, so the cluster survives 1 device failure but...
-     6 x vcpuCount is the default minimum of 5, because no worker reported the cores of the memory node it was placed on
-     5 x vcpuCount is 5, 10% of the 32 logical CPUs of the memory node chosen on worker-02 (the smallest of the fleet) per storage node, at least 5: the control plane assumes this uniform across a cluster's no...
-     4 x vcpuCount is 5, 10% of the 8 logical CPUs of the memory node chosen on worker-01 (the smallest of the fleet) per storage node, at least 5: the control plane assumes this uniform across a cluster's nod...

53 more distinct lines are not shown. Run operator/hack/summarize-discovery-expectations.sh origin/prometheus-operator-migration locally for the rest.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants