Native SMB support via mgr/smb (phase 1)#802
Draft
UtkarshBhatthere wants to merge 31 commits into
Draft
Conversation
Package samba, samba-vfs-modules and ctdb from the noble archive and expose an smbd app (disabled by default) driven by a config under SNAP_DATA/conf/samba. No placement or API integration; this is the packaging half of the native SMB experiment. Assisted-by: claude-code:claude-fable-5 Signed-off-by: Utkarsh Bhatt <utkarsh_bhatt@outlook.com>
Add Go types mirroring the upstream mgr/smb SMBSpec service spec JSON (service_spec.py): SMBSpec, SMBPlacementSpec, SMBPublicAddrSpec and the SMBDestination union decoder, plus the upstream cluster id regex from mgr/smb validation.py. Unknown fields are tolerated so newer mgr serializations do not break decoding. Assisted-by: claude-code:claude-fable-5 Signed-off-by: Utkarsh Bhatt <utkarsh_bhatt@outlook.com>
Add SMBServicePlacement implementing PlacementIntf: SMBSpec payload validation (upstream cluster id regex, feature gate accepting only 'clustered', rejection of Phase-1-unsupported fields), hospitality checks (port 445 free, smb clusters node-disjoint), and DB recording via GroupedServicesQuery with the spec JSON stored verbatim as group config. ServiceInit and PostPlacementCheck are documented no-ops until the M3 lifecycle work. Assisted-by: claude-code:claude-fable-5 Signed-off-by: Utkarsh Bhatt <utkarsh_bhatt@outlook.com>
Add the cluster-scoped /1.0/services/smb endpoint (PUT SMBSpec apply, DELETE cluster removal, GET status) and the node-scoped /1.0/services/smb/node endpoint used by the per-node fan-out. ApplySMB resolves spec placement (hosts/count) against cluster members, diffs it with recorded membership and converges idempotently; specs are canonicalized (json.Compact) so stored group configs compare stably. Extend GroupedServicesQuery with group member/config accessors. Assisted-by: claude-code:claude-fable-5 Signed-off-by: Utkarsh Bhatt <utkarsh_bhatt@outlook.com>
Tighten the samba part to a whitelist prime: the smbd/ctdb daemons and tools, ctdb helpers (incl. ctdb_mutex_ceph_rados_helper), samba module dir and the exact ELF NEEDED closure of those binaries, dropping docs, man pages, AD client tools and python bindings. Add the ctdbd app (snapd-managed, install-mode disable) with its start wrapper pointing CTDB_BASE at SNAP_DATA, layouts for /etc/ctdb and /usr/share/ctdb, a snapctl-based replacement 50.samba event script (CTDB manages smbd via snapd, not systemd), and a sambacc part pinned to 0.9 for config translation. Assisted-by: claude-code:claude-fable-5 Signed-off-by: Utkarsh Bhatt <utkarsh_bhatt@outlook.com>
Replace the samba part's prime whitelist with negative-list pruning: snapcraft resolves shared staged files against every staging part's filter, so whitelisting inside the samba part silently dropped libs other parts prime (ceph-mgr's libpython3.12, dashboard's markdown and dnspython, libldap/libicu symlink targets). Exclude only samba-owned paths: docs, man pages, GPO templates, and AD/DC/NetBIOS tooling. Retire stub patch 0003 and add an mgr-smb part that sparse-clones the ceph tag matching the staged PPA debs (asserted against apt-cache policy at build time) and installs src/pybind/mgr/smb into the snap's mgr module path. Assisted-by: claude-code:claude-fable-5 Signed-off-by: Utkarsh Bhatt <utkarsh_bhatt@outlook.com>
Add 'microceph smb apply-spec/rm/list' hitting the services/smb endpoints directly. Hidden: the supported control plane is mgr/smb via the orchestrator; these keep E2E honest before microceph-orch gains smb support, and remain for support tooling. Assisted-by: claude-code:claude-fable-5 Signed-off-by: Utkarsh Bhatt <utkarsh_bhatt@outlook.com>
The smb hospitality check reads group membership from the cluster DB, and microcluster transactions require the request context (its logger); context.Background() fails with 'Logger does not exist on context', caught live on the first end-to-end spec apply. Thread ctx through PlacementIntf.HospitalityCheck for all services. Assisted-by: claude-code:claude-fable-5 Signed-off-by: Utkarsh Bhatt <utkarsh_bhatt@outlook.com>
Add per-node daemon key handling mirroring cephadm's SMBService key model: mon caps allow reads plus this cluster's smb/config/ config-key prefix; osd caps derive from the spec's RADOS URIs with enhanced rwx on the smb pool's cluster.meta. object prefix (CTDB reclock). Caps converge on re-apply via auth caps rather than get-or-create. The include_ceph_users data-path keys (owned by mgr/smb) are fetched into standard-named keyring files under the conf dir, where the default /etc/ceph search path finds them; entity names are validated before touching disk. Wiring into the enable flow lands with M3 lifecycle. Assisted-by: claude-code:claude-fable-5 Signed-off-by: Utkarsh Bhatt <utkarsh_bhatt@outlook.com>
Render the per-node SMB config set: fetch the mgr/smb sambacc document from config_uri, translate it for this backend (classic ceph vfs module, since the snap's samba 4.19 has no ceph_new; inject a microceph globals section carrying the proven snap paths, netbios name and CTDB wiring) and emit smb.conf through the bundled sambacc print-config (pure-python path). CTDB files render natively: ctdb.conf points the 4.19 'cluster lock' at the bundled rados mutex helper with a per-cluster object in the lock pool's default namespace (the 4.19 helper has no namespace support, so the namespaced cluster_lock_uri object stays with mgr/smb untouched), nodes carries caller-ordered private IPs, public_addresses resolves each VIP's interface on-node. Clustered specs additionally get an osd cap for the reclock prefix. All writes are atomic tmp+rename; golden-file tests cover each rendered file. Assisted-by: claude-code:claude-fable-5 Signed-off-by: Utkarsh Bhatt <utkarsh_bhatt@outlook.com>
Wire the smb placement flow end to end: ServiceInit now creates the per-cluster directories (root-owned; confined daemons have no dac_override), ensures keyrings, renders configs with row-id-ordered node IPs (CTDB node numbers are line indices, so ordering must be stable and append-only), populates CTDB_BASE (stock deb-enabled event script links, the snapctl 50.samba script, functions, script.options) and starts ctdbd; PostPlacementCheck verifies ctdbd stays up. Disable stops ctdbd/smbd, removes keyrings, configs, runtime and per-cluster data dirs, then the DB record. Membership or spec changes now trigger a serialized regenerate (re-render + ctdbd restart) across all desired members via a node-scoped POST, so every nodes file converges to the same content. VIP interfaces resolve against local subnets. Assisted-by: claude-code:claude-fable-5 Signed-off-by: Utkarsh Bhatt <utkarsh_bhatt@outlook.com>
Live single-node bring-up surfaced four gaps, all fixed at the root: ctdbd's compiled-in paths need overriding inside the snap ([database] directories and the logging location in the rendered ctdb.conf, plus CTDB_EVENTD/CTDB_*_HELPER/CTDB_HELPER_BINDIR exports and a /run/ctdb pid dir in the start wrapper); CTDB_BASE additionally needs notify.sh linked. The nodes file rendered empty on first enable because rendering runs before DbUpdate records the local node; the local node is now appended when its row is missing, preserving row-id order. With these, ctdbd reaches recovery NORMAL and takes the RADOS cluster lock with the per-node daemon key. Assisted-by: claude-code:claude-fable-5 Signed-off-by: Utkarsh Bhatt <utkarsh_bhatt@outlook.com>
Removing conf/ctdb outright leaves the snap namespace's /etc/ctdb layout bind pointing at a dead inode, so services read an empty ghost directory until the namespace is rebuilt (observed live after a disable/re-enable cycle). Clear the directory's contents instead. Assisted-by: claude-code:claude-fable-5 Signed-off-by: Utkarsh Bhatt <utkarsh_bhatt@outlook.com>
An enabled ctdbd starting before the deployment engine has rendered ctdb.conf (snap refresh, node reboot) crashlooped into systemd's restart rate limit, which then blocked the legitimate start during enable. Wait for the config file instead of exiting. Assisted-by: claude-code:claude-fable-5 Signed-off-by: Utkarsh Bhatt <utkarsh_bhatt@outlook.com>
The leader-proxied UseTarget fan-out canceled long-running enable and
regenerate legs mid-flight ('context canceled' surfacing at the next
DB or client call on the target node), observed repeatedly during the
three-node apply. Member names and addresses now come from the local
trust store (heartbeat-fresh, no network call) and each node operation
connects straight to its target via Connect().Member(), removing the
proxy hop entirely. Cluster-scope client timeouts sized to the full
multi-node apply (900s) and node-scope ones to a single node's enable
(300s); the unused GetClusterMemberAddresses client wrap is reverted.
Assisted-by: claude-code:claude-fable-5
Signed-off-by: Utkarsh Bhatt <utkarsh_bhatt@outlook.com>
CTDB shuts a node down when its cluster lock command line differs from the leader's, and ours differed twice per node: the helper path used the revision-specific $SNAP directory (x8 vs x10 across nodes) and the lock entity was the per-node daemon key. The lock line now uses the /snap/<name>/current alias and a shared per-cluster entity (client.smb.<cluster_id>) that every node ensures on enable; the reclock cap moves from the daemon keys to that entity, and RemoveSMB retires it after the last node leaves. Assisted-by: claude-code:claude-fable-5 Signed-off-by: Utkarsh Bhatt <utkarsh_bhatt@outlook.com>
Implement apply_smb and remove_service('smb.<id>') in the microceph
orchestrator module, forwarding the SMBSpec to microcephd over the
unix-socket REST client (PUT /1.0/services/smb, DELETE on remove).
ServiceSpec.to_json() nests subclass fields (cluster_id, config_uri,
...) under a 'spec' key; apply_smb flattens that into microcephd's
flat SMBSpec wire format, which the daemon validates strictly (an
unflattened payload fails with "cluster_id '' is not a valid ID").
Adds pytest coverage: conftest stubs the mgr runtime imports
(mgr_module, orchestrator, ceph.deployment, snaphelpers) so the module
imports outside a mgr, FakeSession asserts the wire format, and module
routing tests exercise apply/remove dispatch with a fake spec that
mirrors the real nested to_json() shape. pyproject gains a dev
dependency group (pytest) and a hatchling build so the package is
installable; .gitignore keeps __pycache__ and .venv out of the tree.
uv run pytest tests/ -> 7 passed
Assisted-by: claude-code:claude-fable-5
Signed-off-by: Utkarsh Bhatt <utkarsh_bhatt@outlook.com>
The grouped-service re-enable loop passes the group's service name to snapctl, but the smb group has no snap app of its own: ctdbd is the unit to start, and it brings up smbd via its event script. On every daemon start with an smb cluster in the DB this logged 'snapctl start microceph.smb: unknown service' and left an inactive smb node down after reboot. go test ./ceph/ -run Start -> PASS (incl. TestReEnableSMBGroupStartsCtdbd) Assisted-by: claude-code:claude-fable-5 Signed-off-by: Utkarsh Bhatt <utkarsh_bhatt@outlook.com>
mgr/smb expands the default 'samba-vfs' share provider to the proxied variant, so every default 'ceph smb share create' produces a spec with the cephfs-proxy feature and fails validation. Name the working provider in the error so users are not left guessing. Assisted-by: claude-code:claude-fable-5 Signed-off-by: Utkarsh Bhatt <utkarsh_bhatt@outlook.com>
The enable flow rendered configs and started services but never imported user_sources into the passdb, leaving every mgr-created cluster with no way to authenticate (M3 seeded users by hand). ApplySMB now ends with a node-scoped seeding call on the first placed member: fetch each user_sources document (both rados:// pool objects and the rados:mon-config-key: scheme mgr/smb actually publishes), parse the sambacc users JSON and smbpasswd -s -a each entry against the rendered smb.conf, password over stdin. Seeding runs on every apply, not just spec changes, because the document behind a URI can change while the URI stays identical; the passdb is CTDB-replicated so one member covers the cluster. Imports retry while CTDB finishes recovery. Matching system users remain the admin's task in Phase 1. go test ./ceph/ ./api/... ./client/... -> ok Assisted-by: claude-code:claude-fable-5 Signed-off-by: Utkarsh Bhatt <utkarsh_bhatt@outlook.com>
A spec change with steady membership only regenerates: enable is the sole place keyrings were fetched, so entities added by later spec revisions never reached the nodes. Concretely, the first share on an mgr-created cluster adds client.smb.fs.cluster.<id> (the vfs_ceph user) to include_ceph_users, the regenerated smb.conf references it, and every tree connect fails NT_STATUS_UNSUCCESSFUL on the missing keyring. EnsureSMBKeyrings is idempotent and also re-converges URI-derived caps, so regenerate now runs it before rendering. go test ./ceph/ -> ok Assisted-by: claude-code:claude-fable-5 Signed-off-by: Utkarsh Bhatt <utkarsh_bhatt@outlook.com>
populateCTDBBase linked event scripts, functions and notify.sh into the revisioned $SNAP dir. snapd keeps only two revisions, so the second refresh after an enable garbage-collects the link targets and ctdbd crash-loops on 'Failed to run init event'. Link through /snap/<name>/current instead (SnapStable, already used for the cluster lock helper line). The lifecycle test env now keeps Snap and SnapStable distinct so a regression fails the symlink assertions. go test ./ceph/ -> ok Assisted-by: claude-code:claude-fable-5 Signed-off-by: Utkarsh Bhatt <utkarsh_bhatt@outlook.com>
ceph orch ls constructs a spec per service; a generic ServiceSpec with service_type='smb' dispatches to SMBSpec through the service registry and fails validation without a cluster_id, breaking orch ls for every deployment with an smb cluster. describe_service now pulls the verbatim specs microcephd stores (one list_smb call when any smb service is recorded) and rebuilds them via ServiceSpec.from_json, skipping clusters with no stored spec. uv run pytest tests/ -> 9 passed Assisted-by: claude-code:claude-fable-5 Signed-off-by: Utkarsh Bhatt <utkarsh_bhatt@outlook.com>
Covers the Phase 1 mgr-driven lifecycle on a 3-node inner cluster: orchestrator backend enablement, cluster+share creation via ceph smb (declarative apply for the share, since the imperative default provider expands to the proxied vfs), roundtrip via a CTDB public address with smbclient, VIP-holder kill/failover/rejoin, and cluster removal. The snap is re-installed in devmode by the suite (confined smbd needs the pending smb-support snapd interface). Per the harness rules, decisions parse in Python: smb_ops.py carries the pure helpers (VIP derivation from the cluster CIDR, ctdb -X status health count, VIP-to-pnn lookup, share spec document) with pytest coverage, and the harness class gains the ctdb exec wrapper, a _poll_until-based health waiter and the devmode reinstall step. tox -e robot -- --dryrun --test-suite smb-tests -> 5 tests resolve pytest test_harness_helpers.py -> 116 passed Assisted-by: claude-code:claude-fable-5 Signed-off-by: Utkarsh Bhatt <utkarsh_bhatt@outlook.com>
How-to page for serving SMB shares: devmode requirement and the pending smb-support snapd interface, orchestrator backend enablement, subvolume and system-user preparation, CTDB-clustered cluster create with public addresses, declarative share creation (and why the imperative default provider is rejected), client connection, the reconnect-based failover semantics, and removal. Indexed under 'Consuming cluster storage'. docs: make html (--fail-on-warning) -> build succeeded Assisted-by: claude-code:claude-fable-5 Signed-off-by: Utkarsh Bhatt <utkarsh_bhatt@outlook.com>
Upstream expands the abbreviated 'samba-vfs' provider to the proxied variant, so the plain 'ceph smb share create' always produced a spec with the cephfs-proxy feature and failed microcephd validation, breaking the native CLI experience the mgr/smb integration exists for. Patch the staged smb module so the default expands to samba-vfs/new: microceph serves shares via direct libcephfs and deploys no proxy daemon. Stored resources keep the abbreviated provider, so the same document still expands to proxied on cephadm. An explicit samba-vfs/proxied request keeps its upstream meaning and is still rejected; the rejection message no longer blames the default. Assisted-by: claude-code:claude-fable-5 Signed-off-by: Utkarsh Bhatt <utkarsh_bhatt@outlook.com>
With the default provider patched to the non-proxied vfs, the declarative-apply workaround (yaml document shipped base64 into the container) collapses to one 'ceph smb share create' line; drop the now-unused smb_share_spec_yaml helper and its pytest case. tox -e robot -- --dryrun --test-suite smb-tests -> 5 tests resolve pytest test_harness_helpers.py -> 115 passed Assisted-by: claude-code:claude-fable-5 Signed-off-by: Utkarsh Bhatt <utkarsh_bhatt@outlook.com>
The provider workaround is gone; document the plain CLI and note the patched default expansion plus the proxied-provider rejection. docs: make html (--fail-on-warning) -> build succeeded Assisted-by: claude-code:claude-fable-5 Signed-off-by: Utkarsh Bhatt <utkarsh_bhatt@outlook.com>
The ceph part's patch loop runs against CRAFT_STAGE, where another part's files are only present if that part happened to stage first; patching mgr/smb from there failed on a clean rebuild (and the loop also tripped over the retired stub patch still cached in the part source). Apply the provider-default patch from the mgr-smb part's own override-build against its CRAFT_PART_INSTALL, and move it to patches/mgr-smb/ so the ceph loop's glob no longer picks it up. Assisted-by: claude-code:claude-fable-5 Signed-off-by: Utkarsh Bhatt <utkarsh_bhatt@outlook.com>
EnableService now renders ceph.conf via updateConfigFunc before enabling any service (main's deferred bootstrap change), so the SMB placement tests must bypass the database-dependent rendering the same way the NFS and generic placement suites do. Assisted-by: claude-code:claude-fable-5 Signed-off-by: Utkarsh Bhatt <utkarsh_bhatt@outlook.com>
UtkarshBhatthere
force-pushed
the
smb-native-phase1
branch
from
July 12, 2026 09:19
a9b6dc2 to
95efb05
Compare
UtkarshBhatthere
marked this pull request as draft
July 12, 2026 09:25
Assisted-by: claude-code:claude-fable-5 Signed-off-by: Utkarsh Bhatt <utkarsh_bhatt@outlook.com>
Contributor
Author
|
Spec-review follow-up on Decision from spec review: allow the upstream
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds native SMB file sharing to MicroCeph: CephFS subvolumes served by Samba, CTDB-clustered for HA, driven end to end by the upstream
ceph smbmanager module. MicroCeph implements the mgr/smb orchestrator contract, so the user-facing surface (CLI, declarative resources, dashboard) is the documented upstream experience.Jira: CEPH-1240
Architecture
ceph smb(mgr/smb, packaged in-snap) ->microcephorchestrator module (apply_smb/remove_service/describe_service) -> microcephd REST (/1.0/services/smb*) -> per-node rendering (sambacc-based smb.conf, native ctdb.conf/nodes/public_addresses) -> ctdbd manages smbd via a snapctl event script. The SMBSpec is stored verbatim as the grouped-service config (NFS precedent); apply is idempotent and convergent. Passdb users are seeded automatically from the spec'suser_sourceson every apply.Highlights
patches/mgr-smb/) expanding the default share provider tosamba-vfs/new: MicroCeph serves shares via direct libcephfs and deploys no cephfs-proxy daemon, so the plainceph smb share createworks natively. Explicitsamba-vfs/proxiedrequests are rejected. An upstream RFE to make the default configurable will retire the patch.microceph smbdebug CLI for support.Testing
smb-testssuite (backend enable, imperative cluster+share, smbclient roundtrip via VIP, VIP-holder kill/failover/rejoin, removal) resolving under--dryrun; pure parsers pytest-covered./api/smbverified.Notes for reviewers
smb-supportinterface lands, SMB needs the snap in devmode (confined smbd is blocked on setgroups + setuid/setgid; minimal grants were proven by local profile patching and are written up for the snapd proposal).domainfeature) is deliberately rejected in this phase; standalone user auth only.🤖 Generated with Claude Code
https://claude.ai/code/session_01RHqYMvUeUXxJTsCNdHmwYt