Tenant cluster lifecycle and addon management controller for the Butler platform.
butler-controller is the core controller responsible for tenant cluster lifecycle management in the Butler platform. It handles:
- Tenant Cluster Provisioning: Creates and manages Kubernetes clusters using CAPI and Steward
- Multi-Tenancy: Team-based isolation with RBAC and resource quotas
- Addon Management: Installs and manages cluster addons (CNI, storage, ingress, etc.)
- GitOps Integration: Optional Flux bootstrap for declarative cluster management
butler-controller runs in the Butler management cluster and watches for TenantCluster custom resources. When a TenantCluster is created, the controller orchestrates VM provisioning, control plane setup via Steward, worker node joining, and addon installation.
flowchart TD
subgraph MC[Management Cluster]
BC[butler-controller]
CAPI[Cluster API]
Steward[Steward]
BC -->|Creates| TC[TenantCluster CR]
BC -->|Creates| CAPIC[CAPI Cluster]
BC -->|Creates| SCP[StewardControlPlane]
BC -->|Creates| MD[MachineDeployment]
Steward -->|Hosts| API[Tenant API Server]
end
subgraph TC1[Tenant Cluster]
Workers[Worker Nodes]
Cilium[Cilium CNI]
MetalLB[MetalLB]
Addons[Other Addons]
end
API -->|Manages| Workers
BC -->|Installs| Cilium
BC -->|Installs| MetalLB
BC -->|Installs| Addons
| Controller | Watches | Responsibility |
|---|---|---|
| ButlerConfigReconciler | ButlerConfig | Platform-wide configuration singleton |
| TeamReconciler | Team | Creates namespaces and RBAC for teams |
| TenantClusterReconciler | TenantCluster | Full cluster lifecycle (CAPI, Steward, addons) |
| TenantAddonReconciler | TenantAddon | Post-creation addon management |
| StewardSecretReconciler | Secrets | Kubeconfig format translation for CAPI |
| StewardStatusReconciler | TenantControlPlane | CAPI status synchronization |
butler-controller works with CRDs defined in butler-api:
Platform-wide singleton configuration:
apiVersion: butler.butlerlabs.dev/v1alpha1
kind: ButlerConfig
metadata:
name: butler
spec:
multiTenancy:
mode: Enforced # Enforced | Optional | Disabled
defaultNamespace: butler-tenantsTeam with access control and resource limits:
apiVersion: butler.butlerlabs.dev/v1alpha1
kind: Team
metadata:
name: platform-engineering
spec:
displayName: "Platform Engineering"
access:
users:
- name: admin@example.com
role: admin
groups:
- name: platform-engineers
role: member
resourceLimits:
maxClusters: 10
maxTotalCPU: "200"
maxTotalMemory: "1Ti"Tenant Kubernetes cluster definition:
apiVersion: butler.butlerlabs.dev/v1alpha1
kind: TenantCluster
metadata:
name: production
namespace: platform-engineering
spec:
kubernetesVersion: "v1.30.2"
workers:
replicas: 3
machineTemplate:
cpu: 4
memory: "16Gi"
networking:
podCIDR: "10.244.0.0/16"
serviceCIDR: "10.96.0.0/12"
loadBalancerPool:
start: "10.40.1.100"
end: "10.40.1.150"
addons:
cni:
provider: cilium
version: "1.16.0"
loadBalancer:
provider: metallb
version: "0.14.5"Post-creation addon management:
apiVersion: butler.butlerlabs.dev/v1alpha1
kind: TenantAddon
metadata:
name: production-argocd
namespace: platform-engineering
spec:
clusterRef:
name: production
addon: argocd
version: "2.10.0"Butler supports three multi-tenancy modes:
| Mode | Description | Use Case |
|---|---|---|
| Enforced | Teams required, strict namespace isolation | Enterprise production |
| Optional | Teams available but not required | Mixed environments |
| Disabled | No team functionality | Demos, single-user |
Team/acme (cluster-scoped)
├── Creates: Namespace/acme
└── Contains:
├── TenantCluster/production
│ └── Creates: Namespace/production-a7b8c9
│ ├── CAPI Cluster
│ ├── StewardControlPlane
│ └── MachineDeployment
├── TenantCluster/staging
└── TenantAddon/production-argocd
- Go 1.24+
- Docker
- kubectl
- Access to a Kubernetes cluster with butler-api CRDs installed
make build# Ensure butler-api CRDs are installed
kubectl apply -f ../butler-api/config/crd/bases/
# Run the controller locally
make runmake testmake docker-build IMG=ghcr.io/butlerdotdev/butler-controller:devbutler-controller/
├── cmd/
│ └── main.go # Controller entrypoint
├── internal/
│ ├── controller/
│ │ ├── butlerconfig/ # Platform config controller
│ │ ├── team/ # Team controller
│ │ ├── tenantcluster/ # Main cluster lifecycle controller
│ │ ├── tenantaddon/ # Addon controller
│ │ ├── kamajisecret/ # Kubeconfig translation
│ │ └── kamajistatus/ # Status sync
│ ├── tenant/ # Tenant cluster client management
│ ├── addons/ # Helm-based addon installer
│ └── flux/ # GitOps bootstrapper
├── config/
│ ├── default/ # Kustomize base
│ ├── manager/ # Controller deployment
│ └── rbac/ # RBAC configuration
├── docs/
│ └── ARCHITECTURE.md # Detailed architecture documentation
├── Dockerfile
├── Makefile
└── README.md
Contributions are welcome. Please read the contributing guidelines before submitting a pull request.
All commits must be signed off (git commit -s) to certify the Developer Certificate of Origin.
make lintmake test
make lintCopyright 2025 Butler Labs.
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.