A self-hosted LLM stack as a single Helm umbrella chart. Designed for teams that cannot send data outside and need to integrate with an existing identity provider.
| Component | Chart | Purpose |
|---|---|---|
| Open WebUI | open-webui |
Chat interface |
| Ollama | ollama |
Local model inference |
| LiteLLM | litellm-helm |
OpenAI-compatible gateway |
| Qdrant | qdrant |
Vector store for RAG |
| Keycloak | keycloakx |
SSO / OIDC identity provider |
- Kubernetes ≥ 1.25
- Helm ≥ 3.14
- A default StorageClass (for persistent volumes)
- Optional: an Ingress controller (nginx) and cert-manager for TLS
helm repo add llm-stack https://ufukdev.github.io/llm-stack
helm repo update
helm install llm-stack llm-stack/llm-stack \
--namespace llm --create-namespace \
--set global.domain=example.comNote: GitHub Pages must be enabled on the
gh-pagesbranch forhelm repo addto work. The release workflow publishes there automatically on every merge tomainthat touchescharts/.
# Create a local cluster
kind create cluster --name llm-stack
helm dependency update charts/llm-stack
helm install llm-stack charts/llm-stack \
--kube-context kind-llm-stack \
--namespace llm --create-namespace \
-f charts/llm-stack/ci/kind-values.yaml
# Access Open WebUI
kubectl -n llm port-forward svc/llm-stack-open-webui 8080:80
# Open http://localhost:8080All top-level keys are defined in charts/llm-stack/values.yaml and validated by values.schema.json.
| Key | Default | Description |
|---|---|---|
global.domain |
"" |
Base domain for Ingress hosts (chat.{domain}, auth.{domain}) |
inference.mode |
local |
local (Ollama) or external (bring your own endpoint) |
openWebui.enabled |
true |
Deploy Open WebUI |
ollama.enabled |
true |
Deploy Ollama for local inference |
qdrant.enabled |
true |
Deploy Qdrant vector store |
keycloak.enabled |
true |
Deploy Keycloak for SSO |
litellm.enabled |
true |
Deploy LiteLLM gateway |
networkPolicy.enabled |
false |
Deploy NetworkPolicies (requires Calico/Cilium) |
ingress.enabled |
false |
Deploy Ingress resources |
Secrets (litellmMasterKey, oidcClientSecret, keycloakAdminPassword) are generated on first install and remain stable across upgrades via Helm lookup. You can pin them explicitly:
secrets:
litellmMasterKey: "your-master-key"
oidcClientSecret: "your-oidc-secret"
keycloakAdminPassword: "your-admin-password"Or use --set secrets.litellmMasterKey=... without committing secrets to version control.
Ollama runs inside the cluster. All model weights are downloaded on first start.
inference:
mode: local
ollama:
enabled: true
ollama:
models:
pull:
- qwen2.5:0.5b # or llama3.1:8b, mistral, etc.See ci/kind-values.yaml for a complete local example.
Requires the NVIDIA GPU Operator. See examples/values-gpu.yaml.
Point Open WebUI (via LiteLLM) at an external OpenAI-compatible API:
inference:
mode: external
external:
baseUrl: "https://api.example.com/v1"
apiKey: "sk-..."
ollama:
enabled: falseSee ci/external-inference-values.yaml.
All images mirrored to an internal registry, no internet egress required.
See examples/values-airgapped.yaml.
Use Okta, Azure AD, Authentik, or any other OIDC-compatible IdP:
keycloak:
enabled: false
oidc:
external:
enabled: true
issuerUrl: "https://sso.example.com/auth/realms/my-realm"
clientSecret: "..." # use --set, never commit
open-webui:
sso:
oidc:
clientId: "open-webui" # must match the client registered in your IdPSee examples/values-external-idp.yaml and ci/external-oidc-values.yaml.
When keycloak.enabled=true, a post-install Job imports the llm-stack realm automatically. The realm includes an open-webui OIDC client, llm-user/llm-admin roles, and matching groups.
When global.domain is set the chart automatically configures KC_HOSTNAME_URL so Keycloak always advertises its public address (auth.<domain>) in the OIDC discovery document — regardless of which internal address Open WebUI uses to fetch it. Both the browser redirect and the server-side token exchange therefore reach Keycloak through the Ingress.
global:
domain: "mycompany.com"
ingress:
enabled: true
className: nginx
tls:
issuer: letsencrypt # cert-manager ClusterIssuerWith this config SSO works end-to-end:
Browser → https://chat.mycompany.com
→ "Sign in with Keycloak"
→ https://auth.mycompany.com/auth/realms/llm-stack/... (Keycloak login page)
→ back to Open WebUI with a valid session
Port-forward cannot provide SSO because the browser cannot resolve the internal Kubernetes service address (llm-stack-keycloakx-http.llm.svc.cluster.local) that Keycloak embeds in redirect URLs when no public domain is configured.
Workaround — skip SSO and register with email/password:
# Start port-forward
kubectl -n llm port-forward svc/llm-stack-open-webui 8080:80
# Create the first (admin) account via API
curl -X POST http://localhost:8080/api/v1/auths/signup \
-H "Content-Type: application/json" \
-d '{"name":"Admin","email":"admin@local.test","password":"changeme"}'Then open http://localhost:8080 and sign in with email/password. The first account is automatically granted the admin role.
See docs/sso.md for full details.
When networkPolicy.enabled=true, three policies are applied:
- default-deny-egress — blocks all outbound traffic from the namespace by default
- allow-internal — allows intra-namespace traffic and DNS (kube-system port 53)
- allow-webui-egress-https — allows Open WebUI pods to reach HTTPS endpoints (required for model registry and HuggingFace embedding model download on startup)
Requires a CNI that enforces NetworkPolicies (Calico, Cilium). To test locally:
bash hack/kind-with-calico.sh# Lint the chart
helm lint charts/llm-stack --strict
# Lint with chart-testing
ct lint --config ct.yaml
# Update dependencies
helm dependency update charts/llm-stack
# Render templates without installing
helm template llm charts/llm-stack -f charts/llm-stack/ci/kind-values.yaml