Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,18 @@ bot-prod/ bot-rooms/ mariadb/ legacy AKS folders, disabled
Keep Auth behind the tunnel and ensure Cloudflare does not remove the header.
Cluster-internal IdP calls carry no visitor address and share Auth's bounded
token budget of 60 requests per minute.
- **Backend exposure:** the backend is in-cluster only except its declared
`/public` prefix, routed on `api.polinetwork.org/public/...` by one
IngressRoute with a path whitelist, body limit and rate limits
(`apps/backend/public-route.yaml`; `tests/` enforces it). New public routes
are added in the backend, never as new Ingresses here.
- **Regulations editor:** `regulations-collab` (backend image, `backend`
namespace) publishes only its websocket on `collab.polinetwork.org`.
`latex-worker` runs untrusted LaTeX with no egress, a read-only root and a
namespace quota. Both wait for the `regulations-secrets` Kustomization,
which is Ready only once the Key Vault entries `regulations-internal-secret`
and `regulations-worker-secret` exist; the worker also waits for its image.
Until then nothing of theirs runs; afterwards they start by themselves.
- **Secrets:** `ExternalSecret`s sync from Azure Key Vault (`kv-pn-apps`,
`kv-pn-infra`). No secrets live in Git.
- **Images:** PoliNetwork apps run `ghcr.io/polinetworkorg/<app>:latest`. The
Expand Down
69 changes: 69 additions & 0 deletions apps/backend/deployment.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,75 @@ spec:
secretKeyRef:
name: azure-kv
key: oauth-private-jwk
# Regulations editor: every variable is optional and the feature
# stays off without them. Empty values (TODO) are pending from
# Lorenzo; the backend reads env with emptyStringAsUndefined. The
# secrets come from apps/regulations-secrets; env from a Secret is
# read at container start, so restart the backend once after they
# first sync.
- name: REGULATIONS_INTERNAL_SECRET
valueFrom:
secretKeyRef:
name: regulations-core
key: internal-secret
optional: true
- name: REGULATIONS_COLLAB_PUBLIC_URL # handed to browsers in collab tickets
value: "wss://collab.polinetwork.org"
- name: REGULATIONS_COLLAB_INTERNAL_URL
value: "http://regulations-collab.backend.svc.cluster.local:4101"
- name: REGULATIONS_WORKER_URL
value: "http://latex-worker.latex-worker.svc.cluster.local:4200"
- name: REGULATIONS_WORKER_SECRET
valueFrom:
secretKeyRef:
name: regulations-core
key: worker-secret
optional: true
- name: REGULATIONS_GITHUB_MODE
value: "app"
# GitHub App webhook: POST https://api.polinetwork.org/public/webhooks/github
# (public-route.yaml), verified with this secret. Polling is only a slow
# reconciliation for missed deliveries.
- name: REGULATIONS_GITHUB_WEBHOOK_SECRET
valueFrom:
secretKeyRef:
name: regulations-github
key: webhook-secret
optional: true
- name: REGULATIONS_GITHUB_APP_ID
value: "" # TODO: GitHub App ID
- name: REGULATIONS_GITHUB_PRIVATE_KEY
valueFrom:
secretKeyRef:
name: regulations-github
key: private-key
optional: true
- name: REGULATIONS_GITHUB_REPO
value: "PoliNetworkOrg/legal"
- name: REGULATIONS_GITHUB_BASE
value: "main"
- name: REGULATIONS_LLM_PROVIDER
value: "" # TODO: "anthropic" (the only provider implemented) once the key exists
- name: REGULATIONS_LLM_MODEL
value: "" # TODO
- name: REGULATIONS_LLM_API_KEY
valueFrom:
secretKeyRef:
name: regulations-llm
key: api-key
optional: true
- name: REGULATIONS_TELEGRAM_CHAT_ID
value: "" # TODO: Soci group chat id
- name: REGULATIONS_TELEGRAM_TOPIC_ID
value: "" # TODO: Soci group topic id
- name: REGULATIONS_EMAIL_SENDER
value: "noreply@polinetwork.org" # TODO: confirm the final sender
- name: REGULATIONS_EMAIL_TO
value: "soci@polinetwork.org"
- name: REGULATIONS_EMAIL_REPLY_TO
value: "direttivo@polinetwork.org"
- name: REGULATIONS_DIRETTIVO_EMAIL
value: "direttivo@polinetwork.org"
startupProbe:
tcpSocket:
port: 3000
Expand Down
1 change: 1 addition & 0 deletions apps/backend/kustomization.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -7,4 +7,5 @@ resources:
- namespace.yaml
- external-secret.yaml
- deployment.yaml
- public-route.yaml
- network-policy.yaml
11 changes: 10 additions & 1 deletion apps/backend/network-policy.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,10 @@
# dashboard and the website (tRPC, esign agent proxy), the Telegram bot (tRPC
# and its socket), the IdP (access-changed events on /internal/events) and
# Kuvasz (uptime monitoring).
# Nothing reaches it from outside the cluster since the Ingress was removed.
# From outside the cluster, Traefik reaches it only through the backend-public
# IngressRoute (public-route.yaml), which matches the declared /public prefix
# and nothing else; a NetworkPolicy cannot filter paths, so that route is the
# guard.
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
Expand Down Expand Up @@ -45,6 +48,12 @@ spec:
podSelector:
matchLabels:
app: kuvasz
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: kube-system
podSelector:
matchLabels:
app.kubernetes.io/name: traefik
ports:
- port: 3000
protocol: TCP
80 changes: 80 additions & 0 deletions apps/backend/public-route.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
# The backend's only public surface: its declared /public prefix (regulations
# spec, "Public backend routes"), on api.polinetwork.org through the tunnel.
# Every other path, on any host, stays in-cluster only (#28): this is the only
# router that targets backend-service, and the test suite enforces it.
#
# The match is a whitelist: after Traefik's own path sanitizing, the path must
# be /public followed by plain segments of letters, digits, "-" and "_". Dots,
# "%" (encoded characters), empty segments and a trailing slash never match,
# so /public/../api or /public/%2e%2e/api cannot reach another backend route.
# The backend checks again, authenticates every route itself (GitHub HMAC) and
# keeps its own body and rate limits. Bun resolves ".." and "%2e%2e" before the
# app sees the path, so the backend can't spot an escape from the path alone:
# public-ingress-marker stamps every request from this route, overwriting any
# client value, and the backend lets a stamped request reach only its declared
# /public routes.
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
name: backend-public
spec:
entryPoints: [web]
routes:
- kind: Rule
match: Host(`api.polinetwork.org`) && PathPrefix(`/public/`) && PathRegexp(`^/public(/[A-Za-z0-9_-]+)+$`)
middlewares:
- name: public-rate-limit-total
- name: public-rate-limit-client
- name: public-body-limit
- name: public-ingress-marker
services:
- name: backend-service
port: 3000
---
# GitHub webhook payloads are a few tens of KiB; refuse anything over 1 MiB
# with 413 before it reaches the backend.
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
name: public-body-limit
spec:
buffering:
maxRequestBodyBytes: 1048576
memRequestBodyBytes: 1048576
---
# Per visitor: Cloudflare sets Cf-Connecting-Ip at the edge (see README,
# Traffic); the TCP peer is always a cloudflared Pod.
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
name: public-rate-limit-client
spec:
rateLimit:
average: 1
period: 1s
burst: 20
sourceCriterion:
requestHeaderName: Cf-Connecting-Ip
---
# Whole prefix: every request shares the api.polinetwork.org bucket.
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
name: public-rate-limit-total
spec:
rateLimit:
average: 5
period: 1s
burst: 50
sourceCriterion:
requestHost: true
---
# Set, never appended: Traefik replaces whatever the client sent.
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
name: public-ingress-marker
spec:
headers:
customRequestHeaders:
X-PoliNetwork-Ingress: "public"
133 changes: 133 additions & 0 deletions apps/latex-worker/deployment.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,133 @@
apiVersion: v1
kind: Service
metadata:
name: latex-worker
spec:
selector:
app: latex-worker
ports:
- name: http
port: 4200
targetPort: 4200
---
# Compiles regulation drafts with xelatex for the backend and collab. It runs untrusted
# LaTeX, so it has no network egress, no service account token, a read-only
# root and a size-limited tmpfs workdir. The worker runs each xelatex in a clean
# environment it builds itself: no shell escape, openin_any/openout_any paranoid,
# caches under WORKER_TMP_DIR, and an address-space prlimit. One pod with a small internal pool:
# warm per-draft workdirs (.aux) live in its memory, and the node has 2 vCPUs.
apiVersion: apps/v1
kind: Deployment
metadata:
name: latex-worker
spec:
replicas: 1
strategy:
type: Recreate
selector:
matchLabels:
app: latex-worker
template:
metadata:
labels:
app: latex-worker
spec:
automountServiceAccountToken: false
enableServiceLinks: false
securityContext:
runAsNonRoot: true
# A uid no other workload uses (not the image's 1000): the worker caps
# each xelatex with prlimit --nproc, and RLIMIT_NPROC counts every task
# of the uid on the node, so sharing 1000 with Bun and Node apps made
# xelatex unable to fork xdvipdfmx (tested: compiles fail at ~90 tasks).
runAsUser: 10200
runAsGroup: 10200
fsGroup: 10200
seccompProfile:
type: RuntimeDefault
containers:
- name: latex-worker
image: ghcr.io/polinetworkorg/latex-worker:latest
imagePullPolicy: IfNotPresent
ports:
- containerPort: 4200
env:
- name: NODE_ENV
value: "production"
- name: TZ
value: "Europe/Rome"
- name: WORKER_PORT
value: "4200"
- name: WORKER_SECRET
valueFrom:
secretKeyRef:
name: regulations
key: worker-secret
- name: WORKER_WORK_DIR
value: "/work"
- name: WORKER_TMP_DIR
value: "/tmp"
# Parallel xelatex runs; a third only contends for the same 1.5 CPUs.
- name: WORKER_CONCURRENCY
value: "2"
- name: WORKER_TIMEOUT_MS
value: "45000"
# Per-xelatex prlimit; real usage is ~200Mi per run, see resources.
- name: WORKER_MEMORY_LIMIT_MB
value: "1024"
# Warm workdirs kept on /work (~8Mi each with the RRI).
- name: WORKER_MAX_WORKDIRS
value: "40"
startupProbe:
httpGet:
path: /health
port: 4200
periodSeconds: 2
failureThreshold: 30
readinessProbe:
httpGet:
path: /health
port: 4200
periodSeconds: 5
timeoutSeconds: 2
failureThreshold: 3
livenessProbe:
httpGet:
path: /health
port: 4200
periodSeconds: 10
timeoutSeconds: 2
failureThreshold: 3
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
capabilities:
drop: [ALL]
# Measured on the RRI (19 pages) in this sandbox: a cold latexmk takes
# 1.8 s, a warm pass 0.4 s, 3 parallel passes peak at 650Mi including
# tmpfs. Memory-backed volumes count against the memory limit, so the
# limit covers a full 512Mi /work too. CPU stays below the node's
# 2 vCPUs so a compile burst cannot starve the other workloads.
resources:
requests:
cpu: "250m"
memory: "768Mi"
ephemeral-storage: "64Mi"
limits:
cpu: "1500m"
memory: "1536Mi"
ephemeral-storage: "256Mi"
volumeMounts:
- name: work
mountPath: /work
- name: tmp
mountPath: /tmp
volumes:
- name: work
emptyDir:
medium: Memory
sizeLimit: 512Mi
- name: tmp
emptyDir:
medium: Memory
sizeLimit: 64Mi
10 changes: 10 additions & 0 deletions apps/latex-worker/kustomization.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

# The namespace and the worker secret belong to apps/regulations-secrets.
namespace: latex-worker

resources:
- limits.yaml
- deployment.yaml
- network-policy.yaml
38 changes: 38 additions & 0 deletions apps/latex-worker/limits.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# Caps the namespace so a runaway compile cannot starve the single node: no
# container above the worker's own limits, and no more than one worker's worth
# of resources in total (the Deployment uses Recreate, never two pods at once).
apiVersion: v1
kind: LimitRange
metadata:
name: latex-worker
spec:
limits:
- type: Container
max:
cpu: "1500m"
memory: "1536Mi"
ephemeral-storage: "256Mi"
default:
cpu: "500m"
memory: "512Mi"
ephemeral-storage: "128Mi"
defaultRequest:
cpu: "50m"
memory: "128Mi"
ephemeral-storage: "64Mi"
---
apiVersion: v1
kind: ResourceQuota
metadata:
name: latex-worker
spec:
hard:
pods: "2"
requests.cpu: "500m"
requests.memory: "768Mi"
requests.ephemeral-storage: "128Mi"
limits.cpu: "1500m"
limits.memory: "1536Mi"
limits.ephemeral-storage: "256Mi"
services.loadbalancers: "0"
services.nodeports: "0"
Loading
Loading