Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
192 changes: 96 additions & 96 deletions docs/index.md
Original file line number Diff line number Diff line change
@@ -1,142 +1,142 @@
# Gthulhu

<div class="gth-hero" markdown>

## Keep critical workloads fast when CPUs get busy

Gthulhu is a cloud-native runtime scheduling platform built with eBPF and Linux `sched_ext`. It protects latency- and throughput-sensitive Linux tasks inside Kubernetes workloads when CPU contention would otherwise slow them down.

<div class="gth-hero-actions">
<a href="k8s.md" class="md-button md-button--primary">Get Started</a>
<a href="https://github.com/Gthulhu/Gthulhu" class="md-button">View on GitHub</a>
</div>

</div>

<div class="gth-proof-grid" markdown>

<div class="gth-proof-card" markdown>
<span class="gth-proof-eyebrow">free5GC / 5G user plane</span>

### 97.66% lower average latency

**88.98 ms → 2.079 ms** average UE ping latency under CPU stress.

Maximum latency also fell from **130.95 ms → 8.45 ms**.

[Read the published free5GC case study →](https://free5gc.org/blog/20251126/20251126/)
</div>

<div class="gth-proof-card" markdown>
<span class="gth-proof-eyebrow">vLLM / GPU inference</span>

### ~3.2× decode throughput

**~6.7 t/s → ~21.3 t/s** on `tg128` under CPU pressure with Gthulhu + tiered scheduling policy.

*Reproducible community benchmark currently under upstream vLLM blog review.*

[Review the benchmark and methodology →](https://github.com/vllm-project/vllm-project.github.io/pull/300)
</div>

</div>

<div class="gth-trust-row" markdown>
<a href="https://landscape.cncf.io/?item=provisioning--automation-configuration--gthulhu" target="_blank"><img src="https://img.shields.io/badge/CNCF%20Landscape-5699C6?style=for-the-badge&logo=cncf&label=cncf" alt="cncf landscape" /></a>
<a href="https://ebpf.io/applications/" target="_blank"><img src="https://img.shields.io/badge/eBPF%20Application%20Landscape-5699C6?style=for-the-badge&logo=ebpf&label=ebpf" alt="ebpf landscape" /></a>
<a href="https://insights.linuxfoundation.org/project/gthulhu"><img src="https://insights.linuxfoundation.org/api/badge/health-score?project=gthulhu" alt="LFX Health Score" /></a>
Comment on lines 45 to +47
</div>

[![LFX Health Score](https://insights.linuxfoundation.org/api/badge/health-score?project=gthulhu)](https://insights.linuxfoundation.org/project/gthulhu)
## The problem Gthulhu solves

# Gthulhu
Kubernetes can place a workload on the right node and allocate the right resources. That still does not guarantee the workload's critical Linux threads will get CPU time when they need it.

## Protect critical workloads from CPU contention
Under contention, GPU feeder threads, `EngineCore`, packet-processing workers, IRQ-related work, or other latency-sensitive tasks can be delayed by background CPU load. The result is simple: **allocated resources, but missed SLOs**.

Gthulhu uses eBPF and Linux `sched_ext` to make workload scheduling observable and controllable across Kubernetes nodes.
Gthulhu closes that execution gap.

**Measured results under CPU contention:**
<div class="gth-flow" markdown>

| Workload | Baseline | With Gthulhu | Result |
|---|---:|---:|---:|
| free5GC / GTP data path — average UE ping latency | **88.98 ms** | **2.079 ms** | **97.66% lower** |
| free5GC / GTP data path — maximum UE ping latency | **130.95 ms** | **8.45 ms** | **93.55% lower** |
| vLLM Qwen2.5-0.5B decode (`tg128`) under CPU pressure | **~6.7 t/s** | **~21.3 t/s** with tiered policy | **~3.2× throughput** |
**1. Observe**
Use eBPF to see which tasks are waiting, running, migrating, and competing for CPU.

The free5GC numbers are published in the free5GC community blog. The vLLM result comes from a reproducible community benchmark currently proposed to the vLLM project blog; treat it as an experiment under upstream review rather than a project-wide guarantee.
**2. Target**
Resolve Kubernetes workload intent down to the Linux process or thread that actually matters.

[Read the free5GC case study](https://free5gc.org/blog/20251126/20251126/){: .md-button .md-button--primary }
[vLLM benchmark PR](https://github.com/vllm-project/vllm-project.github.io/pull/300){: .md-button }
[Get Started](k8s.md){: .md-button }
**3. Control**
Use `sched_ext` to apply bounded runtime scheduling policy and protect critical execution paths.

> **DRA chooses what and where; Gthulhu controls how it actually runs.**
</div>

## Why this matters
> **DRA chooses what and where. Gthulhu controls how it actually runs.**

A workload can already own a GPU, NIC, CPU set, or Kubernetes placement and still miss its latency or throughput target because its host-side Linux tasks are delayed by CPU contention.
## Built for workload-aware runtime scheduling

Gthulhu focuses on that execution gap:
<div class="grid cards" markdown>

```text
Kubernetes admission / placement / allocation
│
▼
Gthulhu runtime plane
│
Pod / cgroup / TGID / TID resolution
│
▼
sched_ext + eBPF
│
▼
latency / throughput / jitter / SLO
```
- :material-eye-outline:{ .lg .middle } **Scheduling observability**

## Proven use cases
---

### 5G user-plane latency
Pod-level scheduling metrics with eBPF, plus Prometheus and Grafana integration.

The free5GC community published a GTP-driven scheduling experiment that combines `gtp5g-tracer`, a userspace operator, and Gthulhu. Under the same CPU stress, average UE ping latency dropped from **88.98 ms to 2.079 ms**, while maximum latency dropped from **130.95 ms to 8.45 ms**.
- :material-tune-variant:{ .lg .middle } **Fine-grained control**

[Read: Implementing GTP-driven Automatic Scheduling Optimization with eBPF-based Scheduler](https://free5gc.org/blog/20251126/20251126/)
---

A separate earlier free5GC case study also documents using Gthulhu to reduce RTT by combining application/domain knowledge with custom `sched_ext` policy.
Apply scheduling intent to specific workloads, processes, or non-leader worker threads with TID-aware matching.

[Read: Improving Network Performance with Custom eBPF-based Schedulers](https://free5gc.org/blog/20250726/index.en/)
- :material-server-network:{ .lg .middle } **Cloud-native operation**

### vLLM inference under CPU pressure
---

A reproducible DGX Spark / GB10 experiment uses MicroK8s, vLLM, `stress-ng`, and Gthulhu to isolate the effect of CPU scheduling on GPU inference. In the submitted benchmark, decode throughput under CPU pressure is around **6–7 t/s** with the default scheduler and reaches roughly **21 t/s** on `tg128` with Gthulhu plus tiered policies targeting GPU-related work and vLLM's `EngineCore` thread.
Manager + per-node Decision Makers distribute scheduling intent across Kubernetes nodes.

This result is linked here as an **upstream-reviewing community benchmark**, not as a generalized performance guarantee.
- :material-chart-line:{ .lg .middle } **SLO-oriented automation**

[Review the benchmark and methodology in vLLM blog PR #300](https://github.com/vllm-project/vllm-project.github.io/pull/300)
---

## What Gthulhu provides today
Feed scheduler signals into Prometheus, Grafana, and KEDA to support runtime-aware operations and scaling.

- **Pod-level scheduling observability** with eBPF.
- **Prometheus / Grafana / KEDA integration** for scheduler-aware operations and scaling.
- **Distributed scheduling intent** through a Manager and per-node Decision Makers.
- **Custom CPU scheduling** on Linux 6.12+ with `sched_ext`.
- **TID-aware node-policy matching** so non-leader worker threads can be targeted directly.
- **Explicit priority semantics** across user-space and kernel scheduler modes.
</div>

[How It Works](how-it-works.md){: .md-button }
[Claim2Core Roadmap](claim2core.md){: .md-button }
[See how Gthulhu works](how-it-works.md){: .md-button }

## Claim2Core: from allocation to delivered performance
## Where Gthulhu is going: Claim2Core

The next architecture step is to connect actual Kubernetes allocation to runtime task scheduling:
Today, Gthulhu can observe and control Linux task scheduling at runtime. The next step is to connect that control directly to Kubernetes' **actual resource allocation**.

```text
Kueue / Workload API
│ admission / quota
▼
↓
kube-scheduler / DRA
│ Node + device + topology allocation
▼
↓ ResourceClaim + topology
Gthulhu Runtime Plane
│ ResourceClaim → Pod/cgroup → TGID/TID
▼
↓ Pod / cgroup / TGID / TID
sched_ext + eBPF
│ runtime policy + verification
▼
↓
Delivered workload SLO
```

The critical correctness rule is:
The principle is simple:

- `ResourceSlice` is **inventory**.
- `ResourceClaim.status.allocation` is the workload's **actual allocation**.
- `ResourceSlice` tells us what resources exist.
- `ResourceClaim.status.allocation` tells us what the workload actually received.
- Gthulhu turns that allocation into a verifiable runtime execution policy **without crossing the CPU/resource boundaries Kubernetes already established**.

Gthulhu should not reimplement kube-scheduler, DRA, or Kueue. It should consume their decisions and control Linux CPU execution **inside** the resource envelope established by Kubernetes/cgroups.
[Explore the Claim2Core roadmap](claim2core.md){: .md-button }
[Follow roadmap issue #141](https://github.com/Gthulhu/Gthulhu/issues/141){: .md-button }

Read [Claim2Core](claim2core.md) for the implementation phases and safety boundaries.
## Start with a real workload

## Architecture at a glance
<div class="gth-cta" markdown>

```text
User / Web UI / CRD
│
▼
Manager API ───────▶ MongoDB / Kubernetes API
│
▼
Decision Maker DaemonSet
│
├── eBPF scheduling metrics collector ──▶ Prometheus / Grafana / KEDA
│
└── task resolution / scheduling intent
│
▼
Gthulhu daemon
│
▼
sched_ext / BPF
│
▼
Linux scheduler
```
### See what CPU scheduling is doing to your workload

Deploy Gthulhu on Kubernetes, inspect scheduler behavior, then apply policy only where the data shows it matters.

## Get involved
[Deploy Gthulhu](k8s.md){: .md-button .md-button--primary }
[Read the free5GC case study](https://free5gc.org/blog/20251126/20251126/){: .md-button }
[Contribute](contributing.md){: .md-button }

- [Deploy Gthulhu with Kubernetes](k8s.md)
- [Understand the architecture and scheduler semantics](how-it-works.md)
- [Read the Claim2Core roadmap](claim2core.md)
- [Contribute](contributing.md)
- [GitHub repository](https://github.com/Gthulhu/Gthulhu)
- [Roadmap issue #141](https://github.com/Gthulhu/Gthulhu/issues/141)
</div>
Loading