diff --git a/profile/README.md b/profile/README.md index fe257b7..906b950 100644 --- a/profile/README.md +++ b/profile/README.md @@ -1,56 +1,75 @@ -# πŸ™ Gthulhu: Steering the Cloud-Native Depthstext - ^(;,;)^ -The Orchestrable Heart of Linux Scheduling - - -[![CNCF Landscape](https://img.shields.io/badge/CNCF-Landscape-blue?style=for-the-badge&logo=cncf)](https://landscape.cncf.io/)[![](https://img.shields.io/badge/eBPF-Application-orange?style=for-the-badge&logo=ebpf)](https://ebpf.io/applications/)[![Go Version](https://img.shields.io/badge/Go-1.22+-00ADD8?style=for-the-badge&logo=go&logoColor=white)](https://go.dev/)[![License](https://img.shields.io/badge/License-Apache_2.0-green.svg?style=for-the-badge)](https://opensource.org/licenses/Apache-2.0) - -**Gthulhu** is an orchestrable, distributed scheduler platform designed for the cloud-native ecosystem. By leveraging the **Linux Scheduler Extension (sched_ext)** and **eBPF**, Gthulhu enables developers to bypass the limitations of traditional fair-share scheduling (CFS/EEVDF) and optimize for specific, high-performance workloads like trading systems, big data analytics, and 5G network processing. - ---- - -## πŸ›οΈ Philosophy: Technical Prowess & Cultural Roots - -- **The Name**: **Gthulhu** is a portmanteau of **Go** and **Cthulhu**. Its "tentacles" symbolize the ability to grasp and steer complex distributed systems, much like the ship's wheel (Helm) of Kubernetes. -- **The Core**: Our underlying framework, **qumun**, is named after the **Bunun** (Indigenous people of Taiwan) word for "**Heart**". Just as the heart drives the rhythm of life, qumun serves as the heartbeat of the operating system, orchestrating tasks and resource cycles. - ---- - -## πŸ—οΈ Architecture: Distributed Control Plane - -Gthulhu ensures coordinated scheduling across a multi-node cluster through three key layers: - -1. **Manager (Global Plane)**: Central control point that transforms high-level "scheduling intents" (via REST API) into actionable policies. -2. **Decision Maker (Local Plane)**: A sidecar running on each node that identifies target processes (PIDs) and maps global policies to local execution. -3. **Scheduler Agent & qumun (Execution Plane)**: The eBPF-powered agent that interacts directly with the Linux kernel to apply millisecond-level scheduling decisions. - ---- - -## πŸ—ΊοΈ Repository Map - -| Repository | Role | Tech Stack | -| :--- | :--- | :--- | -| [**Gthulhu/Gthulhu**](https://github.com/Gthulhu/Gthulhu) | **Main Repo**: The orchestrable distributed scheduler core. | Go, C, eBPF | -| [**Gthulhu/qumun**](https://github.com/Gthulhu/qumun) | **Framework**: Go interface for custom Linux schedulers. | C, Go | -| [**Gthulhu/api**](https://github.com/Gthulhu/api) | **Interface**: Communication protocols and policy definitions. | Go (REST) | -| [**Gthulhu/plugin**](https://github.com/Gthulhu/plugin) | **Strategies**: Pre-built user-space scheduling plugins. | Go | -| [**Gthulhu/chart**](https://github.com/Gthulhu/chart) | **Deployment**: Helm charts for K8s integration. | Helm | -| [**Gthulhu/mcp**](https://github.com/Gthulhu/mcp) | **Future**: AI-driven scheduling via MCP protocol (Note: mcp is currently archived, the gthulhu will embed AI-powered assistant in R2). | TypeScript | - ---- - -## πŸš€ Optimized Workloads - -- βš–οΈ **Low-Latency**: Financial trading systems and gaming servers requiring zero-preemption. -- πŸ“Š **High-Throughput**: Big data (Spark) and Machine Learning training to maximize CPU cycles. -- πŸ“Ά **5G Telecom**: Optimized Data Plane packet processing to reach the URLLC. - ---- - -## 🀝 Join the Cult(ure) - -We welcome acolytes from all corners of the open-source world! - -- **Documentation**: Visit [gthulhu.org](https://gthulhu.org) for technical deep dives. -- **Prerequisites**: Ensure you are running **Linux Kernel 6.12+** with `sched_ext` enabled. -- **Contribute**: Feel free to open [Issues](https://github.com/Gthulhu/Gthulhu/issues) or join our(https://github.com/Gthulhu/Gthulhu/discussions). +# πŸ™ Gthulhu + +**From Kubernetes resource allocation to Linux task scheduling.** + +[![CNCF Landscape](https://img.shields.io/badge/CNCF-Landscape-blue?style=for-the-badge&logo=cncf)](https://landscape.cncf.io/) +[![eBPF Application](https://img.shields.io/badge/eBPF-Application-orange?style=for-the-badge&logo=ebpf)](https://ebpf.io/applications/) +[![License](https://img.shields.io/badge/License-Apache_2.0-green.svg?style=for-the-badge)](https://opensource.org/licenses/Apache-2.0) + +Gthulhu is a cloud-native runtime scheduling project built around Kubernetes, eBPF, and Linux `sched_ext`. + +Our current direction is **Claim2Core**: + +> **DRA chooses what and where; Gthulhu controls how it actually runs.** + +Kubernetes can allocate a workload to a Node, GPU, NIC, CPU set, NUMA domain, or other device. Gthulhu focuses on the execution gap that follows: mapping workload intent and actual allocation to the Linux tasks that need CPU service, then applying bounded runtime policy through `sched_ext` and verifying the result with eBPF telemetry. + +```text +Kueue / Workload API + β”‚ admission / quota + β–Ό +kube-scheduler / DRA + β”‚ Node + device + topology allocation + β–Ό +Gthulhu Runtime Plane + β”‚ Claim β†’ Pod/cgroup β†’ TGID/TID + β–Ό +sched_ext + eBPF + β”‚ runtime policy + verification + β–Ό +Delivered workload SLO +``` + +## What exists today + +- pod-level scheduling observability with eBPF; +- Prometheus / Grafana / KEDA integration; +- distributed scheduling intents through a Manager and per-node Decision Makers; +- custom CPU scheduling on Linux 6.12+ with `sched_ext`; +- TID-aware node-policy matching for non-leader worker threads; +- user-space and kernel-mode priority handling with explicit non-boosting semantics. + +The next roadmap steps focus on DRA semantic correctness, read-only `ResourceClaim` observation, Claim-to-Task provenance, and a static DRA-aware execution policy. See [Gthulhu 2026 Roadmap β€” Claim2Core](https://github.com/Gthulhu/Gthulhu/issues/141). + +## Repository map + +| Repository | Status | Role | +|---|---|---| +| [**Gthulhu/Gthulhu**](https://github.com/Gthulhu/Gthulhu) | Active | Main runtime/control-plane implementation | +| [**Gthulhu/qumun**](https://github.com/Gthulhu/qumun) | Active | Go framework for custom `sched_ext` schedulers | +| [**Gthulhu/plugin**](https://github.com/Gthulhu/plugin) | Active | User-space scheduling strategy implementation | +| [**Gthulhu/docs**](https://github.com/Gthulhu/docs) | Active | Official documentation at [gthulhu.org](https://gthulhu.org) | +| [**Gthulhu/gtp5g-operator**](https://github.com/Gthulhu/gtp5g-operator) | Active | Telecom / GTP5G integration work | +| [**Gthulhu/kina**](https://github.com/Gthulhu/kina) | Active | Related runtime experimentation | +| [**Gthulhu/libbpfgo**](https://github.com/Gthulhu/libbpfgo) | Active | Project-maintained libbpfgo fork | +| [**Gthulhu/api**](https://github.com/Gthulhu/api) | Archived | Historical standalone API repository; code has moved into the main repository | +| [**Gthulhu/chart**](https://github.com/Gthulhu/chart) | Archived | Historical standalone Helm chart repository; deployment assets now live with the main project | +| [**Gthulhu/mcp**](https://github.com/Gthulhu/mcp) | Archived | Experimental MCP work | + +## Current research / engineering themes + +- **Claim2Core** β€” `ResourceClaim β†’ Pod/cgroup β†’ TGID/TID β†’ sched_ext` execution lineage. +- **Device-local execution domains** β€” respect allocated CPU/cgroup boundaries while preferring NUMA / PCIe locality. +- **Scheduling provenance** β€” preview, explain, intended-vs-actual runtime state, and stale-state verification. +- **Workload adapters** β€” free5GC/UPF first for fast end-to-end validation; LLM prefill/decode/NCCL roles for accelerator-focused research. +- **Closed-loop runtime control** β€” only after static policy and provenance are trustworthy. + +## Join the project + +- **Main repository:** [Gthulhu/Gthulhu](https://github.com/Gthulhu/Gthulhu) +- **Roadmap:** [Issue #141](https://github.com/Gthulhu/Gthulhu/issues/141) +- **Documentation:** [gthulhu.org](https://gthulhu.org) +- **Contributing:** [Contribution guide](https://gthulhu.org/contributing/) +- **Discussions:** [GitHub Discussions](https://github.com/Gthulhu/Gthulhu/discussions) + +Gthulhu is Apache-2.0 licensed and welcomes contributions across Kubernetes, DRA, eBPF, `sched_ext`, NUMA/topology, observability, telecom, and accelerator workloads.