Skip to content

Commit 2e976bf

Browse files
jpascucci-nvjpascucci-nv
authored
ci: generate ci pipeline diagram from Mermaid (#2957)
* ci: generate ci pipeline diagram from Mermaid Closes #2740 Description Replace the manually maintained `ci/ci-pipeline.svg` with a reviewable Mermaid source file. - Add `ci/ci-pipeline.mmd` as the canonical diagram source. - Add a command-line tool to regenerate or validate the committed SVG using Mermaid CLI 12.0.0. - Remove the old agent-oriented SVG regeneration instructions. - Update `CONTRIBUTING.md` with the regeneration command. - Add a focused CI job that detects stale generated output. - Update the CI work planner so diagram-only changes run the diagram check without triggering the full CUDA build and test matrix. Regenerate the diagram with: ```console python ci/tools/generate_ci_pipeline.py Validate it without modifying the committed SVG with: python ci/tools/generate_ci_pipeline.py --check Testing - python3 ci/tools/generate_ci_pipeline.py --check - python3 -m unittest ci/tools/tests/test_compute_ci_plan.py - pixi run ruff check ci/tools/generate_ci_pipeline.py ci/tools/compute_ci_plan.py ci/tools/tests/test_compute_ci_plan.py - pixi run ruff format --check ci/tools/generate_ci_pipeline.py ci/tools/compute_ci_plan.py ci/tools/tests/test_compute_ci_plan.py - SPDX validation - GitHub Actions workflow YAML validation - git diff --check * Make it look more like the original * Make boxes more like original * Final newline issue. * Workaround ci failures with puppeteer config. * Try to pin mermaid version with docker * Remove machinery to generate .svg. * Try to make it more readable.
1 parent 5fbfc6d commit 2e976bf

5 files changed

Lines changed: 100 additions & 361 deletions

File tree

‎CONTRIBUTING.md‎

Lines changed: 1 addition & 82 deletions
Original file line numberDiff line numberDiff line change
@@ -317,88 +317,7 @@ The CUDA Python project uses a comprehensive CI pipeline that builds, tests, and
317317

318318
### CI Pipeline Flow
319319

320-
![CUDA Python CI Pipeline Flow](ci/ci-pipeline.svg)
321-
322-
Alternative Mermaid diagram representation:
323-
324-
```mermaid
325-
flowchart TD
326-
%% Trigger Events
327-
subgraph TRIGGER["🔄 TRIGGER EVENTS"]
328-
T1["• Push to main branch"]
329-
T2["• Pull request<br/>• Manual workflow dispatch"]
330-
T1 --- T2
331-
end
332-
333-
%% Build Stage
334-
subgraph BUILD["🔨 BUILD STAGE"]
335-
subgraph BUILD_PLATFORMS["Parallel Platform Builds"]
336-
B1["linux-64<br/>(Self-hosted)"]
337-
B2["linux-aarch64<br/>(Self-hosted)"]
338-
B3["win-64<br/>(GitHub-hosted)"]
339-
end
340-
BUILD_DETAILS["• Python versions: 3.10, 3.11, 3.12, 3.13, 3.14<br/>• CUDA version: 13.0.0 (build-time)<br/>• Components: cuda-core, cuda-bindings,<br/> cuda-pathfinder, cuda-python"]
341-
end
342-
343-
%% Artifact Storage
344-
subgraph ARTIFACTS["📦 ARTIFACT STORAGE"]
345-
subgraph GITHUB_ARTIFACTS["GitHub Artifacts"]
346-
GA1["• Wheel files (.whl)<br/>• Test artifacts<br/>• Documentation<br/>(30-day retention)"]
347-
end
348-
subgraph GITHUB_CACHE["GitHub Cache"]
349-
GC1["• Mini CTK cache"]
350-
end
351-
end
352-
353-
%% Test Stage
354-
subgraph TEST["🧪 TEST STAGE"]
355-
subgraph TEST_PLATFORMS["Parallel Platform Tests"]
356-
TS1["linux-64<br/>(Self-hosted)"]
357-
TS2["linux-aarch64<br/>(Self-hosted)"]
358-
TS3["win-64<br/>(GitHub-hosted)"]
359-
end
360-
TEST_DETAILS["• Download wheels from artifacts<br/>• Test against multiple CUDA runtime versions<br/>• Run Python unit tests, Cython tests, examples"]
361-
ARTIFACT_FLOWS["Artifact Flows:<br/>• cuda-pathfinder: main → backport<br/>• cuda-bindings: backport → main"]
362-
end
363-
364-
%% Release Pipeline
365-
subgraph RELEASE["🚀 RELEASE PIPELINE"]
366-
subgraph RELEASE_STAGES["Sequential Release Steps"]
367-
R1["Validation<br/>• Artifact integrity<br/>• Git tag verification"]
368-
R2["Publishing<br/>• PyPI/TestPyPI<br/>• Component or all releases"]
369-
R3["Documentation<br/>• GitHub Pages<br/>• Release notes"]
370-
R1 --> R2 --> R3
371-
end
372-
RELEASE_DETAILS["• Manual workflow dispatch with run ID<br/>• Supports individual component or full releases"]
373-
end
374-
375-
%% Main Flow
376-
TRIGGER --> BUILD
377-
BUILD -.->|"wheel upload"| ARTIFACTS
378-
ARTIFACTS -.-> TEST
379-
TEST --> RELEASE
380-
381-
%% Artifact Flow Arrows (Cache Reuse)
382-
GITHUB_CACHE -.->|"mini CTK reuse"| BUILD
383-
GITHUB_CACHE -.->|"mini CTK reuse"| TEST
384-
385-
%% Artifact Flow Arrows (Wheel Fetch)
386-
GITHUB_ARTIFACTS -.->|"wheel fetch"| TEST
387-
GITHUB_ARTIFACTS -.->|"wheel fetch"| RELEASE
388-
389-
%% Styling
390-
classDef triggerStyle fill:#e8f4fd,stroke:#2196F3,stroke-width:2px,color:#1976D2
391-
classDef buildStyle fill:#f3e5f5,stroke:#9C27B0,stroke-width:2px,color:#7B1FA2
392-
classDef artifactStyle fill:#fff3e0,stroke:#FF9800,stroke-width:2px,color:#F57C00
393-
classDef testStyle fill:#e8f5e8,stroke:#4CAF50,stroke-width:2px,color:#388E3C
394-
classDef releaseStyle fill:#ffebee,stroke:#f44336,stroke-width:2px,color:#D32F2F
395-
396-
class TRIGGER,T1,T2 triggerStyle
397-
class BUILD,BUILD_PLATFORMS,B1,B2,B3,BUILD_DETAILS buildStyle
398-
class ARTIFACTS,GITHUB_ARTIFACTS,GITHUB_CACHE,GA1,GC1 artifactStyle
399-
class TEST,TEST_PLATFORMS,TS1,TS2,TS3,TEST_DETAILS,ARTIFACT_FLOWS testStyle
400-
class RELEASE,RELEASE_STAGES,R1,R2,R3,RELEASE_DETAILS releaseStyle
401-
```
320+
The CI pipeline diagram is maintained as Mermaid source in [`ci/ci-pipeline.mmd`](ci/ci-pipeline.mmd).
402321

403322
### Pipeline Execution Details
404323

‎ci/.ci-pipeline-regen.md‎

Lines changed: 0 additions & 106 deletions
This file was deleted.

‎ci/ci-pipeline.mmd‎

Lines changed: 99 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,99 @@
1+
---
2+
# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
3+
# SPDX-License-Identifier: Apache-2.0
4+
title: CUDA Python CI Pipeline Flow
5+
config:
6+
theme: base
7+
look: handDrawn
8+
layout: elk
9+
handDrawnSeed: 2740
10+
deterministicIds: true
11+
deterministicIDSeed: cuda-python-ci-pipeline
12+
htmlLabels: true
13+
themeVariables:
14+
fontFamily: "Comic Sans MS, Comic Sans, cursive"
15+
fontSize: 14px
16+
flowchart:
17+
curve: basis
18+
diagramPadding: 8
19+
minNodeWidth: 140
20+
nodeSpacing: 24
21+
padding: 10
22+
rankSpacing: 48
23+
wrappingWidth: 300
24+
---
25+
flowchart TB
26+
subgraph TRIGGER["TRIGGER EVENTS"]
27+
direction LR
28+
T1["Push to main branch"] ~~~ T2["Pull request"] ~~~ T3["Manual workflow dispatch"]
29+
end
30+
31+
subgraph BUILD["BUILD STAGE"]
32+
direction TB
33+
subgraph BUILD_PLATFORMS["Parallel Platform Builds"]
34+
direction LR
35+
B1["linux-64<br/>(Self-hosted)"] ~~~ B2["linux-aarch64<br/>(Self-hosted)"] ~~~ B3["win-64<br/>(GitHub-hosted)"]
36+
end
37+
BUILD_DETAILS["Python versions: 3.10, 3.11, 3.12, 3.13, 3.14<br/>CUDA version: 13.0.0 (build-time)<br/>Components: cuda-core, cuda-bindings,<br/>cuda-pathfinder, cuda-python"]
38+
end
39+
40+
subgraph ARTIFACTS["ARTIFACT STORAGE"]
41+
direction TB
42+
subgraph GITHUB_ARTIFACTS["GitHub Artifacts"]
43+
GA1["Wheel files (.whl)<br/>Test artifacts<br/>Documentation<br/>(30-day retention)"]
44+
end
45+
subgraph GITHUB_CACHE["GitHub Cache"]
46+
GC1["Mini CTK cache"]
47+
end
48+
end
49+
50+
subgraph TEST["TEST STAGE"]
51+
direction TB
52+
subgraph TEST_PLATFORMS["Parallel Platform Tests"]
53+
direction LR
54+
TS1["linux-64<br/>(Self-hosted)"] ~~~ TS2["linux-aarch64<br/>(Self-hosted)"] ~~~ TS3["win-64<br/>(GitHub-hosted)"]
55+
end
56+
TEST_DETAILS["Download wheels from artifacts<br/>Test against multiple CUDA runtime versions<br/>Run Python unit tests, Cython tests, and examples"]
57+
ARTIFACT_FLOWS["Artifact Flows:<br/>cuda-pathfinder: main to backport<br/>cuda-bindings: backport to main"]
58+
end
59+
60+
subgraph RELEASE["RELEASE PIPELINE"]
61+
direction TB
62+
subgraph RELEASE_STAGES["Sequential Release Steps"]
63+
direction LR
64+
R1["Validation<br/>Artifact integrity<br/>Git tag verification"] --> R2["Publishing<br/>PyPI/TestPyPI<br/>Component or all releases"] --> R3["Documentation<br/>GitHub Pages<br/>Release notes"]
65+
end
66+
RELEASE_DETAILS["Manual workflow dispatch with run ID<br/>Supports individual component or full releases"]
67+
end
68+
69+
TRIGGER --> BUILD
70+
BUILD -. "wheel upload" .-> ARTIFACTS
71+
ARTIFACTS --> TEST
72+
TEST --> RELEASE
73+
74+
GITHUB_CACHE -. "mini CTK reuse" .-> BUILD
75+
GITHUB_CACHE -. "mini CTK reuse" .-> TEST
76+
GITHUB_ARTIFACTS -. "wheel fetch" .-> TEST
77+
GITHUB_ARTIFACTS -. "wheel fetch" .-> RELEASE
78+
79+
classDef triggerStyle fill:#e8f4fd,stroke:#2196F3,stroke-width:2px,color:#1976D2
80+
classDef triggerNodeStyle fill:#ffffff,stroke:#2196F3,stroke-width:2px,color:#333333
81+
classDef buildStyle fill:#f3e5f5,stroke:#9C27B0,stroke-width:2px,color:#7B1FA2
82+
classDef buildNodeStyle fill:#ffffff,stroke:#9C27B0,stroke-width:2px,color:#333333
83+
classDef artifactStyle fill:#fff3e0,stroke:#FF9800,stroke-width:2px,color:#F57C00
84+
classDef artifactNodeStyle fill:#ffffff,stroke:#FF9800,stroke-width:2px,color:#333333
85+
classDef testStyle fill:#e8f5e8,stroke:#4CAF50,stroke-width:2px,color:#388E3C
86+
classDef testNodeStyle fill:#ffffff,stroke:#4CAF50,stroke-width:2px,color:#333333
87+
classDef releaseStyle fill:#ffebee,stroke:#f44336,stroke-width:2px,color:#D32F2F
88+
classDef releaseNodeStyle fill:#ffffff,stroke:#f44336,stroke-width:2px,color:#333333
89+
90+
class TRIGGER triggerStyle
91+
class T1,T2,T3 triggerNodeStyle
92+
class BUILD,BUILD_PLATFORMS buildStyle
93+
class B1,B2,B3,BUILD_DETAILS buildNodeStyle
94+
class ARTIFACTS,GITHUB_ARTIFACTS,GITHUB_CACHE artifactStyle
95+
class GA1,GC1 artifactNodeStyle
96+
class TEST,TEST_PLATFORMS testStyle
97+
class TS1,TS2,TS3,TEST_DETAILS,ARTIFACT_FLOWS testNodeStyle
98+
class RELEASE,RELEASE_STAGES releaseStyle
99+
class R1,R2,R3,RELEASE_DETAILS releaseNodeStyle

0 commit comments

Comments
 (0)