Skip to content

cuda.bindings: configure independent CUDA 12 and CUDA 13 packages - #3174

Draft
rwgk wants to merge 19 commits into
mainfrom
rwgk/cb12-on-main-03-package-selection
Draft

rwgk wants to merge 19 commits into
mainfrom
rwgk/cb12-on-main-03-package-selection

Conversation

@rwgk

@rwgk rwgk commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

— codex GPT-6 on behalf of Ralf —

Layer 3 of 8; first of the review group #3174 + #3175 + #3176, which lands as one merge. Next: #3175. Stack map at the bottom.

What this PR does

Declares the two bindings roots and makes each one own its version, without changing the distribution name or the Python namespace. Both roots still build cuda-bindings / cuda.bindings; they are meant for separate environments.

  • Registry. ci/versions.yml becomes schema 2: cuda_bindings_12 is maintenance (toolkit 12.9.1), cuda_bindings_13 is current (13.4.2), plus the shared Python matrix. backport_branch stays until cuda.bindings: document shared maintenance and release branches #3178 removes its last consumer. ci/tools/bindings_config.py reads it; a new pre-commit hook (check-bindings-config) keeps the two roots' setuptools-scm selectors, the metapackage selector table, and Core's bindings floors consistent with it. ci/ becomes an installable package (pip install -e ./ci).
  • Versions. Standard setuptools-scm parsing with one selector per CUDA major: v12.* and v13.*. The metapackage carries the same table in cuda_python/pyproject.toml; cuda_python/setup.py follows an unambiguous exact CUDA 12 tag and otherwise defaults to CUDA 13; CUDA_PYTHON_BUILD_MAJOR explicitly selects either major, and dependencies follow the resolved version (CUDA 12 depends on bindings only; CUDA 13 also on Core and Pathfinder). A CUDA 12 sdist rebuilt outside Git keeps CUDA 12 dependencies.
  • CUDA 12 packaging. Python >= 3.11 (3.10 dropped, as agreed with Mike), 3.14/3.15 classifiers, dependency groups, wheel payload policy copied from CUDA 13 (wheels ship .pxd/.pxi and headers, not .pyx or .in templates). New Pixi workspace and lockfile (13,185 of this PR's 14,517 added lines are the lockfile).
  • Local source tests. Root test-bindings forwards to the selected bindings workspace. Core's Pixi task collects its owned tests/ tree, including Cython tests, rather than vendored CCCL tests.
  • Guidance. Root and package AGENTS.md (a change in one root must be checked against the other; version-lifecycle rules for preview targets), CONTRIBUTING.md, README.md, ci/README.md, rewritten 12.X.Y-notes.rst.

Why it cannot merge alone

The workflows on main still read cuda.build.version from ci/versions.yml, which this layer removes, and they do not yet pass the registry to the reusable workflows. #3175 and #3176 supply those consumers. See the stack map for the merge plan.

Where to look

All changes (38 files; 37 without the lockfile, +1,332 / -290)

Registry and tools: ci/versions.yml, ci/pyproject.toml (new), ci/__init__.py, ci/tools/__init__.py, ci/tools/bindings_config.py (new), ci/tools/check_pixi_cuda_version.py, tests (test_bindings_config.py, test_check_pixi_cuda_version.py, test_cuda_python_setup.py, all new), ci/tools/tests/AGENTS.md (new test policy: keep a test only when a wrong result would be silent).

Hooks: .pre-commit-config.yaml (new check-bindings-config; pixi hook paths; large-file exceptions), toolshed/check_cuda_core_bindings_floor.py (toolkit-versus-floor check moves into the registry hook), cuda_core/cuda/core/_bindings_floor.py, cuda_core/tests/test_bindings_floor.py.

Packaging: cuda_python/pyproject.toml, cuda_python/setup.py, cuda_bindings_12/pyproject.toml, cuda_bindings_12/pixi.toml (new), cuda_bindings_12/pixi.lock (new), cuda_bindings_12/tests/pytest.ini, cuda_bindings_13/pyproject.toml, root pixi.toml, cuda_core/pixi.toml.

Guidance and docs: AGENTS.md, CONTRIBUTING.md, README.md, ci/README.md, cuda_bindings_12/AGENTS.md (new), cuda_bindings_12/README.md, cuda_bindings_12/docs/source/{contribute.md,install.md,support.rst,release/12.X.Y-notes.rst}, cuda_bindings_13/{AGENTS.md,README.md}, cuda_core/AGENTS.md, cuda_python/AGENTS.md.

Review history and validation
Stack map: review scopes and merge units
Layer PR Scope Merge unit
1 #3172 import cuda_bindings_12/, byte-identical to the 12.9.x branch merged
2 #3173 integrate the import with the shared build tooling merged
3 #3174 registry, packaging, version selectors, developer environment, guidance group: #3174 + #3175 + #3176 land together
4 #3175 build both majors from main; planner; version preview bootstrap group
5 #3176 test, document, nightly, coverage; first CI-complete state group
6 #3177 releases driven by the tagged source's configuration after the group
7 #3178 maintenance policy, manual backport workflow, retire backport_branch after #3177
8 #3182 three CUDA 12 runtime fixes independent of the CI layers; stacked only for ordering

Layers 3 to 5 are review boundaries, not working intermediate states: #3174 alone breaks CI (workflows still read the removed cuda.build.version), and #3175 alone fails actionlint (callers pass an input the test workflows only accept in #3176). The plan is to review them separately and land the reviewed group as one merge. #3176 is the full-CI target for the group.

The staged stack #3314-#3322 is the alternative in which every layer is mergeable on its own; both stacks reach the same end state, except that the staged stack keeps compatibility routes for older artifact sources and retained CUDA 13 tags.

@rwgk rwgk added this to the cuda.bindings next milestone Oct 21, 2026
@rwgk rwgk added enhancement Any code-related improvements CI/CD CI/CD infrastructure cuda.bindings Everything related to the cuda.bindings module labels Oct 21, 2026
@copy-pr-bot

copy-pr-bot Bot commented Oct 21, 2026

Copy link
Copy Markdown
Contributor

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@rwgk rwgk self-assigned this Oct 21, 2026
@github-actions github-actions Bot added the cuda.core Everything related to the cuda.core module label Oct 21, 2026
@rwgk
rwgk added this pull request to stack #3179 October 21, 2026 04:48
@rwgk
rwgk force-pushed the rwgk/cb12-on-main-03-package-selection branch from 631efa9 to 051c1ef Compare October 21, 2026 05:27
@rwgk
rwgk force-pushed the rwgk/cb12-on-main-03-package-selection branch 6 times, most recently from 631b63c to ecb50fa Compare October 21, 2026 19:33
@rwgk

rwgk commented Oct 22, 2026

Copy link
Copy Markdown
Contributor Author

@mdboom, below is a codex assessment, based on a static code analysis, of which PRs in the stack might be independently mergable.

I'm not sure we should invest time into building temporary compatibility bridges. Would it work for you to aim for merging 3174, 3175, 3176 as one group? The idea would be to get them reviewed in this stack, but then combine the final reviewed code into one normal PR, to be merged as one unit (so that we don't have non-CI-testable intermediate states on main).


#3174 is not independently mergeable as written, but making it so is practical. Two changes currently require later layers:

These layers were originally and intentionally divided for review; they do not all represent working intermediate states.

Current Boundaries

Cumulative stack through Assessment
#3174 Existing CI breaks on the configuration and packaging changes.
#3175 Build integration arrives, but matching test-workflow changes are still missing.
#3176 First sensible full-CI target for both majors. Release and backport workflows still need later changes.
#3177 Release routing is updated; the old backport workflow remains incompatible.
#3178 First existing boundary that also completes those operational workflow updates.

These are code-based assessments; I have not run new CI.

Recommendation

Make #3174 independently mergeable with a small temporary compatibility bridge:

  1. Retain the old configuration fields alongside the new registry, validating that their values agree.
  2. Explicitly select CUDA 13 in the existing metapackage build callers.
  3. Remove the transitional fields as later layers migrate their consumers.

That means several files and focused regression checks, without redesigning the stack or pulling substantial CI/release work into #3174.

This would let us merge the packaging foundation separately while keeping main operational. #3176 would then be the next useful cumulative full-CI target. Making #3175 independently functional would require substantially more boundary reshuffling, so I would defer that decision.

@rwgk

rwgk commented Oct 22, 2026

Copy link
Copy Markdown
Contributor Author

I'll work ahead a bit, to see where we land:

codex response to my previous comment:

Yes—reviewing the three layers separately, then landing their combined result in one normal PR, looks practical. I read your comment; that approach avoids putting the incomplete intermediate states on main.

At the combined #3176 state, the toolkit configuration consumers and metapackage build callers have already been migrated. The remaining adjustment is considerably smaller than making #3174 independently mergeable.

Required Adjustment

Postpone removing backport_branch: "12.9.x" until #3178 retires its consumers.

That means:

This preserves the existing backport workflow and current-major release-note checking. Both still require that setting; the release-note checker currently errors without it.

There is no need to restore the old toolkit configuration fields or add temporary metapackage defaults: the combined three-layer state already handles those.

Release Boundary

The intended intermediate state would be:

One existing policy change also needs explicit documentation: #3176’s release documentation resolver requires a registry in the tagged source. Dry-runs against older tags therefore require historical tooling. That policy is already intended for the finished stack; it takes effect here.

Keep Review Straightforward

I would organize the work as follows:

  1. Add one focused commit to cuda.bindings: configure independent CUDA 12 and CUDA 13 packages #3174 preserving the still-used backport setting, with its validator checks.
  2. Adjust cuda.bindings: document shared maintenance and release branches #3178 to remove it when the final consumer disappears.
  3. Describe cuda.bindings: configure independent CUDA 12 and CUDA 13 packages #3174–CI: test and document both bindings majors on main #3176 as one merge group, with the release boundary stated clearly.
  4. After review, create the combined PR and verify that its Git tree exactly matches the reviewed cumulative state.

Validation should include focused release/backport checks, plus one full CI run on the combined landing PR, rather than full runs on all three review layers.

This keeps the substantial review boundaries intact: packaging, builds, then tests/documentation. The additional change is a small correction to migration order. Nothing changed or triggered yet.

@rwgk

rwgk commented Oct 22, 2026

Copy link
Copy Markdown
Contributor Author

@mdboom, #3174–#3176 now form a validated landing point, with separate review scopes for package configuration, builds, and tests/docs.

Full CI is green at both the cumulative #3176 head and the stack top, #3182. The #3176 state preserves the existing CUDA 13 release and backport flows; the later release-routing and maintenance-policy changes remain separate.

Could you please review #3174–#3176 with a view to landing them together? The intent is to retain the three review scopes, then combine them for merging (as explained in my previous comment).

Base automatically changed from rwgk/cb12-on-main-02-source-integration to main October 23, 2026 15:04
@mdboom
mdboom force-pushed the rwgk/cb12-on-main-03-package-selection branch from b087aaf to c8571e0 Compare October 23, 2026 15:04
@rwgk
rwgk force-pushed the rwgk/cb12-on-main-03-package-selection branch from c8571e0 to 96c2a38 Compare October 25, 2026 23:56

@mdboom mdboom left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Found with my agent:

The 12.9.x branch builds cp310 wheels. Its build-wheel.yml matrix starts at 3.10, and I checked that on upstream/12.9.x. The shared python.supported list in [ci/versions.yml](vscode-webview://1lq1g48ng0sclg835m7ufv5pq987ujci9cjh6dj4ho9llgobp0ub/ci/versions.yml) starts at 3.11, and #3175 feeds it to every wheel build. After this stack, 12.9.x wheels for 3.10 stop being built. Meanwhile cuda_bindings_12 declares requires-python = ">=3.10", keeps the 3.10 classifier, and its pixi build variants include 3.10. Python 3.10 users would silently fall back to the old 12.9.10 or a source build. The change also drops the 3.9 classifier, and the 12.9.x pyproject has no requires-python. If dropping 3.10 is intended, raise requires-python to match. Otherwise the registry needs a per-package Python list.

I think maybe the bring-over of 12.9.x predates dropping of Python 3.10 support there. It is fine to drop Python 3.10 support across the board (even for 12.9.x cuda-bindings) since it is no longer supported. Let's just make all the updates necessary here to be consistent about that.

Comment thread ci/tools/bindings_config.py Resolved
Comment thread cuda_bindings_12/pyproject.toml Resolved
Comment thread ci/tools/bindings_config.py Resolved
Comment thread ci/tools/bindings_scm_version.py Resolved
Comment thread ci/tools/bindings_config.py Resolved
Comment thread ci/tools/bindings_config.py Resolved
Comment thread ci/tools/bindings_config.py Resolved
Comment thread cuda_python/setup.py Resolved
Comment thread ci/versions.yml Resolved
Comment thread ci/tools/bindings_config.py Resolved
@rwgk
rwgk force-pushed the rwgk/cb12-on-main-03-package-selection branch 2 times, most recently from 58ddd02 to 4d8d156 Compare November 1, 2026 22:50

@rwgk rwgk left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

— codex GPT-6 on behalf of Ralf —

With Codex's help, I rechecked all 11 of Mike's inline comments here and confirmed that each already had a substantive reply.

For every comment, Codex drafted a fresh reply based on the current code and verified its source references. We then compared each draft with the previous reply, and checked a fresh overall summary against the earlier inline answers and cross-PR response. The drafts added references and clarifications, but no new substantive answers.

The fresh drafts are collected below to document the comparison without adding duplicate messages to the original threads.

All 11 previous replies paired with fresh drafts, plus the overall summary

Each link opens the previously posted reply. The indented text is the fresh draft used in the comparison.

rwgk added 19 commits November 3, 2026 21:45
Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Read repository-owned configuration directly, retain release-routing and package consistency checks, and use packaging.Version for canonical release tags. Store toolkit components once as integers while preserving JSON and shell interfaces. Identify the hooks responsible for Core floor checks.

Defer the pre-milestone SCM bootstrap helper and its tests to the following CI-build layer, where its first workflow callers appear.

Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Align package metadata, Pixi build variants, the generated lockfile, and current installation/support documentation with the shared CI Python matrix. Preserve historical release notes.

Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Preserve ordinary external source installs without an environment selector. Explicit CUDA 12 and CUDA 13 builds retain their matching SCM selectors and dependencies; reject unsupported explicit selections.

Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
codex on behalf of Ralf:

Add the maintenance counterparts of the external build dependency group and shared contribution-guide pointer, preserving CUDA 12 build inputs.

Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
@rwgk
rwgk force-pushed the rwgk/cb12-on-main-03-package-selection branch from 6d79acc to f03b990 Compare November 4, 2026 06:18

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CI/CD CI/CD infrastructure cuda.bindings Everything related to the cuda.bindings module cuda.core Everything related to the cuda.core module enhancement Any code-related improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants