Skip to content

DO NOT MERGE: probe compute-sanitizer on Python 3.12 local-CTK rows - #3400

Closed
juenglin wants to merge 8 commits into
NVIDIA:mainfrom
juenglin:juenglin/tmp-sanitizer-probe
Closed

juenglin wants to merge 8 commits into
NVIDIA:mainfrom
juenglin:juenglin/tmp-sanitizer-probe

Conversation

@juenglin

@juenglin juenglin commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

Description

Do not merge. Temporary probe for #3397 (based on its head). No row in ci/test-matrix.yml currently satisfies the compute-sanitizer gate in ci/tools/env-vars (Linux, Python 3.12, LOCAL_CTK: '1'), so the sanitizer has not run in CI since the 3.12 / 12.9.1 rows were disabled in #2437.

This PR flips the amd64 Python 3.12 rows for 12.9.1, 13.0.2, and 13.5.0 to local-CTK so that the bindings and core test suites run under compute-sanitizer installed from the CTK under test. It will be closed once CI results are in.

Checklist

  • I have read the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

Fetch cuda_sanitizer_api from the same pinned redistrib manifest as the
rest of the mini CTK, rather than from whatever release
ci/tools/guess_latest.sh scrapes from the redist directory listing.
Remove guess_latest.sh, which has no other users.
verify-metadata used to warn and pass when a public-server CUDA version had
no pinned manifest hash, so the 13.4.2 -> 13.5.0 bump ran unpinned. Fail
closed instead, with an explicit fetch_ctk allow-unpinned input (and
--allow-unpinned flag) as the opt-out; it never overrides a mismatch against
an existing pin. Manifests from an alternate redist server stay warn-only.

Pin 13.5.0 and add a test that every CUDA version in ci/versions.yml and
every LOCAL_CTK row of ci/test-matrix.yml is pinned.
No caller sets it, and pinning a new CUDA version is a one-line addition to
KNOWN_REDISTRIB_SHA256 that the error message points to. Remove the
fetch_ctk input and the --allow-unpinned flag; fetch_ctk/action.yml is back
to its main-branch form.
Note in ci/versions.yml that each CUDA version there and in LOCAL_CTK
matrix rows needs a pin in KNOWN_REDISTRIB_SHA256, and make the coverage
test's failure message say where to add the pin and how to compute it.
@juenglin juenglin added this to the CI milestone Nov 6, 2026
@juenglin juenglin added CI/CD CI/CD infrastructure github_actions Pull requests that update GitHub Actions code labels Nov 6, 2026
@copy-pr-bot

copy-pr-bot Bot commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@juenglin juenglin self-assigned this Nov 6, 2026
@NVIDIA NVIDIA deleted a comment from copy-pr-bot Bot Nov 6, 2026
@juenglin

juenglin commented Nov 6, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test 26ea52d

@github-actions

github-actions Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor
Doc Preview CI
Preview removed because the pull request was closed or merged.

NVIDIA#2437 made fetch_ctk empty cuda-path before every install. The
sanitizer step runs fetch_ctk a second time for cuda_sanitizer_api
alone, so it unintentionally deleted the mini CTK installed just
before it, and the instrumented test runs failed before any test ran.
NVIDIA#2437 then disabled the only matrix rows that run the sanitizer.

Add an install-mode input. "add" installs on top of the existing mini
CTK at the same cuda-path and keeps the environment the first call
exported; the sanitizer step uses it.
@juenglin

juenglin commented Nov 6, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test 5a7cabf

@juenglin

juenglin commented Nov 6, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test 54c5049

juenglin added a commit to juenglin/cuda-python that referenced this pull request Nov 6, 2026
With the matrix CTK's own sanitizer, the 12.9.1 and 13.0.2 rows
segfaulted partway through the cuda.core tests. With the 13.5.0
sanitizer, both ran the full suite (NVIDIA#3400). This keeps the intent
of NVIDIA#594 (newer sanitizer fixes for older toolkits) without the
run-time scrape: the version comes from cuda.build.version in
ci/versions.yml, which is already pinned.
@juenglin juenglin closed this Nov 6, 2026
github-actions Bot pushed a commit that referenced this pull request Nov 7, 2026
Removed preview folders for the following PRs:
- PR #2951
- PR #3314
- PR #3315
- PR #3371
- PR #3392
- PR #3396
- PR #3397
- PR #3400
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CI/CD CI/CD infrastructure github_actions Pull requests that update GitHub Actions code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant