You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
ci: harden fetch_ctk with manifest/archive verification and optional redist-base-url - #3347
After this lands, both public and private repositories use identical files for fetch_ctk/action.yml, install_windows_deps/action.yml, ci/tools/fetch_ctk_redistrib.py, and ci/tools/tests/test_fetch_ctk_redistrib.py.
Changes
.github/actions/fetch_ctk/action.yml
Add optional redist-base-url input (default: public NVIDIA developer server). The action is now parameterizable for any redistrib-shaped server without changing the action itself.
Split the single Install dependencies step into two OS-gated steps: Install Unix dependencies (Linux) and Install Windows dependencies (Windows, via the new install_windows_deps action). This makes the action usable on Windows runners.
Download the redistrib manifest once in Set up CTK cache variable and reuse it in Get CUDA components, eliminating a redundant network round-trip.
Verify the manifest hash before trusting it (verify-metadata). Public-server manifests are checked against KNOWN_REDISTRIB_SHA256; alternate servers warn only. Per-archive verification is only trustworthy if the JSON carrying those hashes is itself trusted, so this step must come first.
Include the first 8 hex digits of the manifest sha256 in the cache key (-m<sha8>). Alternate servers can publish different manifests under the same CUDA version; without this they would restore each other's cached toolkit. For the public server this is a one-time cache miss on the first run with the new key format.
Verify each downloaded component archive against the sha256 in the (now trusted) manifest before extracting (component-sha256 + sha256sum --check). The old code extracted without any archive integrity check.
Composite action that installs a pinned portable Git for Windows bundle (ARM64 or x64) when curl, unzip, tar, sha256sum, or cygpath are not already on PATH. Checks the archive hash before extracting. Caches the result under RUNNER_TOOL_CACHE.
ci/tools/fetch_ctk_redistrib.py
Add KNOWN_REDISTRIB_SHA256: pinned public manifest hashes for 12.6.3, 12.9.1, 13.0.2, and 13.4.2. Extend as new CUDA versions enter the matrix (the regeneration command is in the dict comment).
Add component-sha256 and verify-metadata CLI subcommands. verify-metadata accepts --metadata-source {public,custom}; custom sources only warn and never fail closed.
Remove --metadata-url from filter-components and component-relative-path. Callers must download the manifest and pass --metadata-path. This removes the urllib dependency and prevents the action from downloading the manifest twice.
Simplify load_metadata to read from a local path only.
ci/tools/tests/test_fetch_ctk_redistrib.py (new)
13 agent-authored unit tests covering get_component_sha256 (hash lookup, alias resolution, missing field, unavailable platform), verify_metadata_sha256 (match, mismatch, unpinned warning, custom source), and the component-sha256 and verify-metadata CLI subcommands.
Run with python -m pytest ci/tools/tests/test_fetch_ctk_redistrib.py.
Compatibility
Removing --metadata-url from filter-components and component-relative-path is a breaking change for any direct caller using the URL form. Within this repository, only fetch_ctk/action.yml calls the script, and it is updated here. The URL form is not part of any public API.
The new -m<sha8> suffix in the cache key causes a one-time cache miss for all existing public-server cached toolkits on the first run. Subsequent runs hit the new key.
Testing
All 13 new tests pass locally (python -m pytest ci/tools/tests/test_fetch_ctk_redistrib.py -v). The action changes will be exercised by CI on this PR.
…se-url
Brings the public fetch_ctk action and redistrib helper into parity with
the private ctk-next branch.
fetch_ctk/action.yml
- Add optional redist-base-url input (default: public NVIDIA developer
server). Callers can override to install from an alternate redistrib
server without changing the action itself.
- Split the single 'Install dependencies' step into two OS-gated steps:
'Install Unix dependencies' (Linux) and 'Install Windows dependencies'
(Windows, via the new install_windows_deps action). This makes the
action usable on Windows runners.
- Download the redistrib manifest once in 'Set up CTK cache variable'
and reuse it in 'Get CUDA components', eliminating a redundant
network round-trip.
- Verify the manifest hash before trusting it (verify-metadata). Public-
server manifests are checked against KNOWN_REDISTRIB_SHA256; alternate
servers warn only.
- Include the first 8 hex digits of the manifest sha256 in the cache
key (-m<sha8>). Alternate servers can publish different manifests
under the same CUDA version; without this they would restore each
other's cached toolkit.
- Verify each downloaded component archive against the sha256 in the
(now trusted) manifest before extracting (component-sha256 +
sha256sum --check). The old code extracted without any archive check.
install_windows_deps/action.yml (new)
- Installs a pinned portable Git for Windows bundle (ARM64 or x64) when
curl, unzip, tar, sha256sum, or cygpath are not already on PATH.
Checks the archive hash before extracting.
ci/tools/fetch_ctk_redistrib.py
- Add KNOWN_REDISTRIB_SHA256: pinned public manifest hashes for 12.6.3,
12.9.1, 13.0.2, and 13.4.2. Extend as new versions enter the matrix.
- Add get_component_subdir_info, get_component_sha256,
compute_file_sha256, verify_metadata_sha256.
- Add component-sha256 and verify-metadata subcommands.
verify-metadata accepts --metadata-source {public,custom}; custom
sources only warn (never fail closed on a hash mismatch).
- Remove --metadata-url from filter-components and component-relative-
path; callers must download the manifest themselves and pass
--metadata-path. Removes the urllib dependency.
- Simplify load_metadata to read from a local path only.
ci/tools/tests/test_fetch_ctk_redistrib.py (new)
- 13 agent-authored unit tests covering get_component_sha256 (hash
lookup, alias resolution, missing field, unavailable platform),
verify_metadata_sha256 (match, mismatch, unpinned warning, custom
source), component-sha256 and verify-metadata CLI subcommands.
On Windows, RUNNER_TEMP contains backslashes, and Git for Windows'
sha256sum prefixes its output line with a backslash for such file
names. The first 8 characters then included the backslash, producing an
invalid cache key and breaking 'tar -czvf' on the cache filename. Read
the manifest from stdin so no file name is printed.
No actionable comments were generated in the recent review.
Summary
Summary by CodeRabbit
New Features
CUDA toolkit downloads support a configurable redistribution server, with NVIDIA’s public server as the default.
Windows CI runners can set up required command-line tools when they are unavailable.
Reliability
Downloaded components are verified against manifest-provided SHA-256 checksums before extraction. Cache entries account for both manifest and component-list hashes.
Selected public metadata versions are checked against pinned SHA-256 values; custom or unpinned metadata may produce a warning.
Tests
Added coverage for metadata verification, component checksums, and custom download sources.
The CTK fetch action accepts a configurable redistributable base URL. It downloads and verifies a local manifest, filters components, and checks archive hashes before extraction. The CLI verifies pinned metadata and reports component hashes. A Windows action checks for required executables and installs Portable Git when needed.
Changes: CTK Fetch and Verification
Layer / File(s)
Summary
Metadata verification and component hashes ci/tools/fetch_ctk_redistrib.py, ci/tools/tests/test_fetch_ctk_redistrib.py
The CLI loads metadata from a local path, verifies pinned metadata hashes, and reports component archive hashes. Tests cover pinned and unpinned metadata, custom sources, component lookup, and CLI results.
Windows dependency installation .github/actions/install_windows_deps/action.yml
The composite action checks required executables and installs a checksum-verified Portable Git bundle when they are unavailable. It adds the bundle’s directories to PATH.
The action selects dependencies by runner OS, downloads and verifies the selected manifest, and uses it to filter components. It includes the manifest hash in the cache key and checks each component archive hash before extraction.
Suggested reviewers:leofang
Change: Bug fix
Merge Risk: Minimal · up to 794c0
No supported current-head defect remains established; the change is mergeable with normal CI validation.
- [ ] Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Reviewing files that changed from the base of the PR and between aa4689f and 794c0f2.
Files selected for processing: 1
.github/actions/install_windows_deps/action.yml
Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 10 remain after this review.
suggestion: Normalize CTK_BASE_URL before the public-server comparison at Line 59. A trailing slash, such as .../redist/, fails the exact string match, so the pinned-hash check is skipped. The action treats the public server as custom and only warns. The same value also produces // in the manifest and archive URLs. Strip the slash first.
Also pass inputs.redist-base-url through env: instead of interpolating it into the script body. This follows the GitHub Actions guidance on script injection.
suggestion: curl -LSs has no --fail and no retry. An HTTP error page is written to $CTK_JSON_FILE and exits with status 0. A transient 5xx on the public server then fails with a metadata hash mismatch, which hides the real cause. For a custom server, the failure shows up later as a JSON parse error. Add --fail and a retry to make the failure clear and the step less flaky.
- install_windows_deps: skip the WSL launcher (System32/SysWOW64/
WindowsApps bash) when choosing the shell for the dependency check, and
fail the check if DEPENDENT_EXES did not reach Bash. A WSL bash does not
receive the Windows environment, so its empty loop reported success and
skipped installing portable Git.
- fetch_ctk: strip a trailing slash from redist-base-url so the public
server is still recognized (and its metadata pin enforced), and add
--fail --retry 3 --retry-delay 2 to the manifest download so HTTP errors
surface clearly instead of as a later hash or JSON error.
…t on PATH
- fetch_ctk: use the full SHA-256 of the manifest in the cache key and
filename instead of 8 hex digits, so an alternate server cannot share a
key with the public server by colliding a truncated hash.
- install_windows_deps: when the validated Git Bash is not the first bash
on PATH (for example the WSL launcher comes first), prepend its directory
to PATH and GITHUB_PATH so later 'shell: bash' steps use Git Bash.
Windows PowerShell 5.1 strips embedded double quotes when passing arguments
to native commands, so `[ -n "$DEPENDENT_EXES" ]` reached bash as
`[ -n curl unzip ... ]` and failed with "too many arguments", even with every
tool present. Count the words with `set --` instead.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
CI/CDCI/CD infrastructuregithub_actionsPull requests that update GitHub Actions code
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
rwgk
left a comment
Summary
After this lands, both public and private repositories use identical files for
fetch_ctk/action.yml,install_windows_deps/action.yml,ci/tools/fetch_ctk_redistrib.py, andci/tools/tests/test_fetch_ctk_redistrib.py.Changes
.github/actions/fetch_ctk/action.ymlredist-base-urlinput (default: public NVIDIA developer server). The action is now parameterizable for any redistrib-shaped server without changing the action itself.Install dependenciesstep into two OS-gated steps:Install Unix dependencies(Linux) andInstall Windows dependencies(Windows, via the newinstall_windows_depsaction). This makes the action usable on Windows runners.Set up CTK cache variableand reuse it inGet CUDA components, eliminating a redundant network round-trip.verify-metadata). Public-server manifests are checked againstKNOWN_REDISTRIB_SHA256; alternate servers warn only. Per-archive verification is only trustworthy if the JSON carrying those hashes is itself trusted, so this step must come first.-m<sha8>). Alternate servers can publish different manifests under the same CUDA version; without this they would restore each other's cached toolkit. For the public server this is a one-time cache miss on the first run with the new key format.component-sha256+sha256sum --check). The old code extracted without any archive integrity check..github/actions/install_windows_deps/action.yml(new)curl,unzip,tar,sha256sum, orcygpathare not already on PATH. Checks the archive hash before extracting. Caches the result underRUNNER_TOOL_CACHE.ci/tools/fetch_ctk_redistrib.pyKNOWN_REDISTRIB_SHA256: pinned public manifest hashes for 12.6.3, 12.9.1, 13.0.2, and 13.4.2. Extend as new CUDA versions enter the matrix (the regeneration command is in the dict comment).get_component_subdir_info,get_component_sha256,compute_file_sha256,verify_metadata_sha256.component-sha256andverify-metadataCLI subcommands.verify-metadataaccepts--metadata-source {public,custom}; custom sources only warn and never fail closed.--metadata-urlfromfilter-componentsandcomponent-relative-path. Callers must download the manifest and pass--metadata-path. This removes theurllibdependency and prevents the action from downloading the manifest twice.load_metadatato read from a local path only.ci/tools/tests/test_fetch_ctk_redistrib.py(new)get_component_sha256(hash lookup, alias resolution, missing field, unavailable platform),verify_metadata_sha256(match, mismatch, unpinned warning, custom source), and thecomponent-sha256andverify-metadataCLI subcommands.python -m pytest ci/tools/tests/test_fetch_ctk_redistrib.py.Compatibility
Removing
--metadata-urlfromfilter-componentsandcomponent-relative-pathis a breaking change for any direct caller using the URL form. Within this repository, onlyfetch_ctk/action.ymlcalls the script, and it is updated here. The URL form is not part of any public API.The new
-m<sha8>suffix in the cache key causes a one-time cache miss for all existing public-server cached toolkits on the first run. Subsequent runs hit the new key.Testing
All 13 new tests pass locally (
python -m pytest ci/tools/tests/test_fetch_ctk_redistrib.py -v). The action changes will be exercised by CI on this PR.