Skip to content

[pathfinder] Respect site-packages root priority across wheel layouts - #3370

Open
0z5a wants to merge 2 commits into
NVIDIA:mainfrom
0z5a:codex/pathfinder-site-root-precedence-20261104
Open

0z5a wants to merge 2 commits into
NVIDIA:mainfrom
0z5a:codex/pathfinder-site-root-precedence-20261104

Conversation

@0z5a

@0z5a 0z5a commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Description

Closes #3369.

In a virtual environment with system site-packages enabled, a CUDA 13 wheel layout in the user site can be selected before a CUDA 12 layout in the venv, despite the venv site having higher priority. The same issue can occur between user and system sites. The existing search loop considers wheel layouts before site roots.

Search site-packages roots first, then each declared wheel layout within a root. Apply the same ordering to dynamic libraries on Linux and Windows, headers, binary utilities, static libraries, and bitcode libraries. The declared preference between layouts within a single root remains unchanged. This addresses a case not covered by #1717's single-layout fix for #1716.

Validation

  • Ubuntu 24.04.4 x86_64, Python 3.12.3, CPU-only: 1,474 passed across the affected pathfinder finder, search, descriptor, and mocked loader tests, including new root/layout precedence, fallback, and non-venv regressions.
  • Before the fix, all six new venv-versus-user/system cases failed; after the fix, all passed.
  • The 30 new cases are marked thread-unsafe because they modify process-wide site state and caches; with --parallel-threads=4, all 30 passed in the serial portion of the run.
  • In a real --system-site-packages venv with placeholder files, bitcode, static library, header, and binary lookups changed from the user site to the venv site. With two small CPU-only test shared libraries, a fresh load_nvidia_dynamic_lib("cudart") process changed from loading the user-site marker 13 to the venv marker 12. These are synthetic test files, not NVIDIA runtimes.
  • Ruff and the applicable pre-commit checks, including mypy-pathfinder, passed. Windows search behavior was exercised by platform-model tests on Linux; no Windows host or GPU test was run.

Checklist

  • I have read the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

The issue is not yet assigned; maintainer coordination and review are requested.

Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
@copy-pr-bot

copy-pr-bot Bot commented Nov 4, 2026

Copy link
Copy Markdown
Contributor

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@github-actions github-actions Bot added the cuda.pathfinder Everything related to the cuda.pathfinder module label Nov 4, 2026
Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
@coderabbitai

coderabbitai Bot commented Nov 4, 2026 •

Copy link
Copy Markdown

Explain Change →

No actionable comments were generated in the recent review.

Summary

Summary by CodeRabbit

  • Bug Fixes
    • CUDA binaries, headers, and libraries are now searched consistently across virtual-environment, user, and system package locations. A matching item in a higher-priority location is selected before considering alternate layouts in lower-priority locations.
    • If no match is available in a higher-priority location, discovery continues through the remaining locations and layouts.

Site-package discovery now searches candidate layouts within each root before moving to the next root. Binary, dynamic-library, header, static-library, and bitcode finders use this ordered traversal. Tests cover root and layout precedence.

Changes: Site-package search precedence

Layer / File(s) Summary
Order site roots and layouts
cuda_pathfinder/cuda/pathfinder/_utils/find_sub_dirs.py, cuda_pathfinder/tests/test_site_packages_precedence.py
The shared helpers search roots in priority order and check the supplied patterns within each root. Parameterized tests cover root priority, layout priority, and fallback behavior.
Use ordered traversal in finders
cuda_pathfinder/cuda/pathfinder/_binaries/*, cuda_pathfinder/cuda/pathfinder/_dynamic_libs/*, cuda_pathfinder/cuda/pathfinder/_headers/*, cuda_pathfinder/cuda/pathfinder/_static_libs/*, cuda_pathfinder/tests/test_find_*.py, cuda_pathfinder/tests/test_search_steps.py
The finders search all candidate layouts through the ordered helper. Tests update their mocks and pattern expectations. The Windows wheel-directory docstring now describes CUDA 13 as preferred and CUDA 12 as an x64 fallback.

Assessment against linked issues

Objective Addressed Explanation
Search all applicable layouts in each higher-priority site-package root before moving to the next root, while preserving layout order within each root. [#3369] ✅

Possibly related issues

  • Issue 1093: The new precedence tests cover multiple pathfinder artifact types, but the supplied issue summary also calls for a dedicated CI matrix and additional installation modes.

Suggested reviewers: rwgk, cpcloud

Change: Bug fix · Severity of issue fixed: Medium

Merge Risk: Minimal · up to b361e

Site-package searches now prioritize roots while preserving layout order. No actionable merge-blocking code risk was established.

- [ ] Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Recent review info
Run configuration
Commits

Reviewing files that changed from the base of the PR and between deb3d04 and b361e24.

Files selected for processing: 13
  • cuda_pathfinder/cuda/pathfinder/_binaries/find_nvidia_binary_utility.py
  • cuda_pathfinder/cuda/pathfinder/_dynamic_libs/descriptor_catalog.py
  • cuda_pathfinder/cuda/pathfinder/_dynamic_libs/search_platform.py
  • cuda_pathfinder/cuda/pathfinder/_headers/find_nvidia_headers.py
  • cuda_pathfinder/cuda/pathfinder/_static_libs/find_bitcode_lib.py
  • cuda_pathfinder/cuda/pathfinder/_static_libs/find_static_lib.py
  • cuda_pathfinder/cuda/pathfinder/_utils/find_sub_dirs.py
  • cuda_pathfinder/tests/test_find_bitcode_lib.py
  • cuda_pathfinder/tests/test_find_nvidia_binaries.py
  • cuda_pathfinder/tests/test_find_nvidia_headers.py
  • cuda_pathfinder/tests/test_find_static_lib.py
  • cuda_pathfinder/tests/test_search_steps.py
  • cuda_pathfinder/tests/test_site_packages_precedence.py

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 8 remain after this review.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cuda.pathfinder Everything related to the cuda.pathfinder module

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG][pathfinder] Wheel layout order can override site-packages root priority

1 participant