Skip to content

feat(pathfinder): opt-in diagnostic logging for load_nvidia_dynamic_lib - #2951

Open
u7k4rs6 wants to merge 7 commits into
NVIDIA:mainfrom
u7k4rs6:feat/pathfinder-diagnostic-logging
Open

u7k4rs6 wants to merge 7 commits into
NVIDIA:mainfrom
u7k4rs6:feat/pathfinder-diagnostic-logging

Conversation

@u7k4rs6

@u7k4rs6 u7k4rs6 commented Sep 25, 2026 •

Copy link
Copy Markdown

Description

closes #650

Opt-in logging of where load_nvidia_dynamic_lib found each library. Cut down per the discussion below to one INFO record per successful load:

$ CUDA_PATHFINDER_LOG_LEVEL=INFO python -c "import logging; logging.basicConfig(); import cuda.pathfinder as p; p.load_nvidia_dynamic_lib('cudart')"
INFO:cuda.pathfinder:loaded cudart from /usr/lib/x86_64-linux-gnu/libcudart.so.12 (found via system-search)
  • Off by default. logging is only imported when CUDA_PATHFINDER_LOG_LEVEL is set (level name or integer); an invalid value warns and leaves it off.
  • Logged once per library, since load_nvidia_dynamic_lib is cached. Nothing is logged on failure; DynamicLibNotFoundError already carries that.
  • cuda.pathfinder logger with a NullHandler only; the root logger is never configured.
  • The env var is mentioned in the load_nvidia_dynamic_lib docstring.

Locally: pytest tests/ in cuda_pathfinder passes (1627 passed, 5 skipped). New tests are marked agent_authored.

Checklist

  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

u7k4rs6 added 3 commits August 5, 2026 22:17
Adds a `cuda.pathfinder` logger, disabled by default, enabled by setting
CUDA_PATHFINDER_LOG_LEVEL to a standard level name or an integer.

`logging` is imported only when the variable is set. Importing it pulls in
seven additional modules and measurably slows `import cuda.pathfinder`, which
sits on the import hot path of every consumer, so the disabled path imports
nothing and costs one module-global lookup plus an identity check (~5 ns) at
each call site. No message string or `extra` dict is built when logging is off.

Instruments the dynamic-library search only: each find step reports whether it
matched, a successful load reports the resolved path and `found_via`, and the
failure path emits the accumulated candidate list as structured fields. Records
carry `pathfinder_*` fields so consumers can filter without parsing messages.

The logger attaches a NullHandler, never calls basicConfig, and never touches
the root logger. Invalid CUDA_PATHFINDER_LOG_LEVEL values warn once and leave
logging disabled rather than raising.

The environment variable is read once at import, matching the documented
read-once policy of get_cuda_path_or_home().

Signed-off-by: Utkarsh Bahuguna <utkarshbahuguna10@gmail.com>
…odules

The cascade fixture reloaded search_steps and load_nvidia_dynamic_lib to pick
up a LOGGER enabled after import. A reload rebinds everything the module
defines, including FindResult, so any test module that imported FindResult
beforehand held a different class object than the one the reloaded
search_steps returned. Dataclass equality compares classes, so those
assertions failed on values that print identically, and which tests broke
depended on execution order.

The consumers bind LOGGER by value at import, so setting that one attribute
through monkeypatch is enough and leaves no state behind. test_silent_when_
disabled asserts the disabled default rather than reloading into it.

Against current main this fixes 13 failures in test_search_steps.py and
test_windows_nsight.py; on the original base it was 2 in
test_load_nvidia_dynamic_lib_using_mocker.py. CI never ran on this PR, so it
was never reported.

Signed-off-by: Utkarsh Bahuguna <utkarshbahuguna10@gmail.com>
@copy-pr-bot

copy-pr-bot Bot commented Sep 25, 2026

Copy link
Copy Markdown
Contributor

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@github-actions github-actions Bot added the cuda.pathfinder Everything related to the cuda.pathfinder module label Sep 25, 2026
@rwgk

rwgk commented Sep 25, 2026

Copy link
Copy Markdown
Contributor

@u7k4rs6 do you have an actual need for this feature? Could you please explain? — #650 was an idea early on, but then nobody asked for the feature in more than a year, therefore I'm not sure it's actually worth an effort.

@u7k4rs6

u7k4rs6 commented Sep 26, 2026

Copy link
Copy Markdown
Author

Fair challenge. I didn't come to this from a production incident. I picked #650 up from the backlog. But when I checked whether anyone actually needs it, it turns out every downstream project is already working around the gap:

To keep this cheap for you to accept, I can cut the PR down to the minimum:

  • one INFO record per successful load, carrying abs_path and found_via
  • nothing on the failure path (DynamicLibNotFoundError already covers it)
  • off by default, and logging is never imported unless CUDA_PATHFINDER_LOG_LEVEL is set, so it costs nothing for anyone who doesn't opt in

That's a few lines in load_nvidia_dynamic_lib plus tests. The per-step search trail can come later, only if someone asks for it. Would the smaller version work for you?

Signed-off-by: Utkarsh Bahuguna <utkarshbahuguna10@gmail.com>
@u7k4rs6

u7k4rs6 commented Oct 1, 2026

Copy link
Copy Markdown
Author

Pushed the smaller version: one INFO record per successful load, off by default, 51 lines of source plus tests. @rwgk could you take a look?

@rparolin rparolin added this to the cuda.pathfinder next milestone Oct 8, 2026
@u7k4rs6 u7k4rs6 closed this Oct 18, 2026
@u7k4rs6
u7k4rs6 deleted the feat/pathfinder-diagnostic-logging branch October 18, 2026 06:04
@u7k4rs6
u7k4rs6 restored the feat/pathfinder-diagnostic-logging branch October 30, 2026 08:52
@u7k4rs6 u7k4rs6 reopened this Oct 30, 2026
@u7k4rs6

u7k4rs6 commented Oct 30, 2026

Copy link
Copy Markdown
Author

Reopened (closed by mistake), merged main, and updated the description to match the smaller diff. @rparolin could you add the feature label and /ok to test? The metadata check only needs the type label.

@coderabbitai

coderabbitai Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

Explain Change →

No actionable comments were generated in the recent review.

Summary

Summary by CodeRabbit

  • New Features
    • Optional diagnostic logging is available for NVIDIA library loading. Set CUDA_PATHFINDER_LOG_LEVEL to a supported level, such as INFO or DEBUG, to see the library name, location, and discovery source. Logging remains off when the setting is blank or unset; invalid values leave it disabled.
  • Documentation
    • Added guidance on enabling diagnostic logging.

Adds optional diagnostic logging controlled by CUDA_PATHFINDER_LOG_LEVEL. When enabled, successful dynamic-library loads log the library name, absolute path, and found_via value.

Changes: Opt-in diagnostic logging

Layer / File(s) Summary
Logger configuration
cuda_pathfinder/cuda/pathfinder/_utils/diagnostic_log.py, cuda_pathfinder/tests/test_diagnostic_log.py
Configures the cuda.pathfinder logger from accepted environment-variable values. Tests cover disabled and enabled settings, invalid values, root-logger isolation, and lazy logging imports.
Successful-load logging
cuda_pathfinder/cuda/pathfinder/_dynamic_libs/load_nvidia_dynamic_lib.py, cuda_pathfinder/tests/test_diagnostic_log.py
Logs library details at INFO level after a successful load when logging is enabled. Tests check successful, failed, and disabled logging behavior.

Assessment against linked issues

Objective Addressed Explanation
Add support for user-opt-in logging in path finder [#650] ✅

Suggested labels: feature

Suggested reviewers: rwgk

Change: Feature

Merge Risk: Minimal · up to 7c79f

Successful-load diagnostics remain opt-in, with no merge-blocking code risk identified.

- [ ] Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Tip

🎯 Customize your risk assessments

Guide how CodeRabbit assesses merge risk and security architecture risk. Highlight sensitive changes and explain your security boundaries using Merge Risk Instructions and Security Risk Instructions in your review settings. Security architecture risk assessments require the Advanced plan or above.

✅ Automatic approvals, with your rules

Let CodeRabbit approve pull requests once required threads are resolved, pre-merge checks pass, and your risk limits are met. Enable Automatic approvals in review settings, with overrides for teams or file paths.

Recent review info

Run configuration

Commits

Reviewing files that changed from the base of the PR and between 4c9b60e and 7c79f25.

Files selected for processing: 2

  • cuda_pathfinder/cuda/pathfinder/_utils/diagnostic_log.py
  • cuda_pathfinder/tests/test_diagnostic_log.py

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1


Review info
Run configuration
Commits

Reviewing files that changed from the base of the PR and between 03b4153 and 4c9b60e.

Files selected for processing: 3
  • cuda_pathfinder/cuda/pathfinder/_dynamic_libs/load_nvidia_dynamic_lib.py
  • cuda_pathfinder/cuda/pathfinder/_utils/diagnostic_log.py
  • cuda_pathfinder/tests/test_diagnostic_log.py

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.

@Andy-Jost

Copy link
Copy Markdown
Contributor

/ok to test 4c9b60e

@github-actions

Copy link
Copy Markdown
Contributor

… decimal levels

Signed-off-by: Utkarsh Bahuguna <utkarshbahuguna10@gmail.com>
@u7k4rs6

u7k4rs6 commented Nov 6, 2026

Copy link
Copy Markdown
Author

The free-threaded failures were from my tests reloading the module under a patched os.environ in parallel; marked them thread_unsafe (7c79f25). @Andy-Jost could you /ok to test again? Thanks!

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cuda.pathfinder Everything related to the cuda.pathfinder module

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEA]: Add support for user-opt-in logging in path finder

4 participants