Skip to content

[CI] Replace the pytest<9.2 cap with pytest-benchmark>=5.3.1 - #12607

Open
pylaterreur wants to merge 1 commit into
NVIDIA:mainfrom
pylaterreur:ci-drop-pytest-cap
Open

pylaterreur wants to merge 1 commit into
NVIDIA:mainfrom
pylaterreur:ci-drop-pytest-cap

Conversation

@pylaterreur

Copy link
Copy Markdown

Description

Follow-up to #12313.

#12313 capped pytest below 9.2 in the cuda.cccl.examples lanes: pytest 9.2.0 removed the private Function._nodeid, and pytest-benchmark 5.3.0 read it in its benchmark fixture (ionelmc/pytest-benchmark#321), so every host-benchmark case errored during setup. pytest-benchmark 5.3.1 (2026-11-01, ionelmc/pytest-benchmark#323) reads the public nodeid instead.

As the TODO in #12313 asks, this PR removes the pytest cap from the Linux and Windows install lines and requires pytest-benchmark>=5.3.1.

Checked locally on Python 3.14 (no GPU), with a parametrized test that uses the benchmark fixture under --benchmark-disable, as the smoke test does:

pytest pytest-benchmark Result
9.2.0 5.3.0 2 errors, AttributeError: 'Function' object has no attribute '_nodeid', as in the CI logs
9.2.0 5.3.1 2 passed
9.1.1 5.3.0 2 passed (the current cap)
  • pip install --dry-run pytest pytest-xdist "pytest-benchmark>=5.3.1" resolves to pytest 9.2.0, pytest-xdist 3.8.0 and pytest-benchmark 5.3.1.
  • Current CI already installs pytest-benchmark 5.3.1, together with the capped pytest 9.1.1.
  • pre-commit run --files passes on both scripts.

Not verified: the cuda.cccl.examples CI lanes themselves.

Checklist

  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

🤖 Generated with Claude Code

https://claude.ai/code/session_01GwUvhmzgfkiZ6vo4uGyesN

NVIDIA#12313 capped pytest below 9.2 in the cuda.cccl.examples lanes because
pytest 9.2.0 removed the private Function._nodeid, which
pytest-benchmark 5.3.0 read in its benchmark fixture
(ionelmc/pytest-benchmark#321). pytest-benchmark 5.3.1 reads the public
nodeid instead, so require it and let pytest float again, as the TODO
asked.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GwUvhmzgfkiZ6vo4uGyesN
@pylaterreur
pylaterreur requested a review from a team as a code owner November 11, 2026 23:15
@copy-pr-bot

copy-pr-bot Bot commented Nov 11, 2026

Copy link
Copy Markdown
Contributor

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@github-project-automation github-project-automation Bot moved this to Todo in CCCL Nov 11, 2026
@cccl-authenticator-app cccl-authenticator-app Bot moved this from Todo to In Review in CCCL Nov 11, 2026
@coderabbitai

coderabbitai Bot commented Nov 11, 2026

Copy link
Copy Markdown
Contributor

Explain Change →

No actionable comments were generated in the recent review.

Summary

Summary by CodeRabbit

  • Chores
    • Updated the examples test setup on Linux and Windows to support newer pytest versions.
    • Raised the minimum supported version of the benchmark testing tool; pytest is no longer restricted to versions below 9.2.

Both Linux and Windows examples test setup scripts remove the pytest version cap and require pytest-benchmark version 5.3.1 or newer.

Changes: Examples test dependencies

Layer / File(s) Summary
Update test install requirements
ci/util/python/run_examples_tests.sh, ci/windows/run_examples_tests.ps1
Both scripts remove the pytest<9.2 constraint and add pytest-benchmark>=5.3.1. The Windows script also removes the related TODO.

Suggested reviewers: gonidelis

Change: Other

Merge Risk: Minimal · up to 653a3

This updates CI test dependency ranges so newer pytest versions work with pytest-benchmark. It does not affect production behavior, and merge risk is minimal. The examples CI lanes should confirm it after merge.

Architecture risk: Low · up to 653a3

The change affects 1 system.

Changed systems: ci

Architecture concerns
No architecture-level concerns identified.

Review details

Systems and components

  • observed — ci (service) was modified; 2 changed files map to changed impact.

Before / after behavior

  • observed — Modified behavior in ci/util/python/run_examples_tests.sh: The install command removes the pytest<9.2 constraint and replaces the unpinned pytest-benchmark requirement with pytest-benchmark>=5.3.1.
  • observed — Modified behavior in ci/windows/run_examples_tests.ps1: The install step removes the pytest<9.2 cap and the corresponding TODO, and adds a pytest-benchmark minimum version of 5.3.1 while retaining upgrades of pip, pytest, and pytest-xdist.
- [ ] Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Recent review info

Run configuration

  • Source mode: Auto
  • Repository: NVIDIA/cccl
  • Branch: main
  • File: .coderabbit.yaml
  • Commit: 5d63af4
  • Layers: Repository YAML
  • Review profile: CHILL
  • Plan: Enterprise
  • Run ID: b88f4601-8cef-441f-ba89-8a951380669e
Commits

Reviewing files that changed from the base of the PR and between 5d63af4 and 653a3c5.

Files selected for processing: 2

  • ci/util/python/run_examples_tests.sh
  • ci/windows/run_examples_tests.ps1

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 10 remain after this review.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: In Review

Development

Successfully merging this pull request may close these issues.

1 participant