Skip to content

Fix openshift/network/third-party suite returning 0 tests.#31007

Open
maheshkurund wants to merge 1 commit intoopenshift:mainfrom
maheshkurund:fix-network-third-party-suite
Open

Fix openshift/network/third-party suite returning 0 tests.#31007
maheshkurund wants to merge 1 commit intoopenshift:mainfrom
maheshkurund:fix-network-third-party-suite

Conversation

@maheshkurund
Copy link
Copy Markdown

@maheshkurund maheshkurund commented Apr 14, 2026

Description

The suite qualifier references [Suite:k8s] which is no longer appended to test names after the migration to the OpenShift Tests Extension (OTE) framework. The k8s-tests-ext binary does not add [Suite:k8s] to test names, causing the qualifier to match zero tests and breaking CNI certification workflows.

What

Remove stale [Suite:k8s] qualifier from the openshift/network/third-party suite definition that causes 0 tests to be selected after the OTE migration.

Why

After the OTE migration, Kubernetes tests are provided by the k8s-tests-ext binary which does not append [Suite:k8s] to test names. The qualifier name.contains("[Suite:k8s]") matches zero tests, making openshift-tests run openshift/network/third-party return "no tests to run".

This breaks the CNI certification workflow documented at Red Hat CNI Certification §34.6.

Evidence

INFO[0047] Discovered 8265 total tests
INFO[0047] Applying filter: suite-qualifiers             before=8265
INFO[0049] Filter suite-qualifiers completed - removed 8265 tests  after=0 before=8265 removed=8265
Suite run returned error: no tests to run

Fix

Remove the stale [Suite:k8s] requirement from the qualifier so that sig-network conformance and NetworkPolicy tests are correctly selected regardless of which binary provides them.

Summary by CodeRabbit

  • Tests
    • Updated network conformance test suite selection criteria to refine the test filtering logic for network-related tests.

…igration

The suite qualifier references [Suite:k8s] which is no longer appended
to test names after the migration to the OpenShift Tests Extension (OTE)
framework. The k8s-tests-ext binary does not add [Suite:k8s] to test
names, causing the qualifier to match zero tests and breaking CNI
certification workflows.

Remove the stale [Suite:k8s] requirement from the qualifier so that
sig-network conformance and NetworkPolicy tests are correctly selected
regardless of which binary provides them.
@openshift-merge-bot
Copy link
Copy Markdown
Contributor

Pipeline controller notification
This repo is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.

For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.

This repository is configured in: automatic mode

@openshift-ci openshift-ci bot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Apr 14, 2026
@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci bot commented Apr 14, 2026

Hi @maheshkurund. Thanks for your PR.

I'm waiting for a openshift member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work.

Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Apr 14, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 00408287-0c3e-4696-8bd1-21b245b7af5a

📥 Commits

Reviewing files that changed from the base of the PR and between 9df27cd and c071341.

📒 Files selected for processing (1)
  • pkg/testsuites/standard_suites.go

Walkthrough

The openshift/network/third-party test suite's filter expression was modified by removing the name.contains("[Suite:k8s]") constraint from the Qualifiers field, simplifying the test selection logic to rely solely on [sig-network] containment alongside existing conformance conditions.

Changes

Cohort / File(s) Summary
Test Suite Filter
pkg/testsuites/standard_suites.go
Removed name.contains("[Suite:k8s]") && constraint from the openshift/network/third-party suite's Qualifiers expression, streamlining the network test filter condition.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~3 minutes

🚥 Pre-merge checks | ✅ 10
✅ Passed checks (10 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main change: fixing the openshift/network/third-party suite that was returning 0 tests after OTE migration by removing the stale [Suite:k8s] qualifier.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Stable And Deterministic Test Names ✅ Passed PR modifies test suite qualifier expressions in standard_suites.go, which are static filter patterns. The custom check targets Ginkgo test title declarations (It(), Describe(), etc.), which are not present in this PR.
Test Structure And Quality ✅ Passed The modified file is a test suite configuration file, not a Ginkgo test implementation file. No Ginkgo test structures or quality concerns are present.
Microshift Test Compatibility ✅ Passed The PR only modifies test suite qualifier filters in standard_suites.go, not adding new Ginkgo e2e tests, so the custom check for MicroShift test compatibility is not applicable.
Single Node Openshift (Sno) Test Compatibility ✅ Passed The PR modifies only a test suite qualifier filter in pkg/testsuites/standard_suites.go, not any Ginkgo e2e test implementations. The SNO compatibility check applies only to newly added Ginkgo tests.
Topology-Aware Scheduling Compatibility ✅ Passed PR modifies test suite filtering logic in a test configuration file, not deployment manifests or controllers. No scheduling constraints, affinity rules, or topology-dependent constructs are introduced.
Ote Binary Stdout Contract ✅ Passed File contains only package-level variable declarations with no process-level code, main functions, or stdout write operations.
Ipv6 And Disconnected Network Test Compatibility ✅ Passed This PR modifies test suite qualifiers, not Ginkgo e2e test definitions. The custom check applies only when new Ginkgo e2e tests are added, which is not the case here.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci bot commented Apr 14, 2026

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: maheshkurund
Once this PR has been reviewed and has the lgtm label, please assign bertinatto for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@maheshkurund maheshkurund marked this pull request as ready for review April 14, 2026 14:14
@openshift-ci openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Apr 14, 2026
@openshift-ci openshift-ci bot requested review from p0lyn0mial and sjenning April 14, 2026 14:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant