Skip to content

CMP-4483: Report Not Applicable instead of ERROR when a CEL rule input type is not served - #1312

Closed
Vincent056 wants to merge 1 commit into
ComplianceAsCode:masterfrom
Vincent056:cmp-4483-cel-not-applicable
Closed

Vincent056 wants to merge 1 commit into
ComplianceAsCode:masterfrom
Vincent056:cmp-4483-cel-not-applicable

Conversation

@Vincent056

Copy link
Copy Markdown

Summary

First half of CMP-4483 (platform applicability for CEL scans): when a CEL rule's kubernetes input references a resource type the cluster does not serve (missing CRD — e.g. SriovNetwork without the SR-IOV Network Operator, or any kubevirt.io type without OpenShift Virtualization), the check currently reports ERROR:

failed to fetch inputs for type kubernetes: failed to fetch Kubernetes resource for input sriovnets:
failed to list resources sriovnetworks: no matches for kind "Sriovnetworks" ...
CEL compilation failed: undeclared reference to 'sriovnets'

The rule isn't in error — it's not applicable. This PR classifies fetch errors whose chain contains meta.NoKindMatchError in the fetcher adapter and maps those rules' results to NOT-APPLICABLE (with an explanatory warning) instead of ERROR. Genuine fetch failures (RBAC, connectivity) still report ERROR. Declaring explicit platform CPEs per rule/profile (the rest of CMP-4483) can layer on top of this.

Validation

  • Unit tests: the classifier accepts the exact wrapped error chain the SDK fetcher produces and rejects generic errors.
  • On-cluster (error shape): reproduced on a live cluster without CNV/SR-IOV — the scanner's fetch path produces precisely the wrapped NoKindMatchError chain the classifier targets (captured in check-result warnings on OCP 4.21, CIS OCP-Virt CEL profile: all 5 kubevirt rules ERROR'd via this path before the fix).
  • Full operator-scan demonstration of the NA results is still pending — the ephemeral validation cluster expired mid-rescan after the image swap. Will attach the result table from the next cluster run; expected: HyperConverged/VM/SR-IOV rules NOT-APPLICABLE, NAD-based rules evaluate normally, zero ERRORs on a virt-less cluster.

Note for reviewers

coverage-baseline.txt is refreshed because the in-image test-unit gate requires exact equality — even a +0.1% improvement fails the build ("baseline is out of date"). It also picks up cmd/celctl, which merged after the baseline was created. The strict-equality behavior itself may be worth revisiting separately: improving coverage in a PR shouldn't require a baseline commit.

🤖 Generated with Claude Code

When a CEL rule's kubernetes input references a resource type the cluster
does not serve (missing CRD - e.g. SriovNetwork without the SR-IOV Network
Operator, or any kubevirt.io type without OpenShift Virtualization), the
fetch fails with NoKindMatchError, the input never binds, CEL compilation
fails with 'undeclared reference', and the check reports ERROR. The rule is
not in error - it is not applicable to the cluster.

Classify a fetch error whose chain contains meta.NoKindMatchError in the
fetcher adapter, record the rule, and map its result to NOT-APPLICABLE
(with an explanatory warning) instead of ERROR in the result conversion.
Genuine fetch errors (RBAC, connectivity) still report ERROR. This is the
CEL-scanner half of the platform-applicability work tracked in CMP-4483;
declaring an explicit platform CPE per rule/profile can build on top.

Includes unit tests for the classification (wrapped NoKindMatchError vs
generic error) and refreshes coverage-baseline.txt (the in-image test-unit
gate requires exact equality, so the baseline also picks up cmd/celctl,
absent since the baseline predates cmd/celctl's merge).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@openshift-ci-robot

Copy link
Copy Markdown
Collaborator

@Vincent056: This pull request references CMP-4483 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "5.0.0" version, but no target version was set.

Details

In response to this:

Summary

First half of CMP-4483 (platform applicability for CEL scans): when a CEL rule's kubernetes input references a resource type the cluster does not serve (missing CRD — e.g. SriovNetwork without the SR-IOV Network Operator, or any kubevirt.io type without OpenShift Virtualization), the check currently reports ERROR:

failed to fetch inputs for type kubernetes: failed to fetch Kubernetes resource for input sriovnets:
failed to list resources sriovnetworks: no matches for kind "Sriovnetworks" ...
CEL compilation failed: undeclared reference to 'sriovnets'

The rule isn't in error — it's not applicable. This PR classifies fetch errors whose chain contains meta.NoKindMatchError in the fetcher adapter and maps those rules' results to NOT-APPLICABLE (with an explanatory warning) instead of ERROR. Genuine fetch failures (RBAC, connectivity) still report ERROR. Declaring explicit platform CPEs per rule/profile (the rest of CMP-4483) can layer on top of this.

Validation

  • Unit tests: the classifier accepts the exact wrapped error chain the SDK fetcher produces and rejects generic errors.
  • On-cluster (error shape): reproduced on a live cluster without CNV/SR-IOV — the scanner's fetch path produces precisely the wrapped NoKindMatchError chain the classifier targets (captured in check-result warnings on OCP 4.21, CIS OCP-Virt CEL profile: all 5 kubevirt rules ERROR'd via this path before the fix).
  • Full operator-scan demonstration of the NA results is still pending — the ephemeral validation cluster expired mid-rescan after the image swap. Will attach the result table from the next cluster run; expected: HyperConverged/VM/SR-IOV rules NOT-APPLICABLE, NAD-based rules evaluate normally, zero ERRORs on a virt-less cluster.

Note for reviewers

coverage-baseline.txt is refreshed because the in-image test-unit gate requires exact equality — even a +0.1% improvement fails the build ("baseline is out of date"). It also picks up cmd/celctl, which merged after the baseline was created. The strict-equality behavior itself may be worth revisiting separately: improving coverage in a PR shouldn't require a baseline commit.

🤖 Generated with Claude Code

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci

openshift-ci Bot commented Jul 28, 2026

Copy link
Copy Markdown

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@openshift-ci

openshift-ci Bot commented Jul 28, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Vincent056

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@github-actions

Copy link
Copy Markdown

🤖 To deploy this PR, run the following command:

make catalog-deploy CATALOG_IMG=ghcr.io/complianceascode/compliance-operator-catalog:1312-87aff2a96230a942497043e0fb06691b5b0da4ce

@openshift-ci

openshift-ci Bot commented Aug 7, 2026

Copy link
Copy Markdown

PR needs rebase.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@openshift-ci

openshift-ci Bot commented Sep 7, 2026

Copy link
Copy Markdown

Stale PRs are closed after 21d of inactivity.

If this PR is still relevant, comment to refresh it or remove the stale label.
Mark the PR as fresh by commenting /remove-lifecycle stale.

If this PR is safe to close now please do so with /close.

/lifecycle stale

@openshift-ci

openshift-ci Bot commented Sep 21, 2026

Copy link
Copy Markdown

Stale PRs rot after 14d of inactivity.

Mark the PR as fresh by commenting /remove-lifecycle rotten.
Rotten PRs close after an additional 7d of inactivity.

If this PR is safe to close now please do so with /close.

/lifecycle rotten
/remove-lifecycle stale

@Vincent056

Copy link
Copy Markdown
Author

We're evaluating supporting CPE in the future instead of this approach, so we're closing this PR.

@Vincent056 Vincent056 closed this Sep 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants