OCPBUGS-97809: e2e: fix broken checks and rework netqueue tests to avoid ARM ethtool blackout flakes by oblau · Pull Request #1557 · openshift/cluster-node-tuning-operator

oblau · 2026-07-02T13:06:27Z

This is a rework of the netqueues test suite

Investigation started from ARM CI hitting unexpected timeouts, but it
uncovered that most of these tests could never actually fail in the first
place, regardless of whether the feature worked:

4 of the 5 tuned-config assertions used strings.ContainsAny instead of
strings.Contains -- ContainsAny matches on individual characters, not
substrings, so these checks passed for almost any tuned output.
Every test body was wrapped in a guard that was only ever true because
BeforeEach/AfterEach forced a specific profile state around it; if the
cluster's real starting state differed, the guard evaluated false and
the entire test body was skipped -- reported as a PASS.
Convergence-check failures called Skip(...) instead of failing,
turning "the feature is broken" into "test skipped, no signal."

On top of that, on ARM ethtool -L on the br-ex uplink NIC causes ~30s of
node unreachability, and every test independently re-discovered NICs via
exec during/after profile updates, so it kept landing exec calls inside
that blackout window. The resulting timeout looked identical whether the
feature was actually broken or NIC discovery was just slow -- there was
no way to tell them apart. Between that and the bugs above masking real
signal, this was enough to justify a full rework of the test structure
rather than a handful of small patches.

Changes:

Fixed the strings.ContainsAny -> strings.Contains bug in all 4
affected tuned-config assertions.
Removed the always-true/always-false profile-state guards -- test
bodies now always execute and assert for real.
Removed test_id:40542 as a duplicate of test_id:40308 (identical once
the guard above was gone).
Replaced per-test BeforeEach/AfterEach profile setup/teardown with a
single BeforeAll/AfterAll pass -- cuts profile updates (and their
ethtool blackouts) per suite run from 10-16 down to 5-6.
Discover all multi-queue-capable NICs once in BeforeAll into a
baseline map instead of every test (and every poll tick) rediscovering
them; each test now builds its own scoped view of that baseline before
triggering a profile update, so exec calls needing node access happen
before the blackout window, not during/after it. Skips the suite
upfront if no multi-queue-capable NICs are found.
Convergence check now requires every NIC in the target set to reach the
reserved CPU count, instead of returning success on the first matching
NIC anywhere -- meaningfully stricter for the "all devices" and
wildcard/vendor-ID tests.
Unified Ginkgo step labels/logging across all 5 tests for consistency.

Summary by CodeRabbit

Tests
- Improved end-to-end validation for network queue behavior on multi-queue NICs.
- Added a more reliable NIC discovery, baseline setup, and polling strategy to confirm queue resizing using the reserved CPU baseline.
- Expanded coverage for interface name, wildcard, negated selection, and vendor/device ID matching.
- Validate tuned configuration by waiting for the expected udev regex updates and confirming convergence via polling ethtool combined channel counts.
- Restores the original performance profile after test runs to keep environments consistent.

coderabbitai · 2026-07-02T13:07:26Z

Walkthrough

This PR updates the netqueue e2e performance test to discover multi-queue NICs before profile changes, normalize and restore the PerformanceProfile baseline, and validate selector updates by polling NIC combined channel counts against the reserved CPU count.

Changes

Netqueue e2e test refactor

Layer / File(s)	Summary
Setup and baseline normalization `test/e2e/performanceprofile/functests/1_performance/netqueues.go`	Introduces shared reserved-CPU state, discovers multi-queue NICs before profile updates, skips when none are found, and restores the original PerformanceProfile spec after the suite when it diverges.
Selector cases and convergence checks `test/e2e/performanceprofile/functests/1_performance/netqueues.go`	Updates the interface-name, wildcard, negative-match, and vendor/device selector cases to apply device selectors, wait for tuned config updates, and validate NIC channel convergence.
Polling and discovery helpers `test/e2e/performanceprofile/functests/1_performance/netqueues.go`	Replaces the queue-check helpers with NIC discovery, ethtool channel parsing, convergence polling, and the updated random device picker.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Test as Netqueue test
  participant Profile as PerformanceProfile
  participant Nodes as worker nodes
  participant Tuned as tuned pod
  participant NIC as Node NIC

  Test->>Profile: read reserved CPU cpuset
  Test->>Nodes: discover multi-queue NICs
  Nodes-->>Test: baseline NIC snapshot

  Test->>Profile: normalize Spec.Net baseline
  Test->>Profile: update device selectors
  Profile->>Tuned: render devices_udev_regex
  Test->>Tuned: wait for tuned config update
  Tuned-->>Test: config confirmed

  Test->>NIC: poll combined channels
  NIC-->>Test: channel counts
  Test->>Test: wait until counts match reservedCPUCount

  Test->>Profile: restore initial spec after suite

Important

Pre-merge checks failed

Please resolve all errors before merging. Addressing warnings is optional.

❌ Failed checks (1 error, 1 warning)

Check name	Status	Explanation	Resolution
No-Sensitive-Data-In-Logs	❌ Error	New test logs print `node.Name`/`nodeName` in several Infof/Warningf calls, which can expose internal hostnames in CI logs.	Remove or mask node identifiers in log messages (or gate them behind debug-only logging) before merging.
Microshift Test Compatibility	⚠️ Warning	It uses the OpenShift-specific performance.openshift.io PerformanceProfile API and no MicroShift skip/apigroup/runtime guard is present.	Add a [Skipped:MicroShift] label, an apigroup/runtime guard, or exclude this suite from MicroShift CI.

✅ Passed checks (13 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names	✅ Passed	All Ginkgo titles in netqueues.go are static string literals; no fmt.Sprintf or runtime values appear in test names.
Test Structure And Quality	✅ Passed	PASS: Each It targets one netqueue scenario, all cluster waits are bounded, cleanup restores the profile in AfterAll, and key assertions include diagnostic messages.
Single Node Openshift (Sno) Test Compatibility	✅ Passed	The suite checks `cluster.IsSingleNode()` for timeouts only and contains no HA/multi-node-only scheduling, failover, or node-scaling assumptions.
Topology-Aware Scheduling Compatibility	✅ Passed	Only an e2e test file changed; it adds no manifests/controllers or scheduling constraints like affinity, spread, node selectors, or replicas.
Ote Binary Stdout Contract	✅ Passed	No stdout writes were added in process-level setup; the only new logs use testlog, which writes to GinkgoWriter, not stdout.
Ipv6 And Disconnected Network Test Compatibility	✅ Passed	netqueues.go has no hardcoded IPv4 literals, URL construction, or external hosts; it only uses cluster-internal node exec/tuned interactions.
No-Weak-Crypto	✅ Passed	netqueues.go only handles NIC discovery/ethtool; no crypto packages, weak algorithms, or secret comparisons are present.
Container-Privileges	✅ Passed	The PR only changes e2e test logic in netqueues.go; no privileged, hostPID/Network/IPC, SYS_ADMIN, root, or allowPrivilegeEscalation settings were added.
Title check	✅ Passed	The title accurately summarizes the netqueue e2e test fixes and ARM-related flake reduction.

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

openshift-ci · 2026-07-02T13:09:54Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: oblau
Once this PR has been reviewed and has the lgtm label, please assign jmencak for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

…n netqueue suite strings.ContainsAny(s, chars) reports whether ANY character in chars appears in s -- it does not check for a substring. Since device is a NIC name like "enP2s2f0np0", this check was true for almost any tuned output containing common letters, regardless of whether that specific device name actually appeared in devices_udev_regex. Replace with strings.Contains in the 4 tests that assert tuned picked up the configured device filter: test_id 40543, 40545, 72051, 40668. Every netqueue test body was wrapped in a guard like: if profile.Spec.Net.UserLevelNetworking != nil && *ULN && len(Devices) == 0 { ... } This guard was true in the common case only because BeforeEach unconditionally set Net whenever it was nil, and AfterEach unconditionally reverted to the initial profile after every test. If the *initial* cluster profile already had a non-nil Net with ULN=false or leftover Devices, the guard evaluated false and the entire test body was skipped -- the test reported PASS while verifying nothing. Guards removed from test_id 40308, 40543, 40545, 72051, and 40668. Once the guard is gone, test_id 40308 and 40542 are identical bodies (both call checkDeviceSetWithReservedCPU unconditionally). Dropping 40542 as a duplicate; 40308 covers the same case. getReservedCPUSize re-parsed profile.Spec.CPU.Reserved into a cpuset on every 5s poll, even though the value is static for the whole test run. Compute it once as reservedCPUCount in the BeforeAll and thread it through instead: checkDeviceSetWithReservedCPU's signature changes from taking the whole *performancev2.PerformanceProfile to taking reservedCPUCount int directly. getReservedCPUSize is removed. AI Attribution: AIA Human-AI blend, Content edits, New content, Human-initiated, Reviewed, opus 4.6 high v1.0

…BeforeAll/AfterAll pass - BeforeEach (enable ULN) and AfterEach (revert to initial) never isolated tests: both call profiles.UpdateWithRetry with no wait after, so the next test's own update fires before tuned reconciles the previous one. - Example: test A AfterEach reverts the profile, but test B immediately calls UpdateWithRetry with its own Devices before that revert lands -- B can still see A leftover devices_udev_regex even though "cleanup" ran in between. - No test actually needs a clean starting profile: each one fully overwrites profile.Spec.Net itself or polls until its own expected state converges. The per-test revert bought nothing. - Replaced with one BeforeAll (set baseline once) and one AfterAll (revert once, only if state changed). Side effect: total profile updates per run drop from 10-16 to 5-6.

…tead of per-test rediscovery - checkDeviceSupport was called independently by every test, re-running ethtool discovery over the tuned pod each time. checkDeviceSetWithReservedCPU then polled that same rediscovery every 5s while waiting for convergence. A slow update and a broken feature produced the same timeout -> Skip. - Split checkDeviceSupport into discoverMultiQueueNICs (per-node NIC enumeration, now via nodes.GetNodeInterfaces/node_inspector instead of tuned-pod exec) and getCombinedChannels (single NIC -> combined channel count). One does discovery, one does the ethtool read; each is reusable on its own. - discoverMultiQueueNICs runs once in BeforeAll into baselineMultiQueueNICs (map[string]map[nodes.NodeInterface]int); BeforeAll Skips the whole suite upfront if none are found. Tests read from this baseline instead of rediscovering. - checkDeviceSetWithReservedCPU -> waitForNICsToMatchReservedCPU: no longer mutates a caller map or rediscovers, just polls the baseline. Also fixes a real bug -- the old function returned success on the first NIC that matched reservedCPUCount, even for tests asserting all supported NICs converge. - getRandomNodeDevice updated for the new map type, returns a NodeInterface instead of a bare string. Drops its defensive empty-name check -- discoverMultiQueueNICs never inserts zero-value entries, unlike the old checkDeviceSupport. - 40543/40545/72051/40668 now pull their target device from the baseline instead of calling checkDeviceSupport themselves. profile updates use `var err error` explicitly to avoid re-shadowing the Describe-scoped profile var now that device is a struct, not a string, everywhere. AI Attribution: AIA Human-AI blend, Content edits, New content, Human-initiated, Reviewed, opus 4.6 high v1.0

…line - checkDeviceSupport split into discoverMultiQueueNICs (enumeration) and getCombinedChannels (per-NIC ethtool parsing). Discovery now runs once in BeforeAll into baselineMultiQueueNICs, Skipping the whole suite upfront if none are found, instead of each test re-discovering and skipping individually. - checkDeviceSetWithReservedCPU -> waitForNICsToMatchReservedCPU: takes a NIC map to poll instead of rediscovering. Also fixes a bug where it returned success on the first matching NIC instead of waiting for all of them to converge. - getRandomNodeDevice: updated for the new map type, returns a NodeInterface, drops its now-unneeded empty-name guard. - Each test builds its own scoped NIC map (targetNICs/matchedNICs/ expectedNICs) instead of polling the full baseline, so convergence checks validate only the NICs that test actually touches. Per-test checkDeviceSupport+Skip guards removed, BeforeAll already guarantees them. - Skip-on-error replaced with Expect(err).ToNot(HaveOccurred()) throughout. - profile.Spec.Net setup now only sets .Devices (UserLevelNetworking=true comes from the BeforeAll baseline). - var err error added before some profile fetches so profile, err = ... assigns the Describe-scoped var instead of shadowing it. - nodes.GetByName moved before scoped-map-building/profile update in every test, to keep the exec call out of the post-update ethtool blackout window. - 40545: wildcard check now searches for the derived udev-regex form ("iface.*") instead of the literal device name, since tuned stores wildcards as regex in devices_udev_regex. - 72051: pick device only from nodes with >=2 NICs, a single-NIC node can't demonstrate negation. Restored the negative assertion (pre/post combined-channel comparison on the negated NIC) lost when checkDeviceSupport was removed. Tuned-config check now searches for the negated pattern ("!iface") instead of the bare device name, to avoid a false positive from a leftover config. - 40668: tuned-config check upgraded to a 3-way match (interface name + vendor ID + device ID); it previously only checked the interface name. AI Attribution: AIA Human-AI blend, Content edits, New content, Human-initiated, Reviewed, opus 4.6 high v1.0

coderabbitai

Actionable comments posted: 4

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/e2e/performanceprofile/functests/1_performance/netqueues.go`:
- Around line 403-405: The discovery logic in getAvailableCombinedChannels is
incorrectly treating Combined: 1 as unsupported, which later makes
waitForNICsToMatchReservedCPU time out during convergence. Update the channel
filtering so only truly single-channel NICs are excluded during discovery, and
allow a valid combined value of 1 to be returned as available; keep the fix
localized to getAvailableCombinedChannels and any related call sites around the
combinedChannels check.
- Around line 145-146: The shell-based command construction in the performance
profile tests is interpolating untrusted values into bash, which can allow
injection via profile paths or interface names. Update the command-building in
the affected test helpers, especially the logic around tunedCmd and the
ethtool/grep pipeline, to use argv-style exec calls instead of bash -c and pass
the path/interface as separate arguments. Keep the existing test behavior in the
relevant helpers by locating the command assembly near tunedCmd and the other
matching blocks, and remove any shell string concatenation.
- Line 178: The wildcard matching in the target NIC filtering logic ignores the
error returned by filepath.Match, which can hide malformed patterns and produce
an incomplete target set. Update the matching check in the netqueues selection
code to handle the error explicitly: keep the match result for nic.Name, and if
filepath.Match returns an error, surface it through the surrounding control flow
instead of discarding it. Use the existing netqueues matching path and the
filepath.Match call site to locate the fix.
- Around line 237-249: Build the negative-match expectation from the full
baseline node NIC map, not only nodesWithMultipleNICs, so single-NIC workers are
also validated against the cluster-wide !<iface> profile. Update the
expectedNICs construction in the netqueues test to iterate over the complete
nodes/NIC inventory while still excluding device.Name, and keep the existing
unique symbols like expectedNICs, nodesWithMultipleNICs, and device.Name to
locate the logic.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

Push a commit to this branch (recommended)
Create a new PR with the fixes

ℹ️ Review info

⚙️ Run configuration

Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 83224ca6-eca4-410d-aa29-30a6b46383af

📥 Commits

Reviewing files that changed from the base of the PR and between 508d51a and 3c75326.

📒 Files selected for processing (1)

test/e2e/performanceprofile/functests/1_performance/netqueues.go

…ests - Converted remaining plain `//` comments to By() calls; unified the 4 tests' tuned-config-check block (identical By text, no blank lines) -- they run the same grep, differing only in what they search for. - Added a By() before each test's final convergence check (previously unlabeled); all now end in "converged to reserved CPU count". - Added testlog.Infof after each getRandomNodeDevice call, logging which NIC/node the run picked. - 40545/40668: added missing By("Updating the performance profile"), matching 40543. 72051: split its stale "Enable ULN..." By into the same two-step shape as the other tests. - 40668: removed a redundant duplicate nodes.GetByName call. - AfterAll now logs the profile diff before reverting, mirroring BeforeAll's equivalent branch. - Added a top-of-file comment on the ARM ethtool -L blackout window, and a doc comment on getRandomNodeDevice explaining its random-pick trick. AI Attribution: AIA Human-AI blend, Content edits, New content, Human-initiated, Reviewed, opus 4.6 high v1.0

oblau · 2026-07-05T08:34:49Z

/retest

oblau · 2026-07-06T09:04:35Z

/retest

openshift-ci-robot · 2026-07-06T09:41:34Z

@oblau: This pull request references Jira Issue OCPBUGS-97809, which is valid. The bug has been moved to the POST state.

3 validation(s) were run on this bug

bug is open, matching expected state (open)
bug target version (5.0.0) matches configured target version for branch (5.0.0)
bug is in the state New, which is one of the valid states (NEW, ASSIGNED, POST)

The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

This is a rework of the netqueues test suite

Investigation started from ARM CI hitting unexpected timeouts, but it
uncovered that most of these tests could never actually fail in the first
place, regardless of whether the feature worked:

4 of the 5 tuned-config assertions used strings.ContainsAny instead of
strings.Contains -- ContainsAny matches on individual characters, not
substrings, so these checks passed for almost any tuned output.

Every test body was wrapped in a guard that was only ever true because
BeforeEach/AfterEach forced a specific profile state around it; if the
cluster's real starting state differed, the guard evaluated false and
the entire test body was skipped -- reported as a PASS.

Convergence-check failures called Skip(...) instead of failing,
turning "the feature is broken" into "test skipped, no signal."

On top of that, on ARM ethtool -L on the br-ex uplink NIC causes ~30s of
node unreachability, and every test independently re-discovered NICs via
exec during/after profile updates, so it kept landing exec calls inside
that blackout window. The resulting timeout looked identical whether the
feature was actually broken or NIC discovery was just slow -- there was
no way to tell them apart. Between that and the bugs above masking real
signal, this was enough to justify a full rework of the test structure
rather than a handful of small patches.

Changes:

Fixed the strings.ContainsAny -> strings.Contains bug in all 4
affected tuned-config assertions.

Removed the always-true/always-false profile-state guards -- test
bodies now always execute and assert for real.

Removed test_id:40542 as a duplicate of test_id:40308 (identical once
the guard above was gone).

Replaced per-test BeforeEach/AfterEach profile setup/teardown with a
single BeforeAll/AfterAll pass -- cuts profile updates (and their
ethtool blackouts) per suite run from 10-16 down to 5-6.

Discover all multi-queue-capable NICs once in BeforeAll into a
baseline map instead of every test (and every poll tick) rediscovering
them; each test now builds its own scoped view of that baseline before
triggering a profile update, so exec calls needing node access happen
before the blackout window, not during/after it. Skips the suite
upfront if no multi-queue-capable NICs are found.

Convergence check now requires every NIC in the target set to reach the
reserved CPU count, instead of returning success on the first matching
NIC anywhere -- meaningfully stricter for the "all devices" and
wildcard/vendor-ID tests.

Unified Ginkgo step labels/logging across all 5 tests for consistency.

Summary by CodeRabbit

Tests

Improved end-to-end validation for network queue behavior on multi-queue NICs.

Added a more reliable NIC discovery, baseline setup, and polling strategy to confirm queue resizing using the reserved CPU baseline.

Expanded coverage for interface name, wildcard, negated selection, and vendor/device ID matching.

Validate tuned configuration by waiting for the expected udev regex updates and confirming convergence via polling ethtool combined channel counts.

Restores the original performance profile after test runs to keep environments consistent.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

oblau · 2026-07-06T09:44:56Z

/verified by oblau

openshift-ci-robot · 2026-07-06T09:45:08Z

@oblau: This PR has been marked as verified by oblau.

Details

In response to this:

/verified by oblau

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-ci · 2026-07-06T15:08:48Z

@oblau: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

yanirq · 2026-07-06T17:58:37Z

-			}
-			By("Updating the performance profile")
+
+		By("Ensuring a baseline of UserLevelNetworking=true, no device filter")


what is the device filter here?

"Device filter" = profile.Spec.Net.Devices which limits which NICs get netqueue tuning from UserLevelNetworking . Baseline profile should have UserLevelNetworking with no device filters.

openshift-ci Bot requested review from ffromani and yanirq July 2, 2026 13:09

oblau added 4 commits July 2, 2026 16:12

oblau force-pushed the refactor/cnf-25170-netqueues-test-rewrite branch from 3c75326 to c67d344 Compare July 2, 2026 13:13

coderabbitai Bot reviewed Jul 2, 2026

View reviewed changes

oblau force-pushed the refactor/cnf-25170-netqueues-test-rewrite branch from c67d344 to c463c66 Compare July 2, 2026 14:09

yanirq reviewed Jul 2, 2026

View reviewed changes

Comment thread test/e2e/performanceprofile/functests/1_performance/netqueues.go

oblau changed the title ~~e2e: fix broken checks and rework netqueue tests to avoid ARM ethtool blackout flakes~~ OCPBUGS-97809: e2e: fix broken checks and rework netqueue tests to avoid ARM ethtool blackout flakes Jul 6, 2026

openshift-ci-robot added jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. labels Jul 6, 2026

openshift-ci-robot added the verified Signifies that the PR passed pre-merge verification criteria label Jul 6, 2026

yanirq reviewed Jul 6, 2026

View reviewed changes

Uh oh!

Conversation

oblau commented Jul 2, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary by CodeRabbit

Uh oh!

coderabbitai Bot commented Jul 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Sequence Diagram(s)

Pre-merge checks failed

❌ Failed checks (1 error, 1 warning)

Uh oh!

openshift-ci Bot commented Jul 2, 2026

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

oblau commented Jul 5, 2026

Uh oh!

oblau commented Jul 6, 2026

Uh oh!

openshift-ci-robot commented Jul 6, 2026

Summary by CodeRabbit

Uh oh!

oblau commented Jul 6, 2026

Uh oh!

openshift-ci-robot commented Jul 6, 2026

Uh oh!

openshift-ci Bot commented Jul 6, 2026

Uh oh!

yanirq Jul 6, 2026

Choose a reason for hiding this comment

Uh oh!

oblau Jul 7, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

oblau commented Jul 2, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Jul 2, 2026 •

edited

Loading