Skip the rebalance-sensitive consumer tests on 9.5 - #2816
Conversation
The ledger drain and fromOffset consumer tests were written against librdkafka 2.12 and do not hold on the 2.3.0 that BB-806 pinned and that 9.5 keeps: a controlled CI A/B over 25 runs per arm measured 40% job failures on the tree at the time of the downgrade and 80% at current HEAD, against 0/25 on the same trees with librdkafka 2.12. The librdkafka upgrade lands on 9.6 instead, so skip these two describes here rather than change the dependency, and unskip them once 9.6 carries the upgrade. Verified over 25 CI runs of this branch: 51 passing, 3 pending, no failures. Two of the 25 pre-skip runs also failed in the concurrency and rebalance describes, which this change leaves enabled. Issue: BB-849
Hello delthas,My role is to assist you with the merge of this Available options
Available commands
Status report is not available. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files
@@ Coverage Diff @@
## development/9.5 #2816 +/- ##
===================================================
- Coverage 75.76% 75.75% -0.01%
===================================================
Files 200 200
Lines 13922 13922
===================================================
- Hits 10548 10547 -1
- Misses 3364 3365 +1
Partials 10 10
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
Waiting for approvalThe following approvals are needed before I can proceed with the merge:
|
francoisferrand
left a comment
There was a problem hiding this comment.
Let's not merge just yet, still trying to confirm the lib upgrade plan
(depending on it, maybe we can just revert in the 9.6 waterflow commit?)
|
Created the waterflow revert commit on w/9.6/bugfix/BB-849/skip-rebalance-sensitive-lib-tests |
Request integration branchesWaiting for integration branch creation to be requested by the user. To request integration branches, please comment on this pull request with the following command: Alternatively, the |
The
ledger drain on rebalanceandfromOffsetconsumer tests were written against librdkafka 2.12 and do not hold on the 2.3.0 that BB-806 pinned and that 9.5 keeps.A controlled A/B on CI, 25 runs per arm over the full
libsuite, isolates the cause — the first two arms differ only inpackage.jsonandyarn.lock:c16e5ddb)d7f3303d)52b9510f)The librdkafka upgrade lands on 9.6 instead — S3C-11389 raises Federation's message-format pin, which removes the constraint that forced the downgrade in the first place. So this skips the two describes on 9.5 rather than touching the dependency.
Two things worth knowing for later:
concurrencyandrebalancedescribes are deliberately left enabled. They accounted for failures in 2 of the 25 pre-skip runs, so 9.5 may still see occasional flakiness from them.Issue: BB-849