Skip to content

fix(normalisation): retry on ValueError when sampling null ensemble - #52

Open
cbueth wants to merge 2 commits into
mainfrom
fix/flaky-reciprocity-test
Open

fix(normalisation): retry on ValueError when sampling null ensemble#52
cbueth wants to merge 2 commits into
mainfrom
fix/flaky-reciprocity-test

Conversation

@cbueth

@cbueth cbueth commented Jul 30, 2026

Copy link
Copy Markdown
Owner

The @normalise_against_random decorator generates directed $G(n,m)$ random graphs and feeds them to the metric function to build the null distribution. For small graphs, a random graph can coincidentally end up with a perfectly symmetric adjacency matrix (all edges reciprocated). The reciprocity metric rejects such matrices with ValueError, causing a flaky test failure.

Fix: catch ValueError from the metric function during the sampling loop and retry with a fresh random graph. A for...else guard raises RuntimeError if valid samples cannot be collected within n_rand_val x 10 attempts, ensuring the null distribution always has exactly the requested number of samples.

@cbueth cbueth self-assigned this Jul 30, 2026
@cbueth cbueth added the bug Something isn't working label Jul 30, 2026
@codspeed-hq

codspeed-hq Bot commented Jul 30, 2026

Copy link
Copy Markdown

Merging this PR will improve performance by 11.74%

⚡ 1 improved benchmark
✅ 27 untouched benchmarks

Performance Changes

Benchmark BASE HEAD Efficiency
test_detrend_method[z_score-p10] 2.1 ms 1.8 ms +11.74%

Tip

Curious why this is faster? Comment @codspeedbot explain why this is faster on this PR, or directly use the CodSpeed MCP with your agent.


Comparing fix/flaky-reciprocity-test (3780481) with main (875adcd)

Open in CodSpeed

@codecov

codecov Bot commented Jul 30, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 55.55556% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 99.52%. Comparing base (875adcd) to head (3780481).
⚠️ Report is 5 commits behind head on main.

Additional details and impacted files
@@             Coverage Diff             @@
##              main      #52      +/-   ##
===========================================
- Coverage   100.00%   99.52%   -0.48%     
===========================================
  Files           29       29              
  Lines          842      849       +7     
  Branches       151      152       +1     
===========================================
+ Hits           842      845       +3     
- Misses           0        3       +3     
- Partials         0        1       +1     
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant