Skip to content

perf: vectorize permutation test in gravity metric - #54

Merged
cbueth merged 2 commits into
mainfrom
perf/vectorize-gravity
Jul 30, 2026
Merged

perf: vectorize permutation test in gravity metric#54
cbueth merged 2 commits into
mainfrom
perf/vectorize-gravity

Conversation

@cbueth

@cbueth cbueth commented Jul 30, 2026

Copy link
Copy Markdown
Owner

I replaced the serial for-loop over permutations with a single rng.permuted call that generates all n_tests permutations as a 2-D array, then computes gravity in a single vectorized expression.

Before serial loop:

count = 0
for _ in range(n_tests):
    np.random.shuffle(b)
    if grav(a, b) > gravity:
        count += 1

After vectorised:

permuted_b = rng.permuted(np.tile(b, (n_tests, 1)), axis=1)
q1 = np.sum(np.exp(a))
q2 = np.sum(np.exp(permuted_b), axis=1)
gravity_permuted = q1 * q2
return np.sum(gravity_permuted > gravity) / n_tests

Speed grows with n_tests, so it would be ~50 times faster at n_tests=10_000.

As gravity is just a toy model, this is not as relevant, but I wanted to contribute this anyways.

Replace serial loop over permutations with a single rng.permuted call
that processes all n_tests permutations in parallel via array ops.

Add property test: constant ts2 yields p=0 since no permutation changes
the gravity value.

Speedup grows with n_tests — ~50x at n_tests=10_000.
@cbueth cbueth self-assigned this Jul 30, 2026
@cbueth cbueth added the enhancement New feature or request label Jul 30, 2026
@codspeed-hq

codspeed-hq Bot commented Jul 30, 2026

Copy link
Copy Markdown

Merging this PR will regress 2 benchmarks

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 2 improved benchmarks
❌ 2 regressed benchmarks
✅ 24 untouched benchmarks

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Benchmark BASE HEAD Efficiency
test_detrend_method[z_score-p10] 2.1 ms 3.2 ms -35.38%
test_ordinal_patterns 15.9 ms 23.5 ms -32.4%
test_reconstruct[nodes10-gv] 309.4 ms 112.4 ms ×2.8
test_reconstruct[nodes5-gv] 67.6 ms 24.6 ms ×2.7

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing perf/vectorize-gravity (b15c350) with main (875adcd)

Open in CodSpeed

@codecov

codecov Bot commented Jul 30, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (875adcd) to head (b15c350).

Additional details and impacted files
@@            Coverage Diff            @@
##              main       #54   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           29        29           
  Lines          842       847    +5     
  Branches       151       151           
=========================================
+ Hits           842       847    +5     
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@cbueth
cbueth merged commit 5c402c5 into main Jul 30, 2026
11 of 12 checks passed
@cbueth
cbueth deleted the perf/vectorize-gravity branch July 30, 2026 19:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant