Skip to content

test: Split test-aaa-auto.R into one generated file per impl category, checked in CI - #3000

Merged
krlmlr merged 6 commits into
claude/v2-warning-reentrancyfrom
claude/k3-split-aaa-auto-tests
Oct 18, 2026
Merged

krlmlr merged 6 commits into
claude/v2-warning-reentrancyfrom
claude/k3-split-aaa-auto-tests

Conversation

@krlmlr

@krlmlr krlmlr commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor

The tests of the generated *_impl() functions move from one 11,000-line test-aaa-auto.R into one file per category, mirroring the R/aaa-<category>.R files that define the impls, and a script keeps them there. It is stacked on #2999.

What it changes

  • The file-level note at the top of test-aaa-auto.R goes. It asked for snapshot and structured tests for each impl, which the "Testing Pattern" section of tools/AGENTS.md already says, and it would have no file to live in after the split.
  • test-aaa-auto.R is split into 26 files test-aaa-<category>.R, and _snaps/aaa-auto.md into the matching _snaps/aaa-<category>.md. This commit is a pure move generated by the script below: each test_that() block, with the comments above it, and each snapshot section moves byte for byte, in the original order, and every file keeps the skip_on_cran() header of the old one.
  • tools/split-aaa-tests.R puts each impl's tests in the file of the category R/aaa-<category>.R assigns it, so a rerun after an impl changes category in tools/aaa-categories.yaml moves its tests along. With --check it writes nothing and fails when a rerun would change a file; CI runs it so in the after-install step. Top-level code other than test blocks is an error, because it would have no file to move to.
  • The handbook (testing/suite/, architecture/r-layer/new-functions/) and tools/AGENTS.md name the per-category files.

Checks

  • Rscript tools/split-aaa-tests.R --check reports all 729 impl tests in place, and a rerun without --check writes nothing.
  • Comparing sorted lines, the 26 new test files hold exactly the lines of the old file plus the two-line header repeated per file, and the snapshot files hold exactly the lines of the old one.
  • All 729 blocks pass under testthat::test_local(). Before the second commit of fix: Calling igraph from a warning handler no longer corrupts the call that warned #2999 they did not: six snapshots lost the call of their warning under load_all(), which is how that problem was found.
  • devtools::document() and air format change nothing, and the handbook check script is clean.

Config/testthat/start-first in DESCRIPTION now names aaa-* in place of aaa-auto, a glob testthat accepts, so the split files still start first.

Starting the split files first changed the RNG state that each test worker hands on to later files, and that exposed an unseeded pin from #2997: eigen_centrality() on make_ring(3) + make_ring(4), whose two rings share the leading eigenvalue, returns seven ones or one ring of zeros depending on the random perturbation of ARPACK's start vector, the zeros for about four seeds in ten. The coverage run of the smoke test hit the zeros. The pin now calls igraph_local_seed(42), like the other seeded tests in test-centrality.R, and so do the other #2997 pins that draw random numbers, hits_scores() on a ring and cluster_infomap() on the karate graph, whose results do not depend on the seed but which left the RNG advanced for the files after them, against the rule in setup.R.

LLM use: this PR was prepared with Claude Code (Claude Opus 5.5), at the maintainer's request. The change comes from the subsession https://claude.ai/code/session_01Dmrf8jeMGkMHFX2nqRKXfD.

  • By submitting this pull request, I assign the copyright of my contribution to The igraph development team.

Generated by Claude Code

The note asked for snapshot tests and structured tests for each impl.
The "Testing Pattern" section of `tools/AGENTS.md` already says so,
and the note would not survive the split of the file by category,
which has no file for a comment outside a test block.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dmrf8jeMGkMHFX2nqRKXfD
Pure move, generated by `tools/split-aaa-tests.R` (next commit).
Each `test_that()` block, with the comments above it, moves byte-for-byte
to `test-aaa-<category>.R`, the file mirroring the `R/aaa-<category>.R`
that defines its impl, in the original order.
Each snapshot section moves likewise to `_snaps/aaa-<category>.md`.
Every file keeps the `skip_on_cran()` header of the old file.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dmrf8jeMGkMHFX2nqRKXfD
…category's file

The script moves each `test_that()` block for a generated `*_impl()`,
and its snapshot section,
to `test-aaa-<category>.R`, the file mirroring the `R/aaa-<category>.R`
that defines the impl.
`R/aaa-<category>.R` comes from `tools/aaa-categories.yaml`
through `tools/split-aaa-auto.R`,
so a rerun after an impl changes category moves its tests along.
With `--check`, it writes nothing and fails when a rerun would change a file;
CI runs it so in the after-install step.

The handbook and `tools/AGENTS.md` now name the per-category files.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dmrf8jeMGkMHFX2nqRKXfD
@krlmlr
krlmlr added this pull request to stack #3001 October 18, 2026 13:00
`Config/testthat/start-first` still named `aaa-auto`,
which matches no file since `test-aaa-auto.R` was split into `test-aaa-<category>.R`.
testthat reads each entry as a glob on the file name without `test-` and `.R`
(`order_test_scripts()` applies `utils::glob2rx()` to it),
so `aaa-*` matches exactly the generated `test-aaa-<category>.R` files,
and no other test file starts with `aaa-`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JXnA76zWTu4ZD8BQCSYRpk
@github-actions

Copy link
Copy Markdown
Contributor

This is how benchmark results would change (along with a 95% confidence interval in relative change) if 8a884a2 is merged into claude/v2-warning-reentrancy:

  • ✔️E_named: 228ms -> 228ms [-1.73%, +1.14%]
  • ✔️V_named: 434ms -> 432ms [-0.89%, +0.15%]
  • ✔️all_simple_paths_named: 149ms -> 150ms [-0.51%, +1.55%]
  • ✔️as_adjacency_matrix: 225ms -> 225ms [-0.66%, +0.35%]
  • ✔️as_biadjacency_matrix: 255ms -> 255ms [-0.37%, +0.32%]
  • ✔️as_data_frame_both: 245ms -> 244ms [-1.57%, +0.38%]
  • ✔️as_long_data_frame: 213ms -> 214ms [-0.52%, +0.75%]
  • ✔️degree_few_vertices: 127ms -> 127ms [-0.32%, +1.47%]
  • ✔️ego_order2_named: 157ms -> 157ms [-0.97%, +0.96%]
  • ✔️es_attr_assign: 238ms -> 239ms [-0.6%, +0.87%]
  • ✔️es_attr_filter: 245ms -> 244ms [-1.33%, +0.64%]
  • ✔️graph_from_adjacency_matrix: 268ms -> 267ms [-1.01%, +0.24%]
  • ✔️graph_from_data_frame: 284ms -> 284ms [-0.62%, +0.6%]
  • ✔️head_of_named: 180ms -> 180ms [-0.97%, +0.95%]
  • ✔️incident_edges: 214ms -> 214ms [-0.96%, +1.01%]
  • ✔️max_cliques_named: 26.5ms -> 26.3ms [-1.06%, +0.04%]
  • ✔️neighbors_by_name: 244ms -> 244ms [-0.98%, +0.64%]
  • ✔️union_named: 234ms -> 233ms [-1.25%, +0.76%]
  • ✔️vs_attr_filter: 329ms -> 328ms [-0.97%, +0.3%]
  • ✔️vs_by_name: 305ms -> 303ms [-1.51%, +0.55%]
    Further explanation regarding interpretation and methodology can be found in the documentation.

Both rings of `make_ring(3) + make_ring(4)` share the leading eigenvalue 2,
so the vector ARPACK returns from that eigenspace follows from the random perturbation of its start vector.
For about four seeds in ten it is one ring of ones and one of zeros, not the expected seven ones.
The pin read whatever RNG state the earlier test files in its worker left behind,
and the split into `test-aaa-*.R` files, started first, changed that state in the coverage run of the smoke test.
`igraph_local_seed()` fixes the state, as elsewhere in the file, and restores it afterwards.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JXnA76zWTu4ZD8BQCSYRpk
@github-actions

Copy link
Copy Markdown
Contributor

This is how benchmark results would change (along with a 95% confidence interval in relative change) if f9de58a is merged into claude/v2-warning-reentrancy:

  • ✔️E_named: 230ms -> 228ms [-2.66%, +1.18%]
  • ✔️V_named: 432ms -> 431ms [-0.74%, +0.26%]
  • ✔️all_simple_paths_named: 149ms -> 149ms [-0.86%, +0.72%]
  • ✔️as_adjacency_matrix: 230ms -> 230ms [-0.79%, +0.89%]
  • ✔️as_biadjacency_matrix: 260ms -> 260ms [-0.53%, +1.22%]
  • ✔️as_data_frame_both: 246ms -> 246ms [-0.66%, +0.48%]
  • ✔️as_long_data_frame: 219ms -> 219ms [-1.11%, +1.29%]
  • ✔️degree_few_vertices: 127ms -> 127ms [-1.02%, +0.85%]
  • ✔️ego_order2_named: 157ms -> 157ms [-1.01%, +0.8%]
  • ✔️es_attr_assign: 244ms -> 243ms [-1.8%, +1.01%]
  • ✔️es_attr_filter: 251ms -> 252ms [-1.22%, +1.7%]
  • ✔️graph_from_adjacency_matrix: 272ms -> 272ms [-0.82%, +0.44%]
  • ✔️graph_from_data_frame: 288ms -> 289ms [-0.56%, +1.23%]
  • ✔️head_of_named: 181ms -> 181ms [-0.73%, +1.32%]
  • ✔️incident_edges: 217ms -> 217ms [-0.68%, +1.03%]
  • ✔️max_cliques_named: 26.8ms -> 26.7ms [-1.03%, +0.5%]
  • ✔️neighbors_by_name: 247ms -> 247ms [-0.71%, +0.36%]
  • ✔️union_named: 249ms -> 248ms [-3.43%, +2.86%]
  • ✔️vs_attr_filter: 335ms -> 334ms [-0.91%, +0.13%]
  • ✔️vs_by_name: 311ms -> 311ms [-0.98%, +1.02%]
    Further explanation regarding interpretation and methodology can be found in the documentation.

The pins of `hits_scores()`, `eigen_centrality()` and `cluster_infomap()` draw from R's RNG,
the first two for ARPACK's start vector and the third for its search,
and left it advanced for the test files that ran after them in the same worker.
`setup.R` asks every test that advances the seed to call `igraph_local_seed()`,
so each of these blocks now starts with it, and `eigen_centrality()` seeds at the top of its block rather than before the second graph.
Across 200 seeds the hits and Infomap results and the star-and-ring case do not change,
so the two-ring case stays the only one whose outcome the seed decides, and it still gives seven ones.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JXnA76zWTu4ZD8BQCSYRpk
@github-actions

Copy link
Copy Markdown
Contributor

This is how benchmark results would change (along with a 95% confidence interval in relative change) if 552eccb is merged into claude/v2-warning-reentrancy:

  • ✔️E_named: 219ms -> 217ms [-2.09%, +0.33%]
  • ✔️V_named: 438ms -> 439ms [-0.34%, +0.76%]
  • ✔️all_simple_paths_named: 150ms -> 150ms [-0.29%, +1.18%]
  • ✔️as_adjacency_matrix: 228ms -> 229ms [-0.31%, +0.63%]
  • ✔️as_biadjacency_matrix: 259ms -> 259ms [-0.52%, +0.54%]
  • ✔️as_data_frame_both: 244ms -> 244ms [-0.83%, +0.26%]
  • ✔️as_long_data_frame: 215ms -> 214ms [-0.8%, +0.37%]
  • ✔️degree_few_vertices: 128ms -> 127ms [-0.95%, +0.5%]
  • ✔️ego_order2_named: 157ms -> 157ms [-1.07%, +0.62%]
  • ✔️es_attr_assign: 239ms -> 238ms [-1.32%, +0.34%]
  • ✔️es_attr_filter: 246ms -> 245ms [-1.03%, +0.55%]
  • ✔️graph_from_adjacency_matrix: 272ms -> 272ms [-0.52%, +0.63%]
  • ✔️graph_from_data_frame: 284ms -> 285ms [-0.4%, +0.7%]
  • ✔️head_of_named: 180ms -> 181ms [-0.33%, +1.64%]
  • ✔️incident_edges: 214ms -> 213ms [-1.09%, +0.48%]
  • ✔️max_cliques_named: 26.4ms -> 26.4ms [-0.36%, +0.5%]
  • ✔️neighbors_by_name: 245ms -> 244ms [-0.84%, +0.54%]
  • ✔️union_named: 234ms -> 236ms [-0.82%, +2.03%]
  • ✔️vs_attr_filter: 332ms -> 331ms [-0.68%, +0.49%]
  • ✔️vs_by_name: 305ms -> 305ms [-0.68%, +0.85%]
    Further explanation regarding interpretation and methodology can be found in the documentation.

@krlmlr
krlmlr merged commit 002c141 into main Oct 18, 2026
12 checks passed
@krlmlr
krlmlr deleted the claude/k3-split-aaa-auto-tests branch October 18, 2026 18:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants