Repository navigation
test: Split test-aaa-auto.R into one generated file per impl category, checked in CI - #3000
Merged
krlmlr merged 6 commits intoOct 18, 2026
Merged
Conversation
The note asked for snapshot tests and structured tests for each impl. The "Testing Pattern" section of `tools/AGENTS.md` already says so, and the note would not survive the split of the file by category, which has no file for a comment outside a test block. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Dmrf8jeMGkMHFX2nqRKXfD
Pure move, generated by `tools/split-aaa-tests.R` (next commit). Each `test_that()` block, with the comments above it, moves byte-for-byte to `test-aaa-<category>.R`, the file mirroring the `R/aaa-<category>.R` that defines its impl, in the original order. Each snapshot section moves likewise to `_snaps/aaa-<category>.md`. Every file keeps the `skip_on_cran()` header of the old file. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Dmrf8jeMGkMHFX2nqRKXfD
…category's file The script moves each `test_that()` block for a generated `*_impl()`, and its snapshot section, to `test-aaa-<category>.R`, the file mirroring the `R/aaa-<category>.R` that defines the impl. `R/aaa-<category>.R` comes from `tools/aaa-categories.yaml` through `tools/split-aaa-auto.R`, so a rerun after an impl changes category moves its tests along. With `--check`, it writes nothing and fails when a rerun would change a file; CI runs it so in the after-install step. The handbook and `tools/AGENTS.md` now name the per-category files. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Dmrf8jeMGkMHFX2nqRKXfD
krlmlr
added this pull request to stack #3001
October 18, 2026 13:00
`Config/testthat/start-first` still named `aaa-auto`, which matches no file since `test-aaa-auto.R` was split into `test-aaa-<category>.R`. testthat reads each entry as a glob on the file name without `test-` and `.R` (`order_test_scripts()` applies `utils::glob2rx()` to it), so `aaa-*` matches exactly the generated `test-aaa-<category>.R` files, and no other test file starts with `aaa-`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JXnA76zWTu4ZD8BQCSYRpk
1 task
Contributor
|
This is how benchmark results would change (along with a 95% confidence interval in relative change) if 8a884a2 is merged into claude/v2-warning-reentrancy:
|
Both rings of `make_ring(3) + make_ring(4)` share the leading eigenvalue 2, so the vector ARPACK returns from that eigenspace follows from the random perturbation of its start vector. For about four seeds in ten it is one ring of ones and one of zeros, not the expected seven ones. The pin read whatever RNG state the earlier test files in its worker left behind, and the split into `test-aaa-*.R` files, started first, changed that state in the coverage run of the smoke test. `igraph_local_seed()` fixes the state, as elsewhere in the file, and restores it afterwards. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JXnA76zWTu4ZD8BQCSYRpk
Contributor
|
This is how benchmark results would change (along with a 95% confidence interval in relative change) if f9de58a is merged into claude/v2-warning-reentrancy:
|
The pins of `hits_scores()`, `eigen_centrality()` and `cluster_infomap()` draw from R's RNG, the first two for ARPACK's start vector and the third for its search, and left it advanced for the test files that ran after them in the same worker. `setup.R` asks every test that advances the seed to call `igraph_local_seed()`, so each of these blocks now starts with it, and `eigen_centrality()` seeds at the top of its block rather than before the second graph. Across 200 seeds the hits and Infomap results and the star-and-ring case do not change, so the two-ring case stays the only one whose outcome the seed decides, and it still gives seven ones. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JXnA76zWTu4ZD8BQCSYRpk
Contributor
|
This is how benchmark results would change (along with a 95% confidence interval in relative change) if 552eccb is merged into claude/v2-warning-reentrancy:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The tests of the generated
*_impl()functions move from one 11,000-linetest-aaa-auto.Rinto one file per category, mirroring theR/aaa-<category>.Rfiles that define the impls, and a script keeps them there. It is stacked on #2999.What it changes
test-aaa-auto.Rgoes. It asked for snapshot and structured tests for each impl, which the "Testing Pattern" section oftools/AGENTS.mdalready says, and it would have no file to live in after the split.test-aaa-auto.Ris split into 26 filestest-aaa-<category>.R, and_snaps/aaa-auto.mdinto the matching_snaps/aaa-<category>.md. This commit is a pure move generated by the script below: eachtest_that()block, with the comments above it, and each snapshot section moves byte for byte, in the original order, and every file keeps theskip_on_cran()header of the old one.tools/split-aaa-tests.Rputs each impl's tests in the file of the categoryR/aaa-<category>.Rassigns it, so a rerun after an impl changes category intools/aaa-categories.yamlmoves its tests along. With--checkit writes nothing and fails when a rerun would change a file; CI runs it so in the after-install step. Top-level code other than test blocks is an error, because it would have no file to move to.testing/suite/,architecture/r-layer/new-functions/) andtools/AGENTS.mdname the per-category files.Checks
Rscript tools/split-aaa-tests.R --checkreports all 729 impl tests in place, and a rerun without--checkwrites nothing.testthat::test_local(). Before the second commit of fix: Calling igraph from a warning handler no longer corrupts the call that warned #2999 they did not: six snapshots lost the call of their warning underload_all(), which is how that problem was found.devtools::document()andair formatchange nothing, and the handbook check script is clean.Config/testthat/start-firstinDESCRIPTIONnow namesaaa-*in place ofaaa-auto, a glob testthat accepts, so the split files still start first.Starting the split files first changed the RNG state that each test worker hands on to later files, and that exposed an unseeded pin from #2997:
eigen_centrality()onmake_ring(3) + make_ring(4), whose two rings share the leading eigenvalue, returns seven ones or one ring of zeros depending on the random perturbation of ARPACK's start vector, the zeros for about four seeds in ten. The coverage run of the smoke test hit the zeros. The pin now callsigraph_local_seed(42), like the other seeded tests intest-centrality.R, and so do the other #2997 pins that draw random numbers,hits_scores()on a ring andcluster_infomap()on the karate graph, whose results do not depend on the seed but which left the RNG advanced for the files after them, against the rule insetup.R.LLM use: this PR was prepared with Claude Code (Claude Opus 5.5), at the maintainer's request. The change comes from the subsession https://claude.ai/code/session_01Dmrf8jeMGkMHFX2nqRKXfD.
Generated by Claude Code