Skip to content

Wire allocation tests into the CPU group - #573

Draft
ChrisRackauckas-Claude wants to merge 2 commits into
SciML:masterfrom
ChrisRackauckas-Claude:test/wire-alloc-tests
Draft

ChrisRackauckas-Claude wants to merge 2 commits into
SciML:masterfrom
ChrisRackauckas-Claude:test/wire-alloc-tests

Conversation

@ChrisRackauckas-Claude

@ChrisRackauckas-Claude ChrisRackauckas-Claude commented Oct 7, 2026 •

Copy link
Copy Markdown
Member

Summary

test/alloc_tests.jl was added in e72e1aa gated behind GROUP == "all" || GROUP == "nopre", but neither group exists in test/test_groups.toml or in any workflow — so the tests have never run in CI. This moves the safetestset into the GROUP == "CPU" block (which also covers the "CPU 32-bit" CI group, group = "CPU") and removes the dead gate.

Second commit fixes a Julia-version sensitivity in the test itself: on Julia <= 1.10, @allocated evaluated directly in a @testset body also counts boxing the measured call's result, so the three == 0 checks fail on lts (32 == 0 for lorenz, 16 == 0 for diffeqgpunorm) even though the code under test does not allocate. The measurements are now made inside zero-arg functions, which report 0 on both 1.10 and 1.13 and still detect real allocations. No VERSION gate and no loosened thresholds.

Verification

Machine has no NVIDIA GPU; verified on CPU only. Envs were resolved per Julia version from test/Project.toml with DiffEqGPU Pkg.develop'd at this checkout (not the registered release).

Safetestset wired exactly as runtests.jl includes it, on Julia 1.10.12 before the fix — reproduces the CI failure:

Test Summary:                                              | Pass  Fail  Total   Time
Allocation Tests                                           |    1     3      4  25.2s
  User-defined function should not allocate with SVector   |          1      1   1.8s   (32 == 0)
  make_prob_compatible should be low allocation            |    1            1   4.8s
  diffeqgpunorm should not allocate for SVector            |          1      1   0.2s   (16 == 0)
  diffeqgpunorm should not allocate for scalars            |          1      1   0.0s   (16 == 0)
ERROR: LoadError: Some tests did not pass: 1 passed, 3 failed, 0 errored, 0 broken.

Root cause probe on 1.10.12 (same calls, different measurement site):

testset-scope lorenz:        32      testset-scope gpunorm:     16
function-wrapped lorenz:      0      function-wrapped gpunorm:   0
escaping-allocation control through a wrapper (writes [x,x] into a Ref): 64  -- still detected

After the fix, GROUP=CPU julia +1.10 --project=<dev-env> test/runtests.jl (1.10.12) — full CPU group green, including:

Test Summary:                 | Pass  Total   Time
Adaptive endpoint termination |  132    132  37.1s
Test Summary:    | Pass  Total  Time
Allocation Tests |    4      4  0.2s

GROUP=CPU julia +1 --project=<dev-env> test/runtests.jl (1.13.1) — full CPU group green, tail:

Test Summary:                 | Pass  Total   Time
Adaptive endpoint termination |  132    132  43.7s
Test Summary:    | Pass  Total  Time
Allocation Tests |    4      4  0.2s

No Fail/Error/did not pass entries in either log. runic --inplace made no changes and typos is clean on both touched files.

Not verified / notes

  • test/reverse_ad_tests.jl stays commented out. It was disabled in c550bb9 ("test without reversediff", Fix master tests (v1.10 and v1.11) #348) during the SciMLBase v3-era test fixes, and it still fails today: Zygote's pullback hits SciMLBaseChainRulesCoreExt's rrule for ODESolution, which constructs the solution with an outdated argument list — an upstream SciMLBase/ChainRules issue, not something this repo's test wiring can fix.
  • The tests / CUDA job failure seen in CI (Enzyme ensemble gradients, test/enzyme/cuda_records.jl) is pre-existing on master — tracked in CUDA CI: test/enzyme/cuda_records.jl fails on released CUDACore (Enzyme undef-shadow fill! for non-numeric eltypes) #546 — and unrelated to this diff. No GPU paths were run locally.
  • 32-bit: not run locally; the measurement is now inside functions so the previously observed 1.10-vs-1.13 difference does not apply, and the thresholds are == 0 byte counts / < 1024 on host-side code.
  • Julia 1.11 is not in the CI matrix and was not run.

Please ignore this draft until reviewed by @ChrisRackauckas.

Risk assessment

  • Risk: Low — adds 4 host-side @allocated assertions (~1s) to the CPU group; no src changes.
  • Blast radius: test/runtests.jl dispatch plus the measurement pattern in test/alloc_tests.jl; the CPU 32-bit job also runs it.
  • Evidence: full GROUP=CPU run of test/runtests.jl observed green locally on Julia 1.10.12 and 1.13.1 against the dev'd checkout, including the newly wired safetestset; red-before/green-after for the measurement fix.
  • Independent review: pending
  • Merge: needs review

🤖 Generated with Devin CLI 3000.11.3 (model: swe-2-high). Local session, transcript at /home/crackauc/sandbox/goals/qa-hygiene/jobs/gpu-fixes/log.txt on amdci2.

ChrisRackauckas and others added 2 commits October 7, 2026 09:55
test/alloc_tests.jl was added gated behind GROUP == "all" || "nopre",
but neither group exists in test_groups.toml or any workflow, so the
tests have never run in CI. Include them in the CPU group, where they
pass, and drop the dead gate.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Devin <noreply@cognition.ai>
Agent-Harness: Devin CLI 3000.11.3
Agent-Model: swe-2-high
Agent-Session: local session, transcript at /home/crackauc/sandbox/goals/qa-hygiene/jobs/gpu-fixes/log.txt on amdci2
On Julia <= 1.10, @allocated evaluated directly in a @testset body also
counts boxing the measured call's result, so the three == 0 checks fail on
lts (32 == 0 for lorenz, 16 == 0 for diffeqgpunorm) even though the code
under test does not allocate. The same calls measured inside a function
report 0 on 1.10 and 1.13, and an escaping-allocation control still reports
> 0 through the wrapper. No VERSION gate and no loosened thresholds.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Devin <noreply@cognition.ai>
Agent-Harness: Devin CLI 3000.11.3
Agent-Model: swe-2-high
Agent-Session: local session, transcript at /home/crackauc/sandbox/goals/qa-hygiene/jobs/gpu-fixes/log.txt on amdci2

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants