Repository navigation
Conversation
…s for the runner, faulthandler in every launched interpreter Three failures on the macOS Intel runner today were the harness, not the code under test: a helper that printed its complete result and then died on signal 11 at interpreter exit; a 4-rank launch that hit its 300 s budget inside a job that took 2 h 15 min for a 15 min suite; and the slot-semaphore test's own 120 s budget. The wrapper the root conftest already puts on every subprocess.run from a test now: * relaunches a command that died by a signal, once, with a RuntimeWarning naming the command and the signal, so the count stays visible in the summary; a repeat fails as before; * scales a test's timeout by PROTEUS_TEST_TIMEOUT_SCALE, a number or "auto" (the load average per core, clamped to [1, 4]); * runs the launched interpreter with PYTHONFAULTHANDLER=1, so a crash in Python prints the stack it happened in, which is what chasing the exit crash down needs. The workflow sets the scale to 3 on the Intel leg and auto elsewhere, and PYTHONFAULTHANDLER=1 for the workers. Six tests. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
[tool.coverage] in pyproject.toml measures the proteus package and nothing else -- not the test or scripts directories, not the decks under proteus/tests -- so that code nothing runs is visible. Branch coverage; parallel data files combined at the end; and coverage's patch = subprocess (7.10+), so every interpreter a test starts measures itself: parun, and the helpers run under mpiexec, where most of the archive's rank code runs. Checked: 40 of the 44 lines of Archiver.allGatherIncremental are reached by the rank processes of the parallel-metadata tests. [tool.coverage.paths] reports the installed package CI runs as the checkout's files. The linux-64 py3.12 leg runs under coverage and keeps coverage.xml and the HTML report as artifacts. pytest-cov >= 7 and coverage >= 7.10 in the environment files (pytest-cov 7 relies on coverage's own subprocess patch rather than a .pth file). Cython extensions are not measured: that needs a linetrace build, a separate and slower one. Four tests pin the configuration and check, with a two-process script, that a child measures itself. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… conftest A conftest is not importable as a module under --import-mode=importlib, which is how CI runs: two tests failed to collect with No module named conftest. The pieces they test come through a launch_policy fixture. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Each xdist worker runs in test_output/<worker id> (test/conftest.py), so --cov-config=../pyproject.toml did not resolve in a worker and coverage raised ConfigError before the first test. Reproduced locally with the same worker layout and fixed: coverage.xml is written and the data files combine. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…r until it has enough points It used the module's shared generator and 40 draws. Where that generator stood depended on which tests the xdist worker had run first, and for n=3 the peak is narrow enough that one linux-aarch64 run got 4 points above the cutoff and failed 'assert checked >= 5'. A generator of its own, and draws until 8 points are above the cutoff, make it independent of order. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…st-cov's settings On the coverage leg this process carries COV_CORE_* and COVERAGE_* variables; the nested 'coverage run' sent its data to the outer measurement and the nested 'coverage report' had nothing to report (exit 1, linux-64). Reproduced locally under --cov and fixed by removing them from the nested environment. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two pieces of test-harness work, one PR.
1. Hardening for the macOS Intel runner
The runner is shared and slow, and the near-term hardware and OS picture is likely to stay unstable; the decision is to chase its failures down and harden the harness rather than retire the platform. Three failures there today were the harness, not the code under test:
test_parallel_space_multigrid: the helper printed its completeRESULTline and then died on signal 11 at interpreter exit (PETSc/MPI teardown).test_without_petsc4py: a 4-rankmpiexechit its 300 s budget inside a job that took 2 h 15 min for a suite that takes 15 min on Linux.test_launch_slots: its driver's own 120 s budget, same cause.The wrapper that
test/conftest.pyalready puts on everysubprocess.runfrom a test (the relaunch of an mpiexec that never started, #95) now also:RuntimeWarningnaming the command, the signal and the tail of stderr, so the count stays visible in the summary instead of vanishing into a green run; a repeat fails as before;timeoutbyPROTEUS_TEST_TIMEOUT_SCALE: a number, orautofor the one-minute load average per core clamped to [1, 4], read at launch;PYTHONFAULTHANDLER=1, so a crash in Python prints the stack it happened in. That, with the warning's stderr tail, is the material for chasing the exit crash down.The workflow sets the scale to 3 on the Intel leg and
autoelsewhere, andPYTHONFAULTHANDLER=1for the xdist workers. Six tests with a fake launcher that finishes its work and then kills itself. The full suite through the new wrapper: 1,806 passed, no relaunch warnings.2. Coverage of the proteus package
[tool.coverage]inpyproject.tomlmeasuresproteus/and nothing else: nottest/, notscripts/, not the decks underproteus/tests. Branch coverage. The point is to make code nothing runs visible, so the measurement has to reach where the suite's work actually happens: most of it is in processes the tests start (parun, the helpers undermpiexec). coverage 7.10'spatch = subprocessdoes that without the old.pthtrick, and it reaches grandchildren throughmpiexec: 40 of the 44 lines ofArchiver.allGatherIncrementalare reached by the rank processes of the parallel-metadata tests. pytest-cov combines the xdist workers' and the children's data files, and[tool.coverage.paths]reports the installed package that CI runs as the checkout's files.The
linux-64 / py3.12leg runs under coverage (one leg; measuring slows the run and the answer is the same everywhere) and keepscoverage.xmland the HTML report as job artifacts. Locally:Environment files:
pytest-cov >= 7.0.0,coverage >= 7.10.Tool choice, briefly.
coverage.pywithpytest-covremains the one that handles this suite's shape: xdist workers, subprocesses, MPI grandchildren, and a combine step with path mapping.slipcoveris faster but has no subprocess or combine story; Codecov or a badge can sit on top of thecoverage.xmlartifact later if wanted. Not measured: the Cython extensions (cfemIntegrals,cmeshTools, the mprans kernels). That needs a build withlinetraceon andCYTHON_TRACEdefined, which is a separate, slower build and aCython.Coverageplugin line in the config; worth doing once the Python-level numbers have settled, since C-level dead code is a different hunt.🤖 Generated with Claude Code