Repository navigation
Add the rubem calibrate command - #344
Merged
Merged
Conversation
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #344 +/- ##
==========================================
+ Coverage 92.87% 93.36% +0.48%
==========================================
Files 64 68 +4
Lines 4365 5260 +895
Branches 558 675 +117
==========================================
+ Hits 4054 4911 +857
- Misses 248 274 +26
- Partials 63 75 +12 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This was referenced Sep 16, 2026
1 task
Add the rubem[calibration] extra (scipy>=1.16.2, the first release with CPython 3.14 wheels on Linux and Windows), include it in the dev extra and in the CI test installation, and give rubem._deps the guard of the optional dependency, whose guidance names the extra. Commit the ci/dev-requirements.in the audit lock was compiled from, with scipy added, and recompile the lock keeping every existing pin.
…ders
rubem.calibration.objective computes the Nash-Sutcliffe efficiency over
the pairs valid in both series (NaN, negative values, -9999 and the
PCRaster missing value are gaps; fewer than two pairs or a constant
observed record give no value), the objective 1000 (100 (1 - NSE))^2 and,
per shared station, the pairs, the means and standard deviations of both
series, the Pearson correlation, the RMSE and the NSE. evaluate_series
aligns a simulated and an observed series on their shared time steps and
station ids after a spin-up window and averages the NSE over a chosen
selection of stations while measuring the others. read_series reads the
table layout of the model's time series ("0;<id>;...") and PCRaster time
series with their header; a file without a header is refused naming the
two accepted layouts, and a byte order mark is ignored.
The search runs over the eight free calibration parameters, with w_3 derived as 1 - w_1 - w_2 and the linear constraint w_1 + w_2 <= 1. decision_space(fixed, bounds) builds the vector of one run: a fixed parameter leaves the vector and is re-inserted in the nine parameters every candidate runs with; a per-run bound may only narrow the range of the application settings; fixing one weight narrows the other so the derived w_3 stays in range and drops the pair constraint, a narrowing that leaves it one value fixes it as well, and fixing both weights checks w_3 before the search.
_worker.evaluate is the module-level entry point a spawned worker runs: an inadmissible candidate is rejected without a run; otherwise the format 1.0 document of the configuration is copied with the candidate's parameters, every raster series disabled and only the target time series enabled into a temporary directory, the model runs in-process through the Python API, the table is compared with the observed series and one JSON record (parameters, NSE, per-station metrics, objective, start time, error) is written atomically; failures are recorded and ranked last. runner.calibrate loads and validates the configuration once, refuses an observed series that shares no time step or station with what the configuration samples, writes observed.csv, and drives SciPy's differential_evolution over a spawn ProcessPoolExecutor with one task per silenced worker. It then consolidates evaluations.csv, runs the best candidate once more to write best_<variable>.csv and stations.csv, and writes result.json and the calibrated configuration in the input's own format. Progress goes through the rubem.progress logger; an interrupted or failed search still consolidates the evaluations, and a worker killed by the system is reported with the advice to lower the worker count.
The command takes the configuration, the observed series and the run directory, and exposes the search as options: the variable, the spin-up window, the generations, the population, the seed, the workers, the temporary directory, per-run bounds (--bound NAME=MIN:MAX), fixed parameters (--fix NAME=VALUE), the stations of the objective, the initialization, the strategy and the polishing. It prints the progress lines and a closing summary listing every artifact, and exits without a traceback on configuration and calibration errors.
…basin Tests marked dataset need the local copy of the published datasets, which RUBEM_DATASET_DIR names; when it is unset, or does not name a directory, they are skipped with the reason, so the suite stays self-contained elsewhere. The smoke test rebases the manifest of the Ipojuca basin onto the local copy, restricts the window to six monthly steps, generates the drainage network once and names it in the configuration (the accumulated runoff is routed on a network lddcreate does not reproduce between calls), takes the series of a base run as the observations, and checks that a one-generation differential evolution from five members finds the configuration again with an objective of exactly zero.
Add the calibration page: what the command fits, the objective with its masks and guards, the free parameters with their bounds, the per-run bounds and fixed parameters, the derived weight, the differential evolution settings and the evaluation budget, the observed series layouts and the fixed drainage network a calibration on the accumulated runoff needs, the artifacts, the process model, what a run prints and hints for large machines. Document the command in the user guide and name the Python entry point in the API page.
soaressgabriel
force-pushed
the
feat/343-calibrate
branch
from
September 18, 2026 19:15
deb85e2 to
e88ed52
Compare
This was referenced Sep 18, 2026
4 tasks done
# Conflicts: # doc/source/api.rst # pyproject.toml
# Conflicts: # doc/source/changelog.rst # rubem/cli.py
- Introduced the --allow-blocking-problems flag in the CLI for the calibrate command to allow searches even with input validation errors. - Updated CalibrationSettings to include allow_blocking_problems parameter, and related documentation in calibration.rst, changelog.rst, and userguide.rst to reflect the new functionality. - Added unit tests to verify the behavior of the new option in both the calibration runner and CLI.
camilabillerbeck
approved these changes
Sep 22, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Checklist
Description
Based on #342 (
feat/341-python-api): the workers run the model throughrubem.api.Model. This pull request carries the whole calibration command; it replaces #346 and #350, whose work is folded in here. The series is meant to be reviewed commit by commit:rubem[calibration](scipy>=1.16.2, the first release with CPython 3.14 wheels on Linux and Windows), in thedevextra and in the CI test installation; the dependency guard inrubem._deps;ci/dev-requirements.incommitted (the lock header pointed at an uncommitted file) andci/dev-requirements.lockrecompiled keeping every pin (scipy==1.18.1,numpy==2.5.3added;pip-audit --strictclean).rubem/calibration/objective.py): the NSE over the pairs valid in both series (NaN, negative values,-9999and the PCRaster missing value are gaps), the objective1000 (100 (1 - NSE))^2,StationMetricsper shared station (pairs, means, standard deviations, Pearsonr, RMSE, NSE),evaluate_series(alignment on shared steps and ids, spin-up, a chosen selection of stations),read_seriesfor the model's table layout and for PCRaster time series with their header; head-less files are refused.parameters.py): eight free parameters,w_3derived,w_1 + w_2 <= 1; per-run bounds that only narrow the settings ranges; fixed parameters that leave the vector; the weight cases (one fixed, one collapsed to a value, both fixed)._worker.py,runner.py): one spawned, silenced worker per evaluation runningModel.run()in-process on a derived format 1.0 document; atomic JSON records;differential_evolution(best1exp,sobol,x0, no polishing,deferred) overexecutor.map; the pre-flight of the observed series;observed.csv,evaluations.csv,stations.csv,best_<variable>.csv,result.jsonand the calibrated configuration; progress throughrubem.progress; interrupt and out-of-memory handling.rubem calibrate): options for the variable, spin-up, generations, population, seed, workers, temporary directory,--bound,--fix,--stations,--init,--strategy,--polish/--no-polish; the progress and the closing summary; exit codes asrubem run.datasetmarker,RUBEM_DATASET_DIRwith a skip reason, and the Ipojuca smoke (six steps, a drainage network generated once, best objective exactly0.0; without the fixed network it is4.4e-7).Related Issue
Motivation and context
Pending, tracked in #343: the rejection of candidates whose weights violate
C_wp <= 1needs the check merged by #336, which this branch does not contain until it is rebased ontomain. Design notes left in the code: a repeated time step of an observed table is aligned on its first row; aBrokenProcessPoolin the final re-run of the best candidate is reported without the--workersadvice (that pool runs one task); the per-generation progress re-reads the records (negligible beside the model runs).How has this been tested
git diff --statagainst that head listsdoc/source/changelog.rstonly.python -m pytest -q -p no:cacheprovider --ignore=tests/integration/doc -n 2: 1445 passed, 2 skipped (Linux, Python 3.13, PCRaster 4.4.2, SciPy 1.18.1). Every numeric expectation was re-derived by hand a second time; mutation checks (record written before the run, constraint dropped,w_3ignoringw_2, mask accepting negatives, record without station metrics, constraint kept with a fixed weight) are caught.--fix x=0 --bound alpha=1:10 --stations 1 --init latinhypercube(7 free parameters, artifacts consistent:stations.csvrecomputed frombest_arn.csvbit for bit); no lingering process.RUBEM_DATASET_DIR=<local datasets> python -m pytest tests/dataset: 1 passed in about 13 s.uvx ruff@0.16.4 format --check .anduvx ruff@0.16.4 check .: clean.sphinx-build -W -b html doc/sourcewith the documentation's Python 3.13 environment: no warnings.uvx pip-audit --strict -r ci/dev-requirements.lock: no known vulnerabilities.Screenshots