Repository navigation
Bring rubem calibrate to parity with the previous calibration workflow #348
Description
Description
A comparison of rubem calibrate (#343, #345) with the calibration workflow the model was calibrated with before (a SciPy differential evolution driving one model process per evaluation, with a spreadsheet handoff and per-station workbooks) found capabilities of that workflow the command does not offer yet, and definitions that differ in ways a user of the previous workflow would notice.
Proposed solution
Required for parity:
- Per-parameter bounds for a run (
--bound NAME=MIN:MAX, repeatable;CalibrationSettings.bounds), narrowing the ranges ofappsettings.jsononly, recorded inresult.json. The previous workflow searched alpha in 1 to 10, the weights in 0.1 to 1 and RCD in 0.1 to 10. - Fixed parameters (
--fix NAME=VALUE, repeatable;CalibrationSettings.fixed): the coordinate leaves the decision vector, the value is kept in the nine emitted parameters and inresult.json. The previous workflow pinnedxat 0. - Negative observed values are missing values, as the previous workflow's
obs >= 0mask treated them; the pre-flight reports, per station, how many observed values the mask dropped. - Progress on the terminal: the budget line, one line per generation (best objective and best NSE) and the final summary go through the
rubem.progresslogger the command line already routes to standard output, asrubem rundoes.
Also delivered, from the same comparison:
- Observed file without a header (
;-separated, first cell of the first line not the step column label) is refused with a message naming the expected layouts (the model's0;<id>;...table or a PCRaster time series with its header); PCRaster time series files without the header are refused too (Write the PCRaster header in the time series outputs #347 makes the model write the header). - Per-station goodness-of-fit in every evaluation record (valid pairs, mean and standard deviation of both series, Pearson correlation, RMSE, NSE);
stations.csvandbest_<variable>.csvfor the best candidate (its simulated series beside the observed one) in the run directory. --stations ID[,ID...]: the stations that enter the objective; the others are still reported, which gives the calibration/validation split the previous workflow obtained by averaging three of its four stations.observed.csvin the run directory: per station, the valid pairs against the compared steps and the summary statistics of the observations, plus the coverage of the compared window logged before the search.- A run interrupted or crashed after the search started still gets its
evaluations.csv; a worker killed by the system (out of memory) is reported with the advice to lower--workers. - A wall-clock
started_atcolumn in the evaluation records and table, so the order of evaluation and the convergence curve can be reconstructed. --init,--strategyand--polish/--no-polishon the command line, to reproduce the previous settings (latinhypercube, polishing on) when wanted.- The calibration page records the definitions that changed on purpose (exact weights derivation instead of a penalty with a 0.01 tolerance, steps counted as months, a
pointstation must mark one cell, no polishing by default, the new budget arithmetic, memory per worker) and how to find the per-station efficiency of the winner.
Alternative solutions
Accepting the head-less observed file by numbering its columns 1..N would silently mislabel stations whenever a header is present but malformed; refusing it with the expected layouts is safer and the conversion is one header line.
Additional context
Follows #343 and #345; depends on #347 for the header of the time series the model writes.
Activity
soaressgabriel commentedon Sep 18, 2026
Absorbed into #343, which is now the single issue for the calibration command; every item listed here is part of its scope and is delivered by #344.