Generating bias curves#2202
Conversation
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
There was a problem hiding this comment.
Pull request overview
This PR adds a new workflow to generate and plot “bias curves” by combining NSB trigger rates derived from sim_telarray log files with proton trigger rates derived from reduced-event HDF5 files, producing plot and ECSV outputs and exposing the functionality via a new CLI application.
Changes:
- Added SIMTEL log crawling/parsing utilities and an NSB trigger-rate calculator with ECSV export.
- Added a bias-curve generator that computes proton trigger rates from HDF5 inputs, plots NSB+proton curves together, and writes combined outputs.
- Registered and documented a new
simtools-derive-bias-curvesapplication; extended plotting utilities and updated trigger-rate API behavior.
Reviewed changes
Copilot reviewed 14 out of 14 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/unit_tests/simtel/test_simtel_log_reader.py | New unit tests for SIMTEL log reading and parsing helpers. |
| tests/unit_tests/simtel/test_nsb_trigger_calculator.py | New unit tests for NSB parsing, aggregation, statistics, and ECSV output. |
| tests/unit_tests/simtel/test_bias_curve_generator.py | New unit tests for end-to-end bias-curve generation logic and helpers. |
| src/simtools/visualization/plot_tables.py | Adds bias-curve plotting helpers and output-path resolution. |
| src/simtools/telescope_trigger_rates.py | Changes trigger-rate function to return results dict and adjusts call flow. |
| src/simtools/simtel/simtel_log_reader.py | New module for reading/crawling SIMTEL log files and extracting counters. |
| src/simtools/simtel/nsb_trigger_calculator.py | New module to derive NSB rates and write ECSV outputs. |
| src/simtools/simtel/bias_curve_generator.py | New module to combine NSB+proton rates, write ECSV, and produce plot. |
| src/simtools/applications/derive_bias_curves.py | New CLI application entrypoint for generating bias curves. |
| pyproject.toml | Registers new simtools-derive-bias-curves console script. |
| docs/source/user-guide/applications/simtools-derive-bias-curves.rst | New application documentation page. |
| docs/source/user-guide/applications.md | Adds the new application to the applications index. |
| docs/source/api-reference/sim_telarray.md | Adds API reference entries for the new sim_telarray support modules. |
| docs/changes/2202.feature.md | Changelog fragment describing the new feature. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| to generate bias curves showing how trigger rates vary with threshold. | ||
|
|
||
| The tool: | ||
| 1. Extracts NSB trigger rates from SIMTEL log files |
There was a problem hiding this comment.
SIMTEL -> sim_telarray
|
|
||
| The tool: | ||
| 1. Extracts NSB trigger rates from SIMTEL log files | ||
| 2. Calculates proton trigger rates from simulation HDF5 files |
There was a problem hiding this comment.
We have different HDF5 files from simulations. I think you mean "reduced event-data HDF5 files"
| The tool: | ||
| 1. Extracts NSB trigger rates from SIMTEL log files | ||
| 2. Calculates proton trigger rates from simulation HDF5 files | ||
| 3. Plots both curves on the same figure for comparison |
There was a problem hiding this comment.
is there any output? Trigger rates as function of thresholds?
|
GernotMaier
left a comment
GernotMaier
left a comment
There was a problem hiding this comment.
Thanks a lot for that!
Below a couple of points and requests for clarification. I will try it tomorrow when you have finalized simulating.
We need to discuss the approach of using log files.
| "--ymin", | ||
| type=float, | ||
| default=1e2, | ||
| help="Minimum y-axis value. Default: 1e2", |
There was a problem hiding this comment.
y-axis of which quantity?
|
|
||
| The input directory should contain both: | ||
| - NSB log files or log_hist archives | ||
| - Proton simulation HDF5 files |
There was a problem hiding this comment.
Maybe add a comment on how the input is generated - simply point to the other application.
| Parameters | ||
| ---------- | ||
| args : dict | ||
| Configuration parameters including: |
There was a problem hiding this comment.
No need to repeat these parameters. I know it is nice to have, but we will forget to update it when changing any of the input parameters.
There was a problem hiding this comment.
Should get an error that none of the input files exist?
simtools-derive-bias-curves --data_dir /path/to/data/ --site North --model_version 7.0.0 --array_layout_name LSTN-01 --output /path/to/output/
INFO: Log messages will be written to: simtools-output/derive_bias_curves_019f662d-5147-705d-8c41-55a5fdce4b3f.log
INFO: simtools application derive_bias_curves started with activity ID 019f662d-5147-705d-8c41-55a5fdce4b3f
INFO: simtools: 0.34.1.dev40+g222adcf23 DB: CTAO-Simulation-Model v0.16.0 CORSIKA: 78010 sim_telarray: v2025-11-30-rc
INFO: Using telescope LSTN-01 from layout ['LSTN-01']
INFO: Telescope LSTN-01: disc_bins=68, fadc_mhz=1024.0 MHz, time_window=0.00 s
INFO: Calculated time window: 66.41 ns
INFO: Extracting NSB trigger rates from log files...
WARNING: Could not extract NSB rates: No *.simtel.log.gz files found in /path/to/data
INFO: Calculating proton trigger rates...
WARNING: No proton HDF5 files with threshold labels found in /path/to/data
INFO: Plotting bias curves...
WARNING: No NSB or proton rates found; writing empty bias-curve plot
INFO: Bias curve plot written to /path/to/output/bias_curve.png
INFO: Bias curve table written to /path/to/output/bias_curve.ecsv
| if not log_files: | ||
| raise FileNotFoundError(f"No log files found in {root_dir} matching pattern '{pattern}'") | ||
|
|
||
| _logger.info(f"Found {len(log_files)} log files in {root_dir}") |
There was a problem hiding this comment.
Is this log statement exactly the same as the one called in line 372 of nsb_trigger_calculator?
|
|
||
| try: | ||
| log_text = read_log_file(file_path) | ||
| except Exception as e: # pylint: disable=broad-exception-caught |
There was a problem hiding this comment.
Note nice. Really absolutely needed?
| _logger.error(f"Failed to read {file_path}: {e}") | ||
| return None | ||
|
|
||
| run_number = extract_run_number(file_path) |
There was a problem hiding this comment.
I think we need to discuss the extraction of information from file names. We don't want to do that.
But let's keep it in this PR and change things in a follow up PR.
| for part in file_path.parts: | ||
| if part.isdigit(): | ||
| threshold = int(part) | ||
| if 10 <= threshold <= 1000: |
There was a problem hiding this comment.
Where does this come from? 10 < ... < 1000?
| if len(run_triggers) > 1 and time_s > 0: | ||
| std_dev = np.std(run_triggers, ddof=1) | ||
| error_triggers = std_dev / np.sqrt(len(run_triggers)) | ||
| error_hz = error_triggers / time_s |
There was a problem hiding this comment.
I am not entirely sure, but I think this underestimates the error. Shouldn't it be something like:
error_hz = 0.0
if len(run_triggers) > 1 and time_s > 0:
std_dev = np.std(run_triggers, ddof=1)
error_triggers = std_dev / np.sqrt(len(run_triggers))
per_run_time_s = time_s / len(run_triggers)
error_hz = error_triggers / per_run_time_s
Added a simtel logfile crawler to read the logfiles to get the number of total events and total triggered events to calculate NSB trigger rate. Outputs a .ecsv file.
Passing hdf5 files to telescope_trigger_rates.py to get the proton trigger rates.
Finally plotting both as a function of threshold on the same plot.
Run with: