Use quantile delta mapping (QDM) to bias adjust forecast, reanalysis - #17
Open
brews wants to merge 5 commits into
Open
Use quantile delta mapping (QDM) to bias adjust forecast, reanalysis#17brews wants to merge 5 commits into
brews wants to merge 5 commits into
Conversation
coldonly projection was merged into the other project_effects script. Removing this to clean up the interface.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #17 +/- ##
=======================================
Coverage 69.79% 69.79%
=======================================
Files 3 3
Lines 96 96
=======================================
Hits 67 67
Misses 29 29 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
brews
marked this pull request as ready for review
August 4, 2026 16:20
Member
Author
|
After talking with the group, I'm holding off on merging this because it has breaking changes to #16. Otherwise, this should be ready to go, AFAIK. I need to switch to work on another project, and I'll be out for a week+ on Friday. I'm moving the assignment to @kemccusker so she has the power. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds prototype quantile-delta mapping (QDM) bias adjustment of ERA5 and season forecast ensembles to GMFD before mortality projection.
To run with this change:
Download forecast ensemble with
scripts/00-download_forecast.py. This now downloads multiple years' worth of forecast ensembles initialized at a particular month. The data is downloaded to a local directory. Currently, this is 1981 - 2026 for May. I let this run overnight. It downloads under 70 GiB of data.Run
scripts/01-parse_forecast.py. This parses the downloaded data and writes its output to POREALLAS_PARSED_FORECAST_URI.Run
scripts/01-parse_era5.pyandscripts/01-parse_gmfd.py. I run this on a daskhub cluster hosted on Google Cloud. It uses ERA5 data from Google's ARCO store (as before) and the CIL's internal GMFD data store. These write output to POREALLAS_PARSED_ERA5_URI and POREALLAS_PARSED_GMFD_URI, respectively.Run
scripts/02-bias_adjust_era5.pyandscripts/02-bias_adjust_forecast.py. These use the data parsed above as input. Output is written to POREALLAS_ERA5_URI and POREALLAS_TAS_FORECAST_URI, respectively. Bias adjustment for ERA5 takes about 3 minutes on my laptop. Bias adjustment of the forecast ensemble takes roughly 2.5 hours.Now you can project mortality as you would have before by running
scripts/05-project_effects.py. This writes full, "hot only" and "cold only" projections of the forecast and baseline period.A few additional things to note:
The README and example.env have been updated to reflect these changes.