Reconciling data years#482
Open
WesIngwersen wants to merge 61 commits into
Open
Conversation
…void unnecessary gcs call
…fear of breaking if false
…ting sector efs in plots
… V in B derivation
…tone_Aq_scaled instead of derive_cornerstone_Aq to ensure proper inflating of A for model 4 and ignoring inflating A for 1-3.
…o reconciling_data_years
…x_analysis and integrate it into main
…ternal callers; add scale_cornerstone_x
…dation check _validate_scaled_x_vnorm_for_B_prerequisites
…http://github.com/cornerstone-data/bedrock into 468-use-equivalent-q-for-a-b-y_nab-in-scaling-method
…nto reconciling_data_years
…rue #get FBS from cache to load from cache. update config flags
…Vnorm_for_B is true
…nto reconciling_data_years
…y with main model builds; removed E caching functions and rely on existing caching. Update handling of model_base_year. Update project-specific model configs to be more complete like existing model configs
…d() when that config flag is set
…_from_authoritative_x flag in derive_cornerstone_Aq_scaled
…hod is true Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add derive_q_from_deflated_ghg_year_x() for model 3a: mirrors the deflate_x_to_detail_io_year_for_B B-matrix approach so q uses GHG-year gross output deflated to 2017 USD rather than 2017 x (which is indistinguishable from the base q). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…is to be the same for all plots
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
cc: @MoLi7
Closes: http://github.com/cornerstone-data/team-in/issues/43
Reconciling data years analysis (
bedrock/analysis/reconciling_data_years/)Four model specifications
model_base_year=yearyearmodel_base_year=yearyearvia summary tablesanalysis.py—main()loopFor each model × year:
LATEST_TARGET_YEAR(2024) USD for cross-year comparison.Output statistics
n_stats.csv— mean and median ofnacross sectors, per model per year.q_ec_correlation.csv— YoY Pearson r (Δq vs Δe_c) andpct_same_sign(fraction of sectors where q and e_c move in the same direction) per model per year-transition.q_ec_correlation_summary.csv— wide-format pivot of the above with per-model averages across all transitions.efs.csv— long-format record of d, n, q, e_c per model × year × sector.Figures
trends_d_q_ec.png— time-series of e_c, q, d, and n indexed to first year = 100, one row per model (including model 1), y-axis shared within each column for direct cross-model comparison.n_yoy_distribution.png— signed YoY violin plot ofnacross models.Additional code changes
B-matrix scaling guard (
derive_cornerstone_B_non_finetuned)scale_cornerstone_Bis now only invoked whenuse_scaled_x_and_scaled_Vnorm_for_B=True, preventing unintended B scaling in configs that do not opt into it.