Skip to content

Reconciling data years#482

Open
WesIngwersen wants to merge 61 commits into
mainfrom
reconciling_data_years
Open

Reconciling data years#482
WesIngwersen wants to merge 61 commits into
mainfrom
reconciling_data_years

Conversation

@WesIngwersen

@WesIngwersen WesIngwersen commented Jun 22, 2026

Copy link
Copy Markdown
Member

cc: @MoLi7
Closes: http://github.com/cornerstone-data/team-in/issues/43

Reconciling data years analysis (bedrock/analysis/reconciling_data_years/)

Four model specifications

Model A matrix B matrix / x q
1 2017 USEEIO (fixed) 2017 E and V-sum x (fixed) 2017 V column sums
2 2017 USEEIO (fixed) Annual E with BEA time-series x Authoritative x at model_base_year=year
3a 2017 USEEIO (fixed) Annual E with GHG-year x deflated to 2017 GHG-year x deflated to 2017
3b Commodity-PI inflated to year Annual E with BEA time-series x Authoritative x at model_base_year=year
4 Summary-table scaled + commodity-PI inflated Annual E; B scaled 2017→year via summary tables Summary-scaled q

analysis.pymain() loop

For each model × year:

  • Clears all function caches and resets config per iteration.
  • Derives B, A, q, e_c, L, d, and n in current-year dollars.
  • Deflates d and n to LATEST_TARGET_YEAR (2024) USD for cross-year comparison.

Output statistics

  • n_stats.csv — mean and median of n across sectors, per model per year.
  • q_ec_correlation.csv — YoY Pearson r (Δq vs Δe_c) and pct_same_sign (fraction of sectors where q and e_c move in the same direction) per model per year-transition.
  • q_ec_correlation_summary.csv — wide-format pivot of the above with per-model averages across all transitions.
  • efs.csv — long-format record of d, n, q, e_c per model × year × sector.

Figures

  • trends_d_q_ec.png — time-series of e_c, q, d, and n indexed to first year = 100, one row per model (including model 1), y-axis shared within each column for direct cross-model comparison.
  • n_yoy_distribution.png — signed YoY violin plot of n across models.

Additional code changes

B-matrix scaling guard (derive_cornerstone_B_non_finetuned)

scale_cornerstone_B is now only invoked when use_scaled_x_and_scaled_Vnorm_for_B=True, preventing unintended B scaling in configs that do not opt into it.

…tone_Aq_scaled instead of derive_cornerstone_Aq to ensure proper inflating of A for model 4 and ignoring inflating A for 1-3.
…dation check _validate_scaled_x_vnorm_for_B_prerequisites
WesIngwersen and others added 19 commits June 18, 2026 17:44
…rue #get FBS from cache to load from cache. update config flags
…y with main model builds; removed E caching functions and rely on existing caching. Update handling of model_base_year. Update project-specific model configs to be more complete like existing model configs
…_from_authoritative_x flag in derive_cornerstone_Aq_scaled
…hod is true

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add derive_q_from_deflated_ghg_year_x() for model 3a: mirrors the
deflate_x_to_detail_io_year_for_B B-matrix approach so q uses
GHG-year gross output deflated to 2017 USD rather than 2017 x
(which is indistinguishable from the base q).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@WesIngwersen WesIngwersen requested a review from MoLi7 June 24, 2026 15:39
@WesIngwersen WesIngwersen added this to the v0.4 milestone Jun 26, 2026
@WesIngwersen WesIngwersen requested review from jvendries and removed request for MoLi7 and jvendries June 28, 2026 13:24
@WesIngwersen WesIngwersen self-assigned this Jun 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants