Skip to content

Add combined NOAA GEFS 0.25 degree forecast and analysis virtual quickstart - #79

Draft
aldenks wants to merge 4 commits into
mainfrom
gefs-virtual-quickstart
Draft

aldenks wants to merge 4 commits into
mainfrom
gefs-virtual-quickstart

Conversation

@aldenks

@aldenks aldenks commented Sep 18, 2026

Copy link
Copy Markdown
Member

One quickstart covering noaa-gefs-forecast-10-day-0-25-degree-virtual, noaa-gefs-analysis-0-25-degree-virtual, noaa-gefs-forecast-16-day-0-5-degree-virtual, and noaa-gefs-forecast-35-day-0-5-degree-virtual, following the multi-dataset shape of ucsb-chc-chirps-analysis.ipynb, the temporary-open pattern of #78, and the AGENTS.md intro/question structure.

Temporary direct open

The datasets are not in the STAC catalog yet, so dynamical_catalog.open(...) cannot resolve them. Cell 2 is the only deviation: an open_virtual(dataset_id, **kwargs) helper opens each Icechunk store directly with anonymous S3 storage plus authorize_virtual_chunk_access for s3://noaa-gefs-pds/. The cell header comment carries complete replacement lines for the three used roots and the two group opens, using group="pressure_level" / group="height_above_mean_sea_level" and ending with the trailing forecast so the repr survives the swap. The unused 35-day root open was removed. The swapped sequence has no undefined names, checked with pyflakes over the concatenated cells with cell 2 replaced by its comment block. Install line stays dynamical-catalog cartopy; no new project dependency.

The four intro links point at the expected catalog pages:

  • https://dynamical.org/catalog/noaa-gefs-forecast-10-day-0-25-degree-virtual/
  • https://dynamical.org/catalog/noaa-gefs-analysis-0-25-degree-virtual/
  • https://dynamical.org/catalog/noaa-gefs-forecast-16-day-0-5-degree-virtual/
  • https://dynamical.org/catalog/noaa-gefs-forecast-35-day-0-5-degree-virtual/

What it shows and why

The original sections retain the 0.25-degree datasets' extra fields and 18 UTC cycles. The new sections show what the 0.5-degree products add: a month-long horizon, pressure-level fields, a 16-day tail, and fixed-height aviation fields. No event is reused from the materialized GEFS quickstarts, #78, another notebook on main, or an earlier section in this notebook.

Section Dataset Field Event / question
Map 0.25-degree analysis snow_thickness_surface (virtual-only) Storm Filomena over Iberia, 2021-01-09T12
Time series 0.25-degree analysis temperature_2m + soil_temperature_0_10cm (virtual-only) February 2021 Texas cold wave at Dallas, 8-22 Feb
How hot did the ensemble expect the July 2022 heat to get? 0.25-degree forecast + analysis maximum_temperature_2m, all 31 members of the 18 UTC run of 13 July vs analysis UK 40 C record at Coningsby
Where did the ensemble put Hurricane Ian's winds? 0.25-degree forecast wind_gust_surface, P(max gust on 28 Sep > 25 m/s), faceted over two 18 UTC runs Ian landfall, 2022-09-28
Did a month-ahead forecast anticipate the January 2021 wind reversal? 0.5-degree 35-day pressure_level 10 hPa / 60 N zonal-mean wind_u, 31 members every 48 h 5 January 2021 sudden stratospheric warming; two members briefly turn easterly after the observed reversal
What did the long-range ensemble show before Portland's record heat? 0.5-degree 16-day root temperature_2m, all 31 members at Portland every 24 h through 384 h June 2021 Pacific Northwest heat wave; 240 h product limit marked
How strong were Storm Ciaran's winds above the Channel Islands? 0.5-degree 16-day height_above_mean_sea_level member wind-speed profiles at all 8 heights 2023-11-02T00, initialized two days earlier
Animation 0.25-degree analysis convective_available_potential_energy_surface (virtual-only), 3-hourly, 15 frames 26-27 April 2024 Plains outbreaks

All reads are bounded before .load(). The new SSW cell reads 18 leads x 31 members = 558 global pressure-level fields; the Portland plume reads 17 leads x 31 members = 527 fields; the Ciaran profile reads 2 wind components x 31 members x 8 heights = 496 fields. No second animation was added. The final notebook is 2,087,159 bytes (1.99 MiB).

Facts re-derived from the stores

The 0.25-degree forecast and analysis carry 34 surface/single-level variables at the beginning of the record, growing to 38 as NOAA added fields through October 2022. Each 0.5-degree forecast has 247 root variables (209 more than the 0.25-degree sets), 12 variables on 31 pressure levels, 6 variables on 4 model levels, and temperature plus two wind components on 8 fixed heights.

The 16-day root has 31 members and 105 leads: 3-hourly through 240 hours, then 6-hourly through 384 hours. The 35-day pressure-level group has 31 members and extends through 840 hours. The Portland selection was re-read from 16-day snapshot XKFE06FV2DCHJ8HW6SR0: the run initialized 13 June 2021 spans 18.3-42.5 C at the nearest grid point on 28 June, well beyond the 240-hour line.

How the 0.5-degree datasets slot in

  • Intro: names all four products, distinguishes the 34-to-38 variable history of the 0.25-degree products from the 247-variable 0.5-degree roots, and includes one documentation bullet per dataset.
  • Cell 2 (open): opens the three used roots plus pressure_level and height_above_mean_sea_level with keyword group=. The header's publication-swap block contains matching dynamical_catalog.open(...) lines for every used object.
  • Structure note: states the measured variable/level counts, initialization cadence, and 3-hour/6-hour lead ranges for both new products.
  • Sections: adds distinct examples for the 35-day pressure-level group, the 16-day forecast tail, and the fixed-height group. The 16-day section is a full 31-member plume and the fixed-height section preserves all members.
  • Community challenge: includes open questions using pressure-level icing, analyzed soil moisture/heat fluxes, and fixed-height wind shear.

Checks (HEAD 6122113)

  • uv run .internal/run_notebooks.py noaa-gefs-forecast-analysis-virtual.ipynb - passed in 158.97 s.
  • uv run python tests/test_notebook_execution.py noaa-gefs-forecast-analysis-virtual.ipynb - passed before and after isolated execution (OK, all notebooks passed validation).
  • uv run .internal/run_notebooks.py --isolated noaa-gefs-forecast-analysis-virtual.ipynb - passed in 117.42 s and printed Every notebook ran against only what its install line names.
  • Publication-swap concatenated sequence - pyflakes passed with no diagnostics.
  • Final file - 2,087,159 bytes, no error or stream outputs, all non-install code cells executed.

Review history

  • 0.25° half: two Claude passes (/tmp/rev79-report.md locally) — fixed the cell-12 xr import after the swap, replaced a non-virtual-only animation, an analysis value quoted as an observation, and the variable-count caveat.
  • 0.5° extension: one Claude pass on 497bc33 — replaced the eclipse section (its plot did not support "spread grows beyond 10 days"), corrected the SSW member count, restored the 34→38 caveat, plotted Ciarán members rather than the mean, dropped the unused 35-day root open. Applied at 6122113.

@aldenks aldenks self-assigned this Sep 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant