From 6cde60005636f646e9b886a88187463104854c6e Mon Sep 17 00:00:00 2001 From: malmans2 Date: Fri, 30 Oct 2026 14:29:48 +0100 Subject: [PATCH 1/7] add Reference API content --- source/_includes/api-reference.md | 169 ++++++++++++++++++++++++++++++ source/conf.py | 3 +- source/data-access.md | 5 +- 3 files changed, 175 insertions(+), 2 deletions(-) create mode 100644 source/_includes/api-reference.md diff --git a/source/_includes/api-reference.md b/source/_includes/api-reference.md new file mode 100644 index 0000000..de47635 --- /dev/null +++ b/source/_includes/api-reference.md @@ -0,0 +1,169 @@ +(api-reference)= + +## API reference + +Under the hood, Earth Data Hub is just two open interfaces: a **STAC catalogue** to find data, and **Zarr over HTTPS** to read it. No SDK required. If your tool can make a web request, it can use Earth Data Hub. + +(stac-api)= + +### STAC catalogue + +[STAC](https://stacspec.org/) (SpatioTemporal Asset Catalog) is a standard, searchable catalogue of Earth data. Every collection and dataset you see in the Earth Data Hub catalogue is also available as STAC metadata, starting from a single root URL: + +```{code-block} text +--- +substitutions: +--- +|stac_api_url|/ +``` + +```{admonition} One URL to discover everything. +--- +class: note +--- +The STAC catalogue is **public**: browsing it requires no API key. You only need your key when you read the data itself. +``` + +Explore the catalogue straight from your terminal: + +```{code-block} bash +--- +substitutions: +--- +# Root catalogue +curl |stac_api_url|/ + +# All collections (e.g. era5, cmip6, climate-dt-2) +curl |stac_api_url|/collections + +# All datasets in a collection +curl |stac_api_url|/collections/era5/items + +# Search across the catalogue +curl "|stac_api_url|/search?collections=era5&limit=5" +``` + +In STAC, each dataset is an **item**. Its `data` asset holds the **dataset URL**, along with the exact arguments needed to open it with Xarray: + +```{code-block} json +--- +emphasize-lines: 3 +--- +"assets": { + "data": { + "href": "https://data.earthdatahub.destine.eu/era5/era5-land-monthly-v0.zarr", + "roles": ["data"], + "xarray:open_kwargs": {"chunks": {}, "engine": "zarr"}, + "xarray:storage_options": {"client_kwargs": {"trust_env": true}} + } +} +``` + +#### From catalogue to dataset in Python + +Install [pystac-client](https://pystac-client.readthedocs.io/) alongside the packages from [](environment-setup): + +````{tab-set} +--- +class: outline +--- +~~~{tab-item} {iconify}`vscode-icons:file-type-pypi` pip +```bash +pip install pystac-client xarray "zarr>3" dask aiohttp +``` +~~~ + +~~~{tab-item} {iconify}`vscode-icons:file-type-uv` uv +```bash +uv add pystac-client xarray "zarr>3" dask aiohttp +``` +~~~ + +~~~{tab-item} {iconify}`vscode-icons:file-type-conda` conda +```bash +conda install -c conda-forge pystac-client xarray "zarr>3" dask aiohttp +``` +~~~ +```` + +Then find a dataset in the catalogue and open it, with no URLs to copy and paste. This example uses your `.netrc` file, see [](protected-datasets): + +```{code-block} pycon +--- +substitutions: +--- +>>> import pystac_client +>>> import xarray as xr + +>>> catalog = pystac_client.Client.open("|stac_api_url|") +>>> item = catalog.get_collection("era5").get_item("era5-land-monthly") +>>> asset = item.assets["data"] + +>>> xr.open_dataset( +... asset.href, +... storage_options=asset.extra_fields["xarray:storage_options"], +... **asset.extra_fields["xarray:open_kwargs"], +... ) + Size: 1TB +Dimensions: (valid_time: 921, latitude: 1801, longitude: 3600) +Coordinates: + * valid_time (valid_time) datetime64[ns] 7kB 1950-01-01 ... 2026-09-01 + * latitude (latitude) float64 14kB 90.0 89.9 89.8 ... -89.8 -89.9 -90.0 + * longitude (longitude) float64 29kB 0.0 0.1 0.2 0.3 ... 359.7 359.8 359.9 +Data variables: (12/60) + asn (valid_time, latitude, longitude) float32 24GB dask.array + cl (latitude, longitude) float32 26MB dask.array + ... ... + u10 (valid_time, latitude, longitude) float32 24GB dask.array + v10 (valid_time, latitude, longitude) float32 24GB dask.array + z (latitude, longitude) float32 26MB dask.array + +``` + +(zarr-over-https)= + +### Zarr over HTTPS + +A Zarr dataset is simply a **folder of small files** on the web: a few JSON files describing the data, plus the data itself split into chunks. Each of those files has its own URL, so a plain HTTPS request is all it takes to read one. + +```{admonition} If it speaks HTTP, it speaks Zarr. +--- +class: tip +--- +Xarray, `curl`, a browser, or your own code in any language: they all read the same files the same way. +``` + +Try it with the public test dataset: + +```{code-block} bash +--- +substitutions: +--- +# Dataset metadata: variables, dimensions and attributes +curl https://|datastore_internal_host|/public/test-dataset-v1.zarr/zarr.json + +# Variable metadata: shape, chunk shape, data type and compression +curl https://|datastore_internal_host|/public/test-dataset-v1.zarr/demographic_totals/zarr.json + +# A single chunk of raw data +curl -o chunk.bin https://|datastore_internal_host|/public/test-dataset-v1.zarr/year/c/0 +``` + +File names follow the standard [Zarr conventions](https://zarr-specs.readthedocs.io/en/latest/v3/core/index.html). To learn more about how chunks are organised, see [](chunking-schemes). + +For protected datasets, let `curl` read your API key from your `.netrc` file, or pass it directly: + +```{code-block} bash +--- +substitutions: +--- +# Using your .netrc file (recommended) +curl --netrc https://|datastore_internal_host||private_test_dataset_path|/zarr.json + +# Using your API key directly +curl -u edh: https://|datastore_internal_host||private_test_dataset_path|/zarr.json +``` + +```{note} +Every request counts towards your monthly quota, whichever tool you use. See [](quotas). +``` diff --git a/source/conf.py b/source/conf.py index b3bf17c..d3b8b05 100644 --- a/source/conf.py +++ b/source/conf.py @@ -55,11 +55,12 @@ copybutton_prompt_is_regexp = True templates_path = ["_templates"] -exclude_patterns = ["_collections"] +exclude_patterns = ["_collections", "_includes"] myst_enable_extensions = ["attrs_block", "substitution"] myst_substitutions = { + "stac_api_url": f"{BASE_URL}/api/stac/v1", "datastore_host": os.getenv("DATASTORE_HOST", "api.earthdatahub.destine.eu"), "datastore_internal_host": os.getenv( "DATASTORE_INTERNAL_HOST", "data.earthdatahub.destine.eu" diff --git a/source/data-access.md b/source/data-access.md index c8044dc..2bbe8c7 100644 --- a/source/data-access.md +++ b/source/data-access.md @@ -1,7 +1,7 @@ --- myst: html_meta: - description lang=en: This page explains how to access Earth Data Hub datasets using open standards such as Xarray and Zarr, including environment setup, API key authentication, protected datasets, quotas, and local caching. + description lang=en: This page explains how to access Earth Data Hub datasets using open standards such as Xarray and Zarr, including environment setup, API key authentication, protected datasets, quotas, and local caching, plus the STAC catalogue API and direct Zarr access over HTTPS. --- # {edh-octicon}`download` Data Access @@ -230,3 +230,6 @@ ds = xr.open_dataset( Behind the scenes, caching is handled by the **fsspec** library. The `simplecache` backend is a great default for most workflows, while other caching backends provide additional features such as customisable cache expiration and size limits. To learn more about available caching options, see the [fsspec documentation](https://filesystem-spec.readthedocs.io/en/latest/features.html#caching-files-locally). + +```{include} _includes/api-reference.md +``` From bb1a759737bd80d2e12bbdbb774d0c1abff56e80 Mon Sep 17 00:00:00 2001 From: malmans2 Date: Fri, 30 Oct 2026 14:49:38 +0100 Subject: [PATCH 2/7] don't expose dataset as it changes over time --- source/_includes/api-reference.md | 39 +++++++++++-------------------- 1 file changed, 13 insertions(+), 26 deletions(-) diff --git a/source/_includes/api-reference.md b/source/_includes/api-reference.md index de47635..609cfca 100644 --- a/source/_includes/api-reference.md +++ b/source/_includes/api-reference.md @@ -88,35 +88,22 @@ conda install -c conda-forge pystac-client xarray "zarr>3" dask aiohttp Then find a dataset in the catalogue and open it, with no URLs to copy and paste. This example uses your `.netrc` file, see [](protected-datasets): -```{code-block} pycon +```{code-block} python --- substitutions: --- ->>> import pystac_client ->>> import xarray as xr - ->>> catalog = pystac_client.Client.open("|stac_api_url|") ->>> item = catalog.get_collection("era5").get_item("era5-land-monthly") ->>> asset = item.assets["data"] - ->>> xr.open_dataset( -... asset.href, -... storage_options=asset.extra_fields["xarray:storage_options"], -... **asset.extra_fields["xarray:open_kwargs"], -... ) - Size: 1TB -Dimensions: (valid_time: 921, latitude: 1801, longitude: 3600) -Coordinates: - * valid_time (valid_time) datetime64[ns] 7kB 1950-01-01 ... 2026-09-01 - * latitude (latitude) float64 14kB 90.0 89.9 89.8 ... -89.8 -89.9 -90.0 - * longitude (longitude) float64 29kB 0.0 0.1 0.2 0.3 ... 359.7 359.8 359.9 -Data variables: (12/60) - asn (valid_time, latitude, longitude) float32 24GB dask.array - cl (latitude, longitude) float32 26MB dask.array - ... ... - u10 (valid_time, latitude, longitude) float32 24GB dask.array - v10 (valid_time, latitude, longitude) float32 24GB dask.array - z (latitude, longitude) float32 26MB dask.array +import pystac_client +import xarray as xr + +catalog = pystac_client.Client.open("|stac_api_url|") +item = catalog.get_collection("era5").get_item("era5-land") +asset = item.assets["data"] + +xr.open_dataset( + asset.href, + storage_options=asset.extra_fields["xarray:storage_options"], + **asset.extra_fields["xarray:open_kwargs"], +) ``` From 4a6ef566ffac32872a1119a5d5c19563e31b93cf Mon Sep 17 00:00:00 2001 From: malmans2 Date: Fri, 30 Oct 2026 15:23:31 +0100 Subject: [PATCH 3/7] use get --- source/_includes/api-reference.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/_includes/api-reference.md b/source/_includes/api-reference.md index 609cfca..2d7d499 100644 --- a/source/_includes/api-reference.md +++ b/source/_includes/api-reference.md @@ -101,8 +101,8 @@ asset = item.assets["data"] xr.open_dataset( asset.href, - storage_options=asset.extra_fields["xarray:storage_options"], - **asset.extra_fields["xarray:open_kwargs"], + storage_options=asset.extra_fields.get("xarray:storage_options"), + **asset.extra_fields.get("xarray:open_kwargs", {}), ) ``` From 3a22698ca6b870ede71a9a0a5710168620f991f6 Mon Sep 17 00:00:00 2001 From: malmans2 Date: Fri, 30 Oct 2026 15:25:49 +0100 Subject: [PATCH 4/7] remove admoniton --- source/_includes/api-reference.md | 7 ------- 1 file changed, 7 deletions(-) diff --git a/source/_includes/api-reference.md b/source/_includes/api-reference.md index 2d7d499..59c672b 100644 --- a/source/_includes/api-reference.md +++ b/source/_includes/api-reference.md @@ -113,13 +113,6 @@ xr.open_dataset( A Zarr dataset is simply a **folder of small files** on the web: a few JSON files describing the data, plus the data itself split into chunks. Each of those files has its own URL, so a plain HTTPS request is all it takes to read one. -```{admonition} If it speaks HTTP, it speaks Zarr. ---- -class: tip ---- -Xarray, `curl`, a browser, or your own code in any language: they all read the same files the same way. -``` - Try it with the public test dataset: ```{code-block} bash From 862f1261a0d500e76f86027aad414fab3b2132b4 Mon Sep 17 00:00:00 2001 From: malmans2 Date: Fri, 30 Oct 2026 16:32:35 +0100 Subject: [PATCH 5/7] Luca's suggestions --- source/_includes/api-reference.md | 22 ++++++++++------------ 1 file changed, 10 insertions(+), 12 deletions(-) diff --git a/source/_includes/api-reference.md b/source/_includes/api-reference.md index 59c672b..3f0819f 100644 --- a/source/_includes/api-reference.md +++ b/source/_includes/api-reference.md @@ -8,14 +8,7 @@ Under the hood, Earth Data Hub is just two open interfaces: a **STAC catalogue** ### STAC catalogue -[STAC](https://stacspec.org/) (SpatioTemporal Asset Catalog) is a standard, searchable catalogue of Earth data. Every collection and dataset you see in the Earth Data Hub catalogue is also available as STAC metadata, starting from a single root URL: - -```{code-block} text ---- -substitutions: ---- -|stac_api_url|/ -``` +[STAC](https://stacspec.org/) (SpatioTemporal Asset Catalog) is a standard, searchable catalogue of Earth data. Every collection and dataset you see in the Earth Data Hub catalogue is also available as STAC metadata, starting from a single root URL: {{ stac_api_url }} ```{admonition} One URL to discover everything. --- @@ -43,6 +36,8 @@ curl |stac_api_url|/collections/era5/items curl "|stac_api_url|/search?collections=era5&limit=5" ``` +For the full list of endpoints and query parameters, see the {edh_url}`interactive STAC API documentation `. + In STAC, each dataset is an **item**. Its `data` asset holds the **dataset URL**, along with the exact arguments needed to open it with Xarray: ```{code-block} json @@ -131,17 +126,20 @@ curl -o chunk.bin https://|datastore_internal_host|/public/test-dataset-v1.zarr/ File names follow the standard [Zarr conventions](https://zarr-specs.readthedocs.io/en/latest/v3/core/index.html). To learn more about how chunks are organised, see [](chunking-schemes). -For protected datasets, let `curl` read your API key from your `.netrc` file, or pass it directly: +For protected datasets, add your API key to the request: ```{code-block} bash --- substitutions: --- -# Using your .netrc file (recommended) -curl --netrc https://|datastore_internal_host||private_test_dataset_path|/zarr.json - # Using your API key directly curl -u edh: https://|datastore_internal_host||private_test_dataset_path|/zarr.json + +# Using your API key (with bearer auth) +curl https://|datastore_internal_host||private_test_dataset_path|/zarr.json -H "Authorization: Bearer " + +# Using your .netrc file +curl --netrc https://|datastore_internal_host||private_test_dataset_path|/zarr.json ``` ```{note} From 10d284d1f6ad2e00c9ce605826985e17ed55a58a Mon Sep 17 00:00:00 2001 From: malmans2 Date: Fri, 30 Oct 2026 17:11:09 +0100 Subject: [PATCH 6/7] API reference in data access --- source/_includes/api-reference.md | 147 ----------------------------- source/conf.py | 2 +- source/data-access.md | 148 ++++++++++++++++++++++++++++++ 3 files changed, 149 insertions(+), 148 deletions(-) delete mode 100644 source/_includes/api-reference.md diff --git a/source/_includes/api-reference.md b/source/_includes/api-reference.md deleted file mode 100644 index 3f0819f..0000000 --- a/source/_includes/api-reference.md +++ /dev/null @@ -1,147 +0,0 @@ -(api-reference)= - -## API reference - -Under the hood, Earth Data Hub is just two open interfaces: a **STAC catalogue** to find data, and **Zarr over HTTPS** to read it. No SDK required. If your tool can make a web request, it can use Earth Data Hub. - -(stac-api)= - -### STAC catalogue - -[STAC](https://stacspec.org/) (SpatioTemporal Asset Catalog) is a standard, searchable catalogue of Earth data. Every collection and dataset you see in the Earth Data Hub catalogue is also available as STAC metadata, starting from a single root URL: {{ stac_api_url }} - -```{admonition} One URL to discover everything. ---- -class: note ---- -The STAC catalogue is **public**: browsing it requires no API key. You only need your key when you read the data itself. -``` - -Explore the catalogue straight from your terminal: - -```{code-block} bash ---- -substitutions: ---- -# Root catalogue -curl |stac_api_url|/ - -# All collections (e.g. era5, cmip6, climate-dt-2) -curl |stac_api_url|/collections - -# All datasets in a collection -curl |stac_api_url|/collections/era5/items - -# Search across the catalogue -curl "|stac_api_url|/search?collections=era5&limit=5" -``` - -For the full list of endpoints and query parameters, see the {edh_url}`interactive STAC API documentation `. - -In STAC, each dataset is an **item**. Its `data` asset holds the **dataset URL**, along with the exact arguments needed to open it with Xarray: - -```{code-block} json ---- -emphasize-lines: 3 ---- -"assets": { - "data": { - "href": "https://data.earthdatahub.destine.eu/era5/era5-land-monthly-v0.zarr", - "roles": ["data"], - "xarray:open_kwargs": {"chunks": {}, "engine": "zarr"}, - "xarray:storage_options": {"client_kwargs": {"trust_env": true}} - } -} -``` - -#### From catalogue to dataset in Python - -Install [pystac-client](https://pystac-client.readthedocs.io/) alongside the packages from [](environment-setup): - -````{tab-set} ---- -class: outline ---- -~~~{tab-item} {iconify}`vscode-icons:file-type-pypi` pip -```bash -pip install pystac-client xarray "zarr>3" dask aiohttp -``` -~~~ - -~~~{tab-item} {iconify}`vscode-icons:file-type-uv` uv -```bash -uv add pystac-client xarray "zarr>3" dask aiohttp -``` -~~~ - -~~~{tab-item} {iconify}`vscode-icons:file-type-conda` conda -```bash -conda install -c conda-forge pystac-client xarray "zarr>3" dask aiohttp -``` -~~~ -```` - -Then find a dataset in the catalogue and open it, with no URLs to copy and paste. This example uses your `.netrc` file, see [](protected-datasets): - -```{code-block} python ---- -substitutions: ---- -import pystac_client -import xarray as xr - -catalog = pystac_client.Client.open("|stac_api_url|") -item = catalog.get_collection("era5").get_item("era5-land") -asset = item.assets["data"] - -xr.open_dataset( - asset.href, - storage_options=asset.extra_fields.get("xarray:storage_options"), - **asset.extra_fields.get("xarray:open_kwargs", {}), -) - -``` - -(zarr-over-https)= - -### Zarr over HTTPS - -A Zarr dataset is simply a **folder of small files** on the web: a few JSON files describing the data, plus the data itself split into chunks. Each of those files has its own URL, so a plain HTTPS request is all it takes to read one. - -Try it with the public test dataset: - -```{code-block} bash ---- -substitutions: ---- -# Dataset metadata: variables, dimensions and attributes -curl https://|datastore_internal_host|/public/test-dataset-v1.zarr/zarr.json - -# Variable metadata: shape, chunk shape, data type and compression -curl https://|datastore_internal_host|/public/test-dataset-v1.zarr/demographic_totals/zarr.json - -# A single chunk of raw data -curl -o chunk.bin https://|datastore_internal_host|/public/test-dataset-v1.zarr/year/c/0 -``` - -File names follow the standard [Zarr conventions](https://zarr-specs.readthedocs.io/en/latest/v3/core/index.html). To learn more about how chunks are organised, see [](chunking-schemes). - -For protected datasets, add your API key to the request: - -```{code-block} bash ---- -substitutions: ---- -# Using your API key directly -curl -u edh: https://|datastore_internal_host||private_test_dataset_path|/zarr.json - -# Using your API key (with bearer auth) -curl https://|datastore_internal_host||private_test_dataset_path|/zarr.json -H "Authorization: Bearer " - -# Using your .netrc file -curl --netrc https://|datastore_internal_host||private_test_dataset_path|/zarr.json -``` - -```{note} -Every request counts towards your monthly quota, whichever tool you use. See [](quotas). -``` diff --git a/source/conf.py b/source/conf.py index d3b8b05..e25fa87 100644 --- a/source/conf.py +++ b/source/conf.py @@ -55,7 +55,7 @@ copybutton_prompt_is_regexp = True templates_path = ["_templates"] -exclude_patterns = ["_collections", "_includes"] +exclude_patterns = ["_collections"] myst_enable_extensions = ["attrs_block", "substitution"] diff --git a/source/data-access.md b/source/data-access.md index 2bbe8c7..b259f0f 100644 --- a/source/data-access.md +++ b/source/data-access.md @@ -233,3 +233,151 @@ To learn more about available caching options, see the [fsspec documentation](ht ```{include} _includes/api-reference.md ``` + +(api-reference)= + +## API reference + +Under the hood, Earth Data Hub is just two open interfaces: a **STAC catalogue** to find data, and **Zarr over HTTPS** to read it. No SDK required. If your tool can make a web request, it can use Earth Data Hub. + +(stac-api)= + +### STAC catalogue + +[STAC](https://stacspec.org/) (SpatioTemporal Asset Catalog) is a standard, searchable catalogue of Earth data. Every collection and dataset you see in the Earth Data Hub catalogue is also available as STAC metadata, starting from a single root URL: {{ stac_api_url }} + +```{admonition} One URL to discover everything. +--- +class: note +--- +The STAC catalogue is **public**: browsing it requires no API key. You only need your key when you read the data itself. +``` + +Explore the catalogue straight from your terminal: + +```{code-block} bash +--- +substitutions: +--- +# Root catalogue +curl |stac_api_url|/ + +# All collections (e.g. era5, cmip6, climate-dt-2) +curl |stac_api_url|/collections + +# All datasets in a collection +curl |stac_api_url|/collections/era5/items + +# Search across the catalogue +curl "|stac_api_url|/search?collections=era5&limit=5" +``` + +For the full list of endpoints and query parameters, see the {edh_url}`interactive STAC API documentation `. + +In STAC, each dataset is an **item**. Its `data` asset holds the **dataset URL**, along with the exact arguments needed to open it with Xarray: + +```{code-block} json +--- +emphasize-lines: 3 +--- +"assets": { + "data": { + "href": "https://data.earthdatahub.destine.eu/era5/era5-land-monthly-v0.zarr", + "roles": ["data"], + "xarray:open_kwargs": {"chunks": {}, "engine": "zarr"}, + "xarray:storage_options": {"client_kwargs": {"trust_env": true}} + } +} +``` + +#### From catalogue to dataset in Python + +Install [pystac-client](https://pystac-client.readthedocs.io/) alongside the packages from [](environment-setup): + +````{tab-set} +--- +class: outline +--- +~~~{tab-item} {iconify}`vscode-icons:file-type-pypi` pip +```bash +pip install pystac-client xarray "zarr>3" dask aiohttp +``` +~~~ + +~~~{tab-item} {iconify}`vscode-icons:file-type-uv` uv +```bash +uv add pystac-client xarray "zarr>3" dask aiohttp +``` +~~~ + +~~~{tab-item} {iconify}`vscode-icons:file-type-conda` conda +```bash +conda install -c conda-forge pystac-client xarray "zarr>3" dask aiohttp +``` +~~~ +```` + +Then find a dataset in the catalogue and open it, with no URLs to copy and paste. This example uses your `.netrc` file, see [](protected-datasets): + +```{code-block} python +--- +substitutions: +--- +import pystac_client +import xarray as xr + +catalog = pystac_client.Client.open("|stac_api_url|") +item = catalog.get_collection("era5").get_item("era5-land") +asset = item.assets["data"] + +xr.open_dataset( + asset.href, + storage_options=asset.extra_fields.get("xarray:storage_options"), + **asset.extra_fields.get("xarray:open_kwargs", {}), +) + +``` + +(zarr-over-https)= + +### Zarr over HTTPS + +A Zarr dataset is simply a **folder of small files** on the web: a few JSON files describing the data, plus the data itself split into chunks. Each of those files has its own URL, so a plain HTTPS request is all it takes to read one. + +Try it with the public test dataset: + +```{code-block} bash +--- +substitutions: +--- +# Dataset metadata: variables, dimensions and attributes +curl https://|datastore_internal_host|/public/test-dataset-v1.zarr/zarr.json + +# Variable metadata: shape, chunk shape, data type and compression +curl https://|datastore_internal_host|/public/test-dataset-v1.zarr/demographic_totals/zarr.json + +# A single chunk of raw data +curl -o chunk.bin https://|datastore_internal_host|/public/test-dataset-v1.zarr/year/c/0 +``` + +File names follow the standard [Zarr conventions](https://zarr-specs.readthedocs.io/en/latest/v3/core/index.html). To learn more about how chunks are organised, see [](chunking-schemes). + +For protected datasets, add your API key to the request: + +```{code-block} bash +--- +substitutions: +--- +# Using your API key directly +curl -u edh: https://|datastore_internal_host||private_test_dataset_path|/zarr.json + +# Using your API key (with bearer auth) +curl https://|datastore_internal_host||private_test_dataset_path|/zarr.json -H "Authorization: Bearer " + +# Using your .netrc file +curl --netrc https://|datastore_internal_host||private_test_dataset_path|/zarr.json +``` + +```{note} +Every request counts towards your monthly quota, whichever tool you use. See [](quotas). +``` From b63bdfe023896f5d5e523682b31489ab7e8d6491 Mon Sep 17 00:00:00 2001 From: malmans2 Date: Fri, 30 Oct 2026 17:12:49 +0100 Subject: [PATCH 7/7] cleanup --- source/data-access.md | 9 --------- 1 file changed, 9 deletions(-) diff --git a/source/data-access.md b/source/data-access.md index b259f0f..c30afb5 100644 --- a/source/data-access.md +++ b/source/data-access.md @@ -231,17 +231,10 @@ Behind the scenes, caching is handled by the **fsspec** library. The `simplecach To learn more about available caching options, see the [fsspec documentation](https://filesystem-spec.readthedocs.io/en/latest/features.html#caching-files-locally). -```{include} _includes/api-reference.md -``` - -(api-reference)= - ## API reference Under the hood, Earth Data Hub is just two open interfaces: a **STAC catalogue** to find data, and **Zarr over HTTPS** to read it. No SDK required. If your tool can make a web request, it can use Earth Data Hub. -(stac-api)= - ### STAC catalogue [STAC](https://stacspec.org/) (SpatioTemporal Asset Catalog) is a standard, searchable catalogue of Earth data. Every collection and dataset you see in the Earth Data Hub catalogue is also available as STAC metadata, starting from a single root URL: {{ stac_api_url }} @@ -338,8 +331,6 @@ xr.open_dataset( ``` -(zarr-over-https)= - ### Zarr over HTTPS A Zarr dataset is simply a **folder of small files** on the web: a few JSON files describing the data, plus the data itself split into chunks. Each of those files has its own URL, so a plain HTTPS request is all it takes to read one.