Given #1024, one potential improvement to the data commons could be found in its ingestion of Google Earth Engine data. Raster data in GEE can be loaded into Xarray via Xee (https://github.com/google/Xee). Using a “streaming”, chunked programming model afforded by Dask, raster data can be transformed into tabular data in a parallel, memory bounded way (https://docs.xarray.dev/en/stable/generated/xarray.Dataset.to_dask_dataframe.html).
Given #1024, one potential improvement to the data commons could be found in its ingestion of Google Earth Engine data. Raster data in GEE can be loaded into Xarray via Xee (https://github.com/google/Xee). Using a “streaming”, chunked programming model afforded by Dask, raster data can be transformed into tabular data in a parallel, memory bounded way (https://docs.xarray.dev/en/stable/generated/xarray.Dataset.to_dask_dataframe.html).