Skip to content

Version 1.5: Support for Remote CFA files - #122

Merged
dwest77a merged 57 commits into
mainfrom
v1.5.1
Oct 2, 2026
Merged

Version 1.5: Support for Remote CFA files#122
dwest77a merged 57 commits into
mainfrom
v1.5.1

Conversation

@dwest77a

@dwest77a dwest77a commented Sep 23, 2026

Copy link
Copy Markdown
Collaborator

Key changes:

  • Adds memory limit for validation based on expected memory limit of kernel
  • Changed default behaviour for collecting attributes to collect to a list if not otherwise specified.
  • Bug fixes and improvements to operation of SHEPARD
  • Partial integration for icechunk
  • Adds support for remote CFA files, reworked completion/remote workflow for end of pipeline.
  • Added 'aggregation_date' attribute for kerchunk and CFA aggregation files.

dwest77a added 30 commits April 10, 2026 11:31
@dwest77a
Added memory limit for validation
90b5f47
@dwest77a
Made init group a public function
9c7f883
@dwest77a
Added further configurations for preparation tasks
080006c
@dwest77a
Updated shepard docs
d7a9e95
@dwest77a
Updated keep-vars default
b1e84fb
@dwest77a
Added shepard config
76c083f
@dwest77a
Fixed variable selection
0017f26
@dwest77a
Minor changes
7dca800
@dwest77a
Changed default behaviour for collecting attributes
ce4e2d5
@dwest77a
Minor changes
7a5ba3b
@dwest77a
Changed shepard command
cd37c80
@dwest77a
Added hook for icechunk
3a6f596
@dwest77a
Partial integration for icechunk
51d896e
@dwest77a
Updated Pending status, now logged in the _run function
d3aa0bd
@dwest77a
Updated to v1.5 for cmip7_beta release
a760724
@dwest77a
Added version separator kwarg for completion
df26f4d
@dwest77a
Mapping for fatal errors
1a8c262
@dwest77a
Changed default zattrs value
3ae14f7
@dwest77a
Integrated version separator
ed08b4e
@dwest77a
Added deleting a project on forceful completion
77840a0
@dwest77a
Added completion to phases/timings mappings
d71fa88
@dwest77a
Minor change to compute config
d0b7bd6
@dwest77a
Fixed a bug with new version kwarg
de96da1
@dwest77a
Allow completion of projects individually
a4ce878
@dwest77a
Added initial icechunk compute operation
aa4aa38
@dwest77a
Fixed issues with keep vars
fe18417
@dwest77a
Added final_delete cli flag to delete on project completion
9f49bea
@dwest77a
Increased default compute times for validate, complete
32b57c4
@dwest77a
Fixed issues with box size and time dim check
616d3e1
@dwest77a
Fixed limiter settings on scan for cfa
f69b4ad
dwest77a added 19 commits June 5, 2026 15:47
@dwest77a
Fixed compute subset on scan
d743e44
@dwest77a
New version
23d2ed4
@dwest77a
Fixed critical issue
16df9f4
@dwest77a
Added remote functions for CFA datasets, changed name of function to …
cf16420 
…make datasets remote
@dwest77a
Added cfa_path change for 'remote' option - backwards compatible
1e0741e
@dwest77a
Added remote flag to revision as default - refactored in other places
13a7d5d
@dwest77a
Refactored remote/complete workflows for CFA remote conversion
2aaf453
@dwest77a
Fixed issue with workdir from kwarg
d18cd22
@dwest77a
Added override to global attrs for virtualizarr that was previously m…
f715011 
…issing
@dwest77a
Added aggregation-date property for CFA and kerchunk
a7fd5a5
@dwest77a
Updated version, dependencies
c3891b2
@dwest77a
Minor version update, more than just bug fixes
e89dc6f
@dwest77a
Test fixes, changes across all scripts for bug fixes
998612f
@dwest77a
Updated pyfive dependency
73d07a8
@dwest77a
Fixed issue with repeating code subsets
1b75d71
@dwest77a
Fixed issue with project code allocations in parallel
d923676
@dwest77a
Fixed bug with 0-size groups
9753217
@dwest77a
Added bypass for CFA failed scan
7aaa74e
@dwest77a
Various changes for shepard status board, updating etc.
340d16d
@dwest77a dwest77a changed the title Version 1.6: Support for Remote CFA files Version 1.5: Support for Remote CFA files Oct 2, 2026
@dwest77a
dwest77a merged commit cc02d2d into main Oct 2, 2026
3 checks passed
@dwest77a
dwest77a deleted the v1.5.1 branch October 2, 2026 12:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@dwest77a