Skip to content

Commit df80e06

Browse files
committed
Tutorial updates, before testing the salt/salinity budget
1 parent c88c7a9 commit df80e06

3 files changed

Lines changed: 4804 additions & 175 deletions

File tree

Tutorials_as_Jupyter_Notebooks/ECCO_v4_Heat_budget_closure.ipynb

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2925,7 +2925,7 @@
29252925
" {'time':str(curr_time_isel[0])+':'+str(curr_time_isel[-1]+1)},\\\n",
29262926
" function(ds,ecco_vars_int_pickled,vol,time_isel=curr_time_isel,k_isel=k_isel))\n",
29272927
" ds_to_write[varname].to_dataset().to_zarr(save_location,mode=\"a\")\n",
2928-
" ds_to_write.close() \n",
2928+
" ds_to_write.close()\n",
29292929
"\n",
29302930
"\n",
29312931
"# the zarr archive will occupy ~15 GB, so require 20 GB free storage as a buffer\n",

Tutorials_as_Jupyter_Notebooks/ECCO_v4_Memory_management.ipynb

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1035,7 +1035,7 @@
10351035
"id": "1a84fa35-3bcd-4a59-a2a7-b07988f0aadd",
10361036
"metadata": {},
10371037
"source": [
1038-
"The same procedure did not help to free up the memory from `copy_array_3`, which was probably too interconnected with Dask since a Dask array was created from it. Usually this is not a problem for memory management since we are more likely to load Dask arrays from files, rather than creating them from NumPy arrays already in memory.\n",
1038+
"So we were able to recover the memory from `copy_array_3` as well.\n",
10391039
"\n",
10401040
"Let's take a look at the memory log and compare the case that did not involve Dask with the one that did."
10411041
]
@@ -1089,7 +1089,7 @@
10891089
"id": "6cc6978b-c31b-458b-a4e4-402588498f67",
10901090
"metadata": {},
10911091
"source": [
1092-
"With NumPy only (no Dask) we were able to recover essentially all of the memory we used during the computation. With Dask we achieved a net loss of ~10 MB at entry #4 on the right (just over the size of one of the arrays) before losing more memory when saving `copy_array_3` to disk.\n",
1092+
"So in both cases we were able to complete the calculation and recover the memory used (i.e., make it available again). The procedure was just a little more complicated when using Dask arrays.\n",
10931093
"\n",
10941094
"It is worth remembering that Dask can also significantly *help* you limit the memory usage in your workspace (as we saw earlier with a [reduction operation](#Dask-arrays-and-delayed-computations))...even though you relinquish some control over memory management when using it."
10951095
]
@@ -2240,7 +2240,7 @@
22402240
"id": "af74327d-83e3-4e3b-9ffe-6544c8134de7",
22412241
"metadata": {},
22422242
"source": [
2243-
"In the code above we did explicitly something that Dask does behind the scenes...looped through chunks so that our data loads are done in smaller pieces. But the change in memory is about the same as before, because by the end of the loop all of the top 25 depth levels of `THETA` have been cached. \n",
2243+
"In the code above we did explicitly something that Dask does behind the scenes...looped through chunks so that our data loads are done in smaller pieces. But the memory used is still a lot more than we should need for 1 depth level, because chunks were cached that include the top **25** depth levels of `THETA`.\n",
22442244
"\n",
22452245
"### A workaround to save workspace memory\n",
22462246
"\n",

0 commit comments

Comments
 (0)