Differences

This shows you the differences between two versions of the page.

--- other:python:jyp_steps [2019/10/04 14:42] – [Graphics related resources] More resources jypeter
+++ other:python:jyp_steps [2020/02/04 09:33] – [Cartopy + Iris] jypeter
@@ Line 18: / Line 18: @@
     * the tutorial is very detailed about string handling, because strings offer an easy way to practice working with indices (indexing and slicing), before indexing numpy arrays. And our usual pre/post-processing scripts often need to do a lot of string handling in order to generate the file/variable/experiment names
   * after reading this tutorial, you should practice with the following:
-    * [[https://files.lsce.ipsl.fr/public.php?service=files&t=9731fdad4521ac5fa6e84b392d3a2e44|Basic python training test (ipython notebook version)]]
+    * [[https://sharebox.lsce.ipsl.fr/index.php/s/S3EO8cLrhVDeQWA|Basic python training test (ipython notebook version)]]
     * {{:other:python:tp_intro_python_oct2013_no_solutions.pdf|Basic python training test (pdf version)}}
     * {{:other:python:tp_intro_python_oct2013_full.pdf|Basic python training test (pdf version, with answers)}}
@@ Line 228: / Line 228: @@
 Examples:
+  * [[other:python:maps_by_jyp|Examples provided by JYP]]
   * [[http://scitools.org.uk/cartopy/docs/latest/gallery.html|Gallery on the Cartopy web site]]
   * [[http://scitools.org.uk/iris/docs/latest/gallery.html|Gallery on the Iris web site]]
@@ Line 258: / Line 259: @@
 We list here some resources about non-NetCDF data formats that can be useful
+==== The shelve package ====
+The [[https://docs.python.org/3/library/shelve.html|built-in shelve package]], can be easily used for storing data (python objects like lists, dictionaries, numpy arrays that are not too big, ...) on disk and retrieving them later
+Use case:
+  - Use a script do to the heavy data pre-processing and store the (intermediate) results in a file using ''shelve'', or update the results
+  - Use another script for plotting the results stored with ''shelve''. This way you don't have to wait for the pre-processing step to finish each time you want to improve your plot(s)
+Warning:
+  * read the [[https://docs.python.org/3/library/shelve.html|documentation]] and the example carefully (it's quite small)
+    * if you get the impression that the data is not saved correctly, re-read the parts about updating correctly the content of the shelve file
+    * you should be able to store most python objects in a shelve file, but it is safer to make tests
+  * do not forget to close the output file
+  * if you are dealing with big arrays and want to avoid performance issues, you should use netCDF files for storing the intermediate results
 ==== json files ====