User Tools

Site Tools


other:python:jyp_steps

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
other:python:jyp_steps [2019/09/05 16:22]
jypeter Moved the bulk of the matplotlib section to its own page when it became too long
other:python:jyp_steps [2020/02/04 09:36] (current)
jypeter [Matplotlib]
Line 18: Line 18:
     * the tutorial is very detailed about string handling, because strings offer an easy way to practice working with indices (indexing and slicing), before indexing numpy arrays. And our usual pre/​post-processing scripts often need to do a lot of string handling in order to generate the file/​variable/​experiment names     * the tutorial is very detailed about string handling, because strings offer an easy way to practice working with indices (indexing and slicing), before indexing numpy arrays. And our usual pre/​post-processing scripts often need to do a lot of string handling in order to generate the file/​variable/​experiment names
   * after reading this tutorial, you should practice with the following:   * after reading this tutorial, you should practice with the following:
-    * [[https://files.lsce.ipsl.fr/​public.php?​service=files&​t=9731fdad4521ac5fa6e84b392d3a2e44|Basic python training test (ipython notebook version)]]+    * [[https://sharebox.lsce.ipsl.fr/​index.php/​s/​S3EO8cLrhVDeQWA|Basic python training test (ipython notebook version)]]
     * {{:​other:​python:​tp_intro_python_oct2013_no_solutions.pdf|Basic python training test (pdf version)}}     * {{:​other:​python:​tp_intro_python_oct2013_no_solutions.pdf|Basic python training test (pdf version)}}
     * {{:​other:​python:​tp_intro_python_oct2013_full.pdf|Basic python training test (pdf version, with answers)}}     * {{:​other:​python:​tp_intro_python_oct2013_full.pdf|Basic python training test (pdf version, with answers)}}
Line 175: Line 175:
  
 <note important>​ <note important>​
-The full content of this //​matplotlib//​ section has been moved to\\ [[other:​python:​matplotlib_by_jyp|Working with matplotlib (JYP version)]]\\ after becoming too big to manage here</​note>​+The full content of this //​matplotlib//​ section has been moved to\\ [[other:​python:​matplotlib_by_jyp|Working with matplotlib (JYP version)]]\\ after becoming too big to manage here 
 + 
 +\\ Note: [[other:​python:​maps_by_jyp|Plotting maps with matplotlib+cartopy]] (examples provided by JYP) 
 +</​note>​
  
 Summary: there are lots of python libraries that you can use for plotting, but Matplotlib has become a //de facto// standard Summary: there are lots of python libraries that you can use for plotting, but Matplotlib has become a //de facto// standard
Line 190: Line 193:
     * See also: [[https://​www.datacamp.com/​community/​tutorials/​seaborn-python-tutorial|     * See also: [[https://​www.datacamp.com/​community/​tutorials/​seaborn-python-tutorial|
 Python Seaborn Tutorial For Beginners]] Python Seaborn Tutorial For Beginners]]
 +  * Communicating/​displaying/​plotting your data (possibly for people not of your field):
 +    * [[https://​uxknowledgebase.com/​introduction-to-designing-data-visualizations-part-1-31c056556133|Introduction to Designing Data Visualizations — Part 1]]
 +    * [[https://​uxknowledgebase.com/​tables-other-charts-data-visualization-part-2-cfc582e4712c|Tables & Other Charts — Data Visualization Part 2]]
 +    * [[https://​uxknowledgebase.com/​tables-other-charts-data-visualization-part-3-5bfab15ce525|Tables & Other Charts — Data Visualization Part 3]]
   * Working with colors   * Working with colors
     * [[https://​matplotlib.org/​users/​colormaps.html|Choosing colormaps]]     * [[https://​matplotlib.org/​users/​colormaps.html|Choosing colormaps]]
-    * [[https://​matplotlib.org/​cmocean/​|Beautiful colormaps for oceanography: ​cmocean]]+    * [[https://​matplotlib.org/​cmocean/​|cmocean: ​Beautiful colormaps for oceanography]] 
 +    * [[https://​jiffyclub.github.io/​palettable/​|Palettable:​ Color palettes for Python]]
     * [[http://​colorbrewer2.org|ColorBrewer 2.0]] is a tool that can help you understand, and experiment with //​sequential//,​ //​diverging//​ and //​qualitative//​ colormaps     * [[http://​colorbrewer2.org|ColorBrewer 2.0]] is a tool that can help you understand, and experiment with //​sequential//,​ //​diverging//​ and //​qualitative//​ colormaps
  
Line 223: Line 231:
  
 Examples: Examples:
 +  * [[other:​python:​maps_by_jyp|Examples provided by JYP]]
   * [[http://​scitools.org.uk/​cartopy/​docs/​latest/​gallery.html|Gallery on the Cartopy web site]]   * [[http://​scitools.org.uk/​cartopy/​docs/​latest/​gallery.html|Gallery on the Cartopy web site]]
   * [[http://​scitools.org.uk/​iris/​docs/​latest/​gallery.html|Gallery on the Iris web site]]   * [[http://​scitools.org.uk/​iris/​docs/​latest/​gallery.html|Gallery on the Iris web site]]
Line 253: Line 262:
 We list here some resources about non-NetCDF data formats that can be useful We list here some resources about non-NetCDF data formats that can be useful
  
 +==== The shelve package ====
 +
 +The [[https://​docs.python.org/​3/​library/​shelve.html|built-in shelve package]], can be easily used for storing data (python objects like lists, dictionaries,​ numpy arrays that are not too big, ...) on disk and retrieving them later
 +
 +Use case:
 +  - Use a script do to the heavy data pre-processing and store the (intermediate) results in a file using ''​shelve'',​ or update the results
 +  - Use another script for plotting the results stored with ''​shelve''​. This way you don't have to wait for the pre-processing step to finish each time you want to improve your plot(s)
 +
 +Warning:
 +  * read the [[https://​docs.python.org/​3/​library/​shelve.html|documentation]] and the example carefully (it's quite small)
 +    * if you get the impression that the data is not saved correctly, re-read the parts about updating correctly the content of the shelve file
 +    * you should be able to store most python objects in a shelve file, but it is safer to make tests
 +  * do not forget to close the output file
 +  * if you are dealing with big arrays and want to avoid performance issues, you should use netCDF files for storing the intermediate results
 ==== json files ==== ==== json files ====
  
other/python/jyp_steps.1567693339.txt.gz · Last modified: 2019/09/05 16:22 by jypeter