User Tools

Site Tools


other:python:jyp_steps

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
Next revision Both sides next revision
other:python:jyp_steps [2018/05/31 14:33]
jypeter Added json
other:python:jyp_steps [2018/10/18 15:57]
jypeter [Misc numpy tricks]
Line 58: Line 58:
     - [[https://​docs.scipy.org/​doc/​numpy-dev/​user/​numpy-for-matlab-users.html|Numpy for Matlab users]]     - [[https://​docs.scipy.org/​doc/​numpy-dev/​user/​numpy-for-matlab-users.html|Numpy for Matlab users]]
     - [[http://​mathesaurus.sourceforge.net/​matlab-numpy.html|NumPy for MATLAB users]] (nice, but does not seem to be maintained any more)     - [[http://​mathesaurus.sourceforge.net/​matlab-numpy.html|NumPy for MATLAB users]] (nice, but does not seem to be maintained any more)
-  - read the [[https://​docs.scipy.org/​doc/​numpy-dev/​user/​quickstart.html|Quickstart tutorial]]+  - read the really nice [[https://​docs.scipy.org/​doc/​numpy/​user/​quickstart.html|numpy Quickstart tutorial]]
   - have a quick look at the full documentation to know where things are   - have a quick look at the full documentation to know where things are
     - Numpy User Guide     - Numpy User Guide
Line 120: Line 120:
        [20, 21, -1, -1, 24, 25, 26, 27, 28, 29]])        [20, 21, -1, -1, 24, 25, 26, 27, 28, 29]])
 </​code></​note>​ </​code></​note>​
 +
 +==== Extra numpy information ====
 +
 +  * More information about array indexing:
 +    * Examples:
 +      * {{ :​other:​python:​indirect_indexing_2.py.txt |}}: Take a vertical slice in a 3D zyx array, along a varying y '​path'​
 +    * [[https://​docs.scipy.org/​doc/​numpy/​user/​basics.indexing.html|Indexing]] (//index arrays//, //boolean index arrays//, //​np.newaxis//,​ //​Ellipsis//,​ //variable numbers of indices//, ...)
 +    * [[https://​docs.scipy.org/​doc/​numpy/​user/​quickstart.html#​fancy-indexing-and-index-tricks|Fancy indexing]] and [[https://​docs.scipy.org/​doc/​numpy/​user/​quickstart.html#​the-ix-function|the ix_() function]]
 +    * [[https://​docs.scipy.org/​doc/​numpy/​reference/​arrays.indexing.html|Indexing (in the numpy reference manual)]]
 +    * [[https://​docs.scipy.org/​doc/​numpy/​reference/​routines.indexing.html#​routines-indexing|Indexing routines]] ​
 +  * More information about arrays:
 +    * [[https://​docs.scipy.org/​doc/​numpy/​reference/​routines.array-creation.html#​routines-array-creation|Array creation routines]]
 +    * [[https://​docs.scipy.org/​doc/​numpy/​reference/​routines.array-manipulation.html|Array manipulation routines]]
 +    * [[https://​docs.scipy.org/​doc/​numpy/​reference/​maskedarray.html|Masked arrays]]
 +      * [[https://​docs.scipy.org/​doc/​numpy/​reference/​routines.ma.html|Masked array operations]]
 +  * [[https://​docs.scipy.org/​doc/​numpy/​user/​misc.html#​ieee-754-floating-point-special-values|Dealing with special numerical values]] (//Nan//, //inf//)
 +    * If you know that your data has missing values, it is cleaner and safer to handle them with [[https://​docs.scipy.org/​doc/​numpy/​reference/​maskedarray.html|masked arrays]]!
 +    * [[https://​docs.scipy.org/​doc/​numpy/​user/​misc.html#​how-numpy-handles-numerical-exceptions|Handling numerical exceptions]]
 +    * [[https://​docs.scipy.org/​doc/​numpy/​reference/​routines.err.html|Floating point error handling]]
  
 ===== cdms2 and netCDF4 ===== ===== cdms2 and netCDF4 =====
Line 167: Line 186:
     - a Matplotlib //Axis// is a plot inside a Figure... [[http://​matplotlib.org/​faq/​usage_faq.html#​parts-of-a-figure|More details]]     - a Matplotlib //Axis// is a plot inside a Figure... [[http://​matplotlib.org/​faq/​usage_faq.html#​parts-of-a-figure|More details]]
     - some examples are more //​pythonic//​ (ie object oriented) than others, some example mix different styles of coding, all this can be confusing. Try to [[http://​matplotlib.org/​faq/​usage_faq.html#​coding-styles|use an object oriented way of doing things]]!     - some examples are more //​pythonic//​ (ie object oriented) than others, some example mix different styles of coding, all this can be confusing. Try to [[http://​matplotlib.org/​faq/​usage_faq.html#​coding-styles|use an object oriented way of doing things]]!
 +    - it may be hard to (remember how to) work with colors. Some examples from the [[http://​matplotlib.org/​gallery.html|Gallery]] can help you!
 +      * [[https://​matplotlib.org/​examples/​pylab_examples/​leftventricle_bulleye.html|leftventricle_bulleye.py]]:​ associating different types of colormaps to a plot and colorbar
 +      * [[https://​matplotlib.org/​examples/​api/​colorbar_only.html|colorbar_only.py]]:​ the different types of colorbars (or plotting only a colorbar)
 +      * [[https://​matplotlib.org/​examples/​color/​colormaps_reference.html|colormaps_reference.py]]:​ pre-defined colormaps
 +      * [[https://​matplotlib.org/​examples/​color/​named_colors.html|named_colors.py]]:​ named colors
 +      * More details about the colors below, in the [[#​graphics_related_resources|Resources section]]
     - sometimes the results of the python/​matplolib commands are displayed directly, sometimes not. It depends if you are in [[http://​matplotlib.org/​faq/​usage_faq.html#​what-is-interactive-mode|interactive or non-interactive]] mode     - sometimes the results of the python/​matplolib commands are displayed directly, sometimes not. It depends if you are in [[http://​matplotlib.org/​faq/​usage_faq.html#​what-is-interactive-mode|interactive or non-interactive]] mode
     - the documentation may mention [[http://​matplotlib.org/​faq/​usage_faq.html#​what-is-a-backend|backends]]. What?? Basically, you use python commands to create a plot, and the backend is the //thing// that will render your plot on the screen or in a file (png, pdf, etc...)     - the documentation may mention [[http://​matplotlib.org/​faq/​usage_faq.html#​what-is-a-backend|backends]]. What?? Basically, you use python commands to create a plot, and the backend is the //thing// that will render your plot on the screen or in a file (png, pdf, etc...)
Line 172: Line 197:
   - Read the [[http://​www.labri.fr/​perso/​nrougier/​teaching/​matplotlib/​|Matplotlib tutorial by Nicolas Rougier]]   - Read the [[http://​www.labri.fr/​perso/​nrougier/​teaching/​matplotlib/​|Matplotlib tutorial by Nicolas Rougier]]
   - Download the [[http://​matplotlib.org/​contents.html|pdf version of the manual]]. **Do not print** the 2800+ pages of the manual! Read the beginner'​s guide (Chapter //FIVE// of //Part II//) and have a super quick look at the table of contents of the whole document.   - Download the [[http://​matplotlib.org/​contents.html|pdf version of the manual]]. **Do not print** the 2800+ pages of the manual! Read the beginner'​s guide (Chapter //FIVE// of //Part II//) and have a super quick look at the table of contents of the whole document.
 +
 +==== Misc Matplotlib tricks ====
 +
 +  * Specifying the background color of a plot (e.g. when plotting a masked variable and you don't want the masked areas to be white)
 +    * ''#​ make the background dark gray (call this before the contourf)''​\\ ''​plt.gca().patch.set_color('​.25'​)''​\\ ''​plt.contourf(d)''​\\ ''​plt.show()''​
 +    * [[https://​stackoverflow.com/​questions/​9797520/​masking-part-of-a-contourf-plot-in-matplotlib|trick source]]
  
 ===== Graphics related resources ===== ===== Graphics related resources =====
Line 217: Line 248:
  
 Help on //stack overflow//: [[https://​stackoverflow.com/​questions/​tagged/​cartopy|cartopy help]] Help on //stack overflow//: [[https://​stackoverflow.com/​questions/​tagged/​cartopy|cartopy help]]
 +
 +===== Maps and projections resources =====
 +
 +==== About projections ====
 +
 +  * [[https://​egsc.usgs.gov/​isb//​pubs/​MapProjections/​projections.html|Map projections from USGS poster]]
 +  * [[https://​pubs.usgs.gov/​pp/​1395/​report.pdf|Map projections - A working manual (USGS)]]
 +
 +==== Libraries ====
 +
 +  * Projections in vcs
 +  * [[http://​matplotlib.org/​basemap/​users/​mapsetup.html|Projections in basemap]]
 +  * [[https://​scitools.org.uk/​cartopy/​docs/​latest/​crs/​projections.html|Projections in cartopy]]
 +
  
 ===== 3D resources ===== ===== 3D resources =====
Line 224: Line 269:
   * [[https://​stackoverflow.com/​questions/​26796997/​how-to-get-vertical-z-axis-in-3d-surface-plot-of-matplotlib|How to get vertical Z axis in 3D surface plot of Matplotlib?​]]   * [[https://​stackoverflow.com/​questions/​26796997/​how-to-get-vertical-z-axis-in-3d-surface-plot-of-matplotlib|How to get vertical Z axis in 3D surface plot of Matplotlib?​]]
  
-===== json files =====+=====  Data file formats =====  
 + 
 +We list here some resources about non-NetCDF data formats that can be useful 
 + 
 +==== json files ====
  
 More and more applications use //json files// as configuration files or as a mean to use text files to exchange data (through serialization/​deserialization ). More and more applications use //json files// as configuration files or as a mean to use text files to exchange data (through serialization/​deserialization ).
Line 234: Line 283:
   * example script: ''/​home/​users/​jypeter/​CDAT/​Progs/​Devel/​beaugendre/​nc2json.py''​   * example script: ''/​home/​users/​jypeter/​CDAT/​Progs/​Devel/​beaugendre/​nc2json.py''​
   * A compact (not easy to read...) //json// file can be pretty-printed with\\ ''​cat file.json | python -m json.tool | less''​   * A compact (not easy to read...) //json// file can be pretty-printed with\\ ''​cat file.json | python -m json.tool | less''​
 +
 +==== LiPD files ====
 +
 +Resources for //Linked PaleoData//:​
 +  * [[http://​linked.earth/​projects/​lipd/​|LiPD]]
 +  * [[https://​doi.org/​10.5194/​cp-12-1093-2016|Technical note: The Linked Paleo Data framework –
 +a common tongue for paleoclimatology]] @ GMD
 +  * [[https://​github.com/​nickmckay/​LiPD-utilities|LiPD-utilities]] @ github
 +
 +==== BagIt files ====
 +
 +//BagIt//, a set of hierarchical file layout conventions for storage and transfer of arbitrary digital content.
 +
 +  * [[https://​tools.ietf.org/​html/​draft-kunze-bagit-16|The BagIt File Packaging Format]]
 +  * [[https://​github.com/​LibraryOfCongress/​bagger|Bagger]] (BagIt GUI)
 +  * [[https://​github.com/​LibraryOfCongress/​bagit-python|bagit-python]]
 ===== Pandas ===== ===== Pandas =====
  
Line 256: Line 321:
  
   * The nice and convenient Python 2.7 Quick Reference: [[http://​rgruet.free.fr/​PQR27/​PQR2.7_printing_a4.pdf|pdf]] - [[http://​rgruet.free.fr/​PQR27/​PQR2.7.html|html]]   * The nice and convenient Python 2.7 Quick Reference: [[http://​rgruet.free.fr/​PQR27/​PQR2.7_printing_a4.pdf|pdf]] - [[http://​rgruet.free.fr/​PQR27/​PQR2.7.html|html]]
 +    * A possibly more [[http://​iysik.com/​PQR2.7/​PQR2.7.html|up-date-version]]
 +
 +  * Python 3 [[https://​perso.limsi.fr/​pointal/​python:​abrege|Quick reference]] and [[https://​perso.limsi.fr/​pointal/​python:​memento|Cheat sheet]]
 +
 +===== Misc tutorials =====
  
 +  * [[https://​pyformat.info/​|PyFormat]]:​ //With this site we try to show you the most common use-cases covered by the old and new style string formatting API with practical examples//
 ===== Some good coding tips ===== ===== Some good coding tips =====
  
other/python/jyp_steps.txt · Last modified: 2024/03/07 10:15 by jypeter