This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
other:python:jyp_steps [2018/07/10 09:02] jypeter [json files] Moved json to a new "Data file formats" section, and dded LiPD and BagIt |
other:python:jyp_steps [2019/05/23 14:41] jypeter [Useful reference pages] Added help for the 'legend' |
||
---|---|---|---|
Line 58: | Line 58: | ||
- [[https://docs.scipy.org/doc/numpy-dev/user/numpy-for-matlab-users.html|Numpy for Matlab users]] | - [[https://docs.scipy.org/doc/numpy-dev/user/numpy-for-matlab-users.html|Numpy for Matlab users]] | ||
- [[http://mathesaurus.sourceforge.net/matlab-numpy.html|NumPy for MATLAB users]] (nice, but does not seem to be maintained any more) | - [[http://mathesaurus.sourceforge.net/matlab-numpy.html|NumPy for MATLAB users]] (nice, but does not seem to be maintained any more) | ||
- | - read the [[https://docs.scipy.org/doc/numpy-dev/user/quickstart.html|Quickstart tutorial]] | + | - read the really nice [[https://docs.scipy.org/doc/numpy/user/quickstart.html|numpy Quickstart tutorial]] |
- have a quick look at the full documentation to know where things are | - have a quick look at the full documentation to know where things are | ||
- Numpy User Guide | - Numpy User Guide | ||
Line 120: | Line 120: | ||
[20, 21, -1, -1, 24, 25, 26, 27, 28, 29]]) | [20, 21, -1, -1, 24, 25, 26, 27, 28, 29]]) | ||
</code></note> | </code></note> | ||
+ | |||
+ | ==== Extra numpy information ==== | ||
+ | |||
+ | * More information about array indexing: | ||
+ | * Examples: | ||
+ | * {{ :other:python:indirect_indexing_2.py.txt |}}: Take a vertical slice in a 3D zyx array, along a varying y 'path' | ||
+ | * [[https://docs.scipy.org/doc/numpy/user/basics.indexing.html|Indexing]] (//index arrays//, //boolean index arrays//, //np.newaxis//, //Ellipsis//, //variable numbers of indices//, ...) | ||
+ | * [[https://docs.scipy.org/doc/numpy/user/quickstart.html#fancy-indexing-and-index-tricks|Fancy indexing]] and [[https://docs.scipy.org/doc/numpy/user/quickstart.html#the-ix-function|the ix_() function]] | ||
+ | * [[https://docs.scipy.org/doc/numpy/reference/arrays.indexing.html|Indexing (in the numpy reference manual)]] | ||
+ | * [[https://docs.scipy.org/doc/numpy/reference/routines.indexing.html#routines-indexing|Indexing routines]] | ||
+ | * More information about arrays: | ||
+ | * [[https://docs.scipy.org/doc/numpy/reference/routines.array-creation.html#routines-array-creation|Array creation routines]] | ||
+ | * [[https://docs.scipy.org/doc/numpy/reference/routines.array-manipulation.html|Array manipulation routines]] | ||
+ | * [[https://docs.scipy.org/doc/numpy/reference/maskedarray.html|Masked arrays]] | ||
+ | * [[https://docs.scipy.org/doc/numpy/reference/routines.ma.html|Masked array operations]] | ||
+ | * [[https://docs.scipy.org/doc/numpy/user/misc.html#ieee-754-floating-point-special-values|Dealing with special numerical values]] (//Nan//, //inf//) | ||
+ | * If you know that your data has missing values, it is cleaner and safer to handle them with [[https://docs.scipy.org/doc/numpy/reference/maskedarray.html|masked arrays]]! | ||
+ | * [[https://docs.scipy.org/doc/numpy/user/misc.html#how-numpy-handles-numerical-exceptions|Handling numerical exceptions]] | ||
+ | * [[https://docs.scipy.org/doc/numpy/reference/routines.err.html|Floating point error handling]] | ||
===== cdms2 and netCDF4 ===== | ===== cdms2 and netCDF4 ===== | ||
Line 166: | Line 185: | ||
- a Matplotlib //Figure// is a graphical window in which you make your plots... | - a Matplotlib //Figure// is a graphical window in which you make your plots... | ||
- a Matplotlib //Axis// is a plot inside a Figure... [[http://matplotlib.org/faq/usage_faq.html#parts-of-a-figure|More details]] | - a Matplotlib //Axis// is a plot inside a Figure... [[http://matplotlib.org/faq/usage_faq.html#parts-of-a-figure|More details]] | ||
+ | - some resources for having multiple plots on the same figure | ||
+ | * [[https://matplotlib.org/gallery/recipes/create_subplots.html#sphx-glr-gallery-recipes-create-subplots-py|Easily creating subplots]] | ||
+ | * [[https://matplotlib.org/gallery/index.html#subplots-axes-and-figures|Subplots, axes and figures]] gallery | ||
+ | * [[https://matplotlib.org/tutorials/intermediate/gridspec.html#sphx-glr-tutorials-intermediate-gridspec-py|Customizing Figure Layouts Using GridSpec and Other Functions]], [[https://matplotlib.org/tutorials/intermediate/constrainedlayout_guide.html|constrained layout]] and [[https://matplotlib.org/tutorials/intermediate/tight_layout_guide.html|tight layout]] | ||
+ | * [[http://matplotlib.org/faq/usage_faq.html#parts-of-a-figure|parts of a figure]] | ||
- some examples are more //pythonic// (ie object oriented) than others, some example mix different styles of coding, all this can be confusing. Try to [[http://matplotlib.org/faq/usage_faq.html#coding-styles|use an object oriented way of doing things]]! | - some examples are more //pythonic// (ie object oriented) than others, some example mix different styles of coding, all this can be confusing. Try to [[http://matplotlib.org/faq/usage_faq.html#coding-styles|use an object oriented way of doing things]]! | ||
- | - sometimes the results of the python/matplolib commands are displayed directly, sometimes not. It depends if you are in [[http://matplotlib.org/faq/usage_faq.html#what-is-interactive-mode|interactive or non-interactive]] mode | + | - it may be hard to (remember how to) work with colors. Some examples from the [[http://matplotlib.org/gallery.html|Gallery]] can help you! |
+ | * [[https://matplotlib.org/examples/pylab_examples/leftventricle_bulleye.html|leftventricle_bulleye.py]]: associating different types of colormaps to a plot and colorbar | ||
+ | * [[https://matplotlib.org/examples/api/colorbar_only.html|colorbar_only.py]]: the different types of colorbars (or plotting only a colorbar) | ||
+ | * [[https://matplotlib.org/examples/color/colormaps_reference.html|colormaps_reference.py]]: pre-defined colormaps | ||
+ | * [[https://matplotlib.org/examples/color/named_colors.html|named_colors.py]]: named colors | ||
+ | * More details about the colors below, in the [[#graphics_related_resources|Resources section]] | ||
+ | - sometimes the results of the python/matplolib commands are displayed immediately, sometimes not. It depends if you are in [[http://matplotlib.org/faq/usage_faq.html#what-is-interactive-mode|interactive or non-interactive]] mode | ||
+ | - if your matplotlib is executed in a batch script, it will generate an error when trying to create (''show()'') a plot, because matplotlib expects to be able to display the figure on a screen by default. | ||
+ | * Check how you can [[https://matplotlib.org/faq/howto_faq.html?highlight=web#generate-images-without-having-a-window-appear|generate images offline]] | ||
- the documentation may mention [[http://matplotlib.org/faq/usage_faq.html#what-is-a-backend|backends]]. What?? Basically, you use python commands to create a plot, and the backend is the //thing// that will render your plot on the screen or in a file (png, pdf, etc...) | - the documentation may mention [[http://matplotlib.org/faq/usage_faq.html#what-is-a-backend|backends]]. What?? Basically, you use python commands to create a plot, and the backend is the //thing// that will render your plot on the screen or in a file (png, pdf, etc...) | ||
- if you don't see a part of what you have plotted, maybe it's hidden behind other elements! Use the [[https://matplotlib.org/examples/pylab_examples/zorder_demo.html|zorder parameter]] to explicitly specify the plotting order/layers | - if you don't see a part of what you have plotted, maybe it's hidden behind other elements! Use the [[https://matplotlib.org/examples/pylab_examples/zorder_demo.html|zorder parameter]] to explicitly specify the plotting order/layers | ||
- Read the [[http://www.labri.fr/perso/nrougier/teaching/matplotlib/|Matplotlib tutorial by Nicolas Rougier]] | - Read the [[http://www.labri.fr/perso/nrougier/teaching/matplotlib/|Matplotlib tutorial by Nicolas Rougier]] | ||
- Download the [[http://matplotlib.org/contents.html|pdf version of the manual]]. **Do not print** the 2800+ pages of the manual! Read the beginner's guide (Chapter //FIVE// of //Part II//) and have a super quick look at the table of contents of the whole document. | - Download the [[http://matplotlib.org/contents.html|pdf version of the manual]]. **Do not print** the 2800+ pages of the manual! Read the beginner's guide (Chapter //FIVE// of //Part II//) and have a super quick look at the table of contents of the whole document. | ||
+ | |||
+ | ==== Useful reference pages ==== | ||
+ | |||
+ | * [[https://matplotlib.org/api/_as_gen/matplotlib.pyplot.plot.html|plot(...)]]: Plot y versus x as lines and/or markers | ||
+ | * [[https://matplotlib.org/api/_as_gen/matplotlib.pyplot.scatter.html|scatter(...)]]: A scatter plot of y vs x with varying marker size and/or color | ||
+ | * The ''plot'' function will be faster for scatterplots where markers don't vary in size or color | ||
+ | * [[https://matplotlib.org/api/_as_gen/matplotlib.lines.Line2D.html|line]] parameters | ||
+ | * ''linestyle'': ''solid'', ''None'', [[https://matplotlib.org/api/_as_gen/matplotlib.lines.Line2D.html#matplotlib.lines.Line2D.set_linestyle|other]] ([[https://matplotlib.org/examples/lines_bars_and_markers/line_styles_reference.html|default styles example]], [[https://matplotlib.org/examples/lines_bars_and_markers/linestyles.html|custom styles example]]) | ||
+ | * [[https://matplotlib.org/api/markers_api.html|marker types]] | ||
+ | * ''fillstyle'': ''full'', ''None'', [[https://matplotlib.org/gallery/lines_bars_and_markers/marker_fillstyle_reference.html|other]] | ||
+ | * Other attributes: ''markersize'', ''markerfacecolor'' (and ''markerfacecoloralt'' for dual color markers), ''markeredgecolor'', ''markeredgewidth'' | ||
+ | * [[https://matplotlib.org/api/colors_api.html|colors]] | ||
+ | * [[https://matplotlib.org/gallery/color/color_demo.html|color demo]] | ||
+ | * [[https://matplotlib.org/examples/color/named_colors.html|named colors]] | ||
+ | * [[https://matplotlib.org/api/pyplot_api.html#matplotlib.pyplot.legend|legend(...)]] ([[https://matplotlib.org/examples/pylab_examples/legend_demo3.html|legend demo]]) | ||
+ | * The legend will //show// the lines (or other objects) that were associated with a //label// with the ''label='' keyword when creating/updating a plot | ||
+ | * If there are some elements of a plot that you do not want to associate with a legend (e.g. there are several lines with the same color and markers, but you want to plot the legend only once), do not specify a ''label='' keyword for these elements, or add a ''_'' at the front of the label strings | ||
+ | * The legend is positioned somewhere (that can be specified) **inside** the plot. In order to place a legend **outside** the plot, use the ''bbox_to_anchor'' parameter | ||
+ | * the parameters of ''bbox_to_anchor'' are in normalized coordinates of the current (sub)plot: | ||
+ | * ''(0, 0)'' is the lower left corner of the plot, and ''(1, 1)'' the upper right corner | ||
+ | * ''legend(... bbox_to_anchor=(1.05, 1.), loc='upper left', ...)'' will put the upper left corner of the legend slightly right (''(1.05, 1.)'') of the upper right corner (''(1, 1)'') of the plot | ||
+ | * if the legend is outside of the plot, you have to **explicitly provide enough space for the legend on the page** | ||
+ | * e.g. with [[https://matplotlib.org/api/_as_gen/matplotlib.pyplot.subplots_adjust.html|subplots_adjust]], ''plt.subplots_adjust(right=0.75)'' will make all the plots use 75% on the left of the page, and leave 25% on the right for the legend | ||
+ | |||
+ | ==== Misc Matplotlib tricks ==== | ||
+ | |||
+ | * Specifying the background color of a plot (e.g. when plotting a masked variable and you don't want the masked areas to be white) | ||
+ | * ''# make the background dark gray (call this before the contourf)''\\ ''plt.gca().patch.set_color('.25')''\\ ''plt.contourf(d)''\\ ''plt.show()'' | ||
+ | * [[https://stackoverflow.com/questions/9797520/masking-part-of-a-contourf-plot-in-matplotlib|trick source]] | ||
===== Graphics related resources ===== | ===== Graphics related resources ===== | ||
* [[http://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1003833|Ten Simple Rules for Better Figures]] | * [[http://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1003833|Ten Simple Rules for Better Figures]] | ||
+ | * [[https://www.machinelearningplus.com/plots/top-50-matplotlib-visualizations-the-master-plots-python/|Top 50 matplotlib Visualizations]] | ||
* [[http://seaborn.pydata.org/|Seaborn]] is a library for making attractive and informative statistical graphics in Python, built on top of matplotlib | * [[http://seaborn.pydata.org/|Seaborn]] is a library for making attractive and informative statistical graphics in Python, built on top of matplotlib | ||
* See also: [[https://www.datacamp.com/community/tutorials/seaborn-python-tutorial| | * See also: [[https://www.datacamp.com/community/tutorials/seaborn-python-tutorial| | ||
Line 217: | Line 279: | ||
Help on //stack overflow//: [[https://stackoverflow.com/questions/tagged/cartopy|cartopy help]] | Help on //stack overflow//: [[https://stackoverflow.com/questions/tagged/cartopy|cartopy help]] | ||
+ | |||
+ | ===== Maps and projections resources ===== | ||
+ | |||
+ | ==== About projections ==== | ||
+ | |||
+ | * [[https://egsc.usgs.gov/isb//pubs/MapProjections/projections.html|Map projections from USGS poster]] | ||
+ | * [[https://pubs.usgs.gov/pp/1395/report.pdf|Map projections - A working manual (USGS)]] | ||
+ | |||
+ | ==== Libraries ==== | ||
+ | |||
+ | * Projections in vcs | ||
+ | * [[http://matplotlib.org/basemap/users/mapsetup.html|Projections in basemap]] | ||
+ | * [[https://scitools.org.uk/cartopy/docs/latest/crs/projections.html|Projections in cartopy]] | ||
+ | |||
===== 3D resources ===== | ===== 3D resources ===== | ||
Line 273: | Line 349: | ||
This is **a really nice and useful document** that is regularly updated and used for the [[https://www.euroscipy.org/|EuroScipy]] tutorials. You will learn more things about python, numpy and matplotlib, debugging and optimizing scripts, and also learn about using python for statistics, image processing, machine learning, washing dishes (this is just to check if you have read this page), etc... | This is **a really nice and useful document** that is regularly updated and used for the [[https://www.euroscipy.org/|EuroScipy]] tutorials. You will learn more things about python, numpy and matplotlib, debugging and optimizing scripts, and also learn about using python for statistics, image processing, machine learning, washing dishes (this is just to check if you have read this page), etc... | ||
- | ===== Quick Reference ===== | + | ===== Quick Reference and cheat sheets ===== |
* The nice and convenient Python 2.7 Quick Reference: [[http://rgruet.free.fr/PQR27/PQR2.7_printing_a4.pdf|pdf]] - [[http://rgruet.free.fr/PQR27/PQR2.7.html|html]] | * The nice and convenient Python 2.7 Quick Reference: [[http://rgruet.free.fr/PQR27/PQR2.7_printing_a4.pdf|pdf]] - [[http://rgruet.free.fr/PQR27/PQR2.7.html|html]] | ||
+ | * A possibly more [[http://iysik.com/PQR2.7/PQR2.7.html|up-date-version]] | ||
+ | |||
+ | * Python 3 [[https://perso.limsi.fr/pointal/python:abrege|Quick reference]] and [[https://perso.limsi.fr/pointal/python:memento|Cheat sheet]] | ||
+ | |||
+ | * [[https://www.cheatography.com/weidadeyue/cheat-sheets/jupyter-notebook/pdf_bw/|Jupyter Notebook Keyboard Shortcuts]] | ||
+ | |||
+ | ===== Misc tutorials ===== | ||
+ | * [[https://pyformat.info/|PyFormat]]: //With this site we try to show you the most common use-cases covered by the old and new style string formatting API with practical examples// | ||
===== Some good coding tips ===== | ===== Some good coding tips ===== | ||
Line 313: | Line 397: | ||
* [[https://www.datacamp.com/community/tutorials/data-science-python-ide|Top 5 Python IDEs For Data Science]] | * [[https://www.datacamp.com/community/tutorials/data-science-python-ide|Top 5 Python IDEs For Data Science]] | ||
* [[http://noeticforce.com/best-python-ide-for-programmers-windows-and-mac|Python IDE: The10 Best IDEs for Python Programmers]] | * [[http://noeticforce.com/best-python-ide-for-programmers-windows-and-mac|Python IDE: The10 Best IDEs for Python Programmers]] | ||
+ | * [[https://www.techbeamers.com/best-python-ide-python-programming/|Get the Best Python IDE]] | ||
* [[https://wiki.python.org/moin/IntegratedDevelopmentEnvironments]] | * [[https://wiki.python.org/moin/IntegratedDevelopmentEnvironments]] | ||