other:python:jyp_steps
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
|
other:python:jyp_steps [2018/08/01 09:47] jypeter [Quick Reference] Added a python 3 quick ref that looked nice |
other:python:jyp_steps [2019/11/18 11:04] jypeter Replaced files links with sharebox links |
||
|---|---|---|---|
| Line 18: | Line 18: | ||
| * the tutorial is very detailed about string handling, because strings offer an easy way to practice working with indices (indexing and slicing), before indexing numpy arrays. And our usual pre/post-processing scripts often need to do a lot of string handling in order to generate the file/variable/experiment names | * the tutorial is very detailed about string handling, because strings offer an easy way to practice working with indices (indexing and slicing), before indexing numpy arrays. And our usual pre/post-processing scripts often need to do a lot of string handling in order to generate the file/variable/experiment names | ||
| * after reading this tutorial, you should practice with the following: | * after reading this tutorial, you should practice with the following: | ||
| - | * [[https://files.lsce.ipsl.fr/public.php?service=files&t=9731fdad4521ac5fa6e84b392d3a2e44|Basic python training test (ipython notebook version)]] | + | * [[https://sharebox.lsce.ipsl.fr/index.php/s/S3EO8cLrhVDeQWA|Basic python training test (ipython notebook version)]] |
| * {{:other:python:tp_intro_python_oct2013_no_solutions.pdf|Basic python training test (pdf version)}} | * {{:other:python:tp_intro_python_oct2013_no_solutions.pdf|Basic python training test (pdf version)}} | ||
| * {{:other:python:tp_intro_python_oct2013_full.pdf|Basic python training test (pdf version, with answers)}} | * {{:other:python:tp_intro_python_oct2013_full.pdf|Basic python training test (pdf version, with answers)}} | ||
| Line 120: | Line 120: | ||
| [20, 21, -1, -1, 24, 25, 26, 27, 28, 29]]) | [20, 21, -1, -1, 24, 25, 26, 27, 28, 29]]) | ||
| </code></note> | </code></note> | ||
| + | |||
| + | ==== Extra numpy information ==== | ||
| + | |||
| + | * More information about array indexing: | ||
| + | * Examples: | ||
| + | * {{ :other:python:indirect_indexing_2.py.txt |}}: Take a vertical slice in a 3D zyx array, along a varying y 'path' | ||
| + | * [[https://docs.scipy.org/doc/numpy/user/basics.indexing.html|Indexing]] (//index arrays//, //boolean index arrays//, //np.newaxis//, //Ellipsis//, //variable numbers of indices//, ...) | ||
| + | * [[https://docs.scipy.org/doc/numpy/user/quickstart.html#fancy-indexing-and-index-tricks|Fancy indexing]] and [[https://docs.scipy.org/doc/numpy/user/quickstart.html#the-ix-function|the ix_() function]] | ||
| + | * [[https://docs.scipy.org/doc/numpy/reference/arrays.indexing.html|Indexing (in the numpy reference manual)]] | ||
| + | * [[https://docs.scipy.org/doc/numpy/reference/routines.indexing.html#routines-indexing|Indexing routines]] | ||
| + | * More information about arrays: | ||
| + | * [[https://docs.scipy.org/doc/numpy/reference/routines.array-creation.html#routines-array-creation|Array creation routines]] | ||
| + | * [[https://docs.scipy.org/doc/numpy/reference/routines.array-manipulation.html|Array manipulation routines]] | ||
| + | * [[https://docs.scipy.org/doc/numpy/reference/maskedarray.html|Masked arrays]] | ||
| + | * [[https://docs.scipy.org/doc/numpy/reference/routines.ma.html|Masked array operations]] | ||
| + | * [[https://docs.scipy.org/doc/numpy/user/misc.html#ieee-754-floating-point-special-values|Dealing with special numerical values]] (//Nan//, //inf//) | ||
| + | * If you know that your data has missing values, it is cleaner and safer to handle them with [[https://docs.scipy.org/doc/numpy/reference/maskedarray.html|masked arrays]]! | ||
| + | * [[https://docs.scipy.org/doc/numpy/user/misc.html#how-numpy-handles-numerical-exceptions|Handling numerical exceptions]] | ||
| + | * [[https://docs.scipy.org/doc/numpy/reference/routines.err.html|Floating point error handling]] | ||
| ===== cdms2 and netCDF4 ===== | ===== cdms2 and netCDF4 ===== | ||
| Line 154: | Line 173: | ||
| ===== Matplotlib ===== | ===== Matplotlib ===== | ||
| + | |||
| + | <note important> | ||
| + | The full content of this //matplotlib// section has been moved to\\ [[other:python:matplotlib_by_jyp|Working with matplotlib (JYP version)]]\\ after becoming too big to manage here</note> | ||
| Summary: there are lots of python libraries that you can use for plotting, but Matplotlib has become a //de facto// standard | Summary: there are lots of python libraries that you can use for plotting, but Matplotlib has become a //de facto// standard | ||
| Line 160: | Line 182: | ||
| Help on //stack overflow//: [[https://stackoverflow.com/questions/tagged/matplotlib|matplotlib help]] | Help on //stack overflow//: [[https://stackoverflow.com/questions/tagged/matplotlib|matplotlib help]] | ||
| - | |||
| - | The documentation is good, but not always easy to use. <wrap hi>A good way to start with matplotlib</wrap> is to: | ||
| - | - Look at the [[http://matplotlib.org/gallery.html|matplotlib gallery]] to get an idea of all you can do with matplotlib. Later, when you need to plot something, come back to the gallery to find some examples that are close to what you need and click on them to get the sources | ||
| - | - Use the free hints provided by JY! | ||
| - | - a Matplotlib //Figure// is a graphical window in which you make your plots... | ||
| - | - a Matplotlib //Axis// is a plot inside a Figure... [[http://matplotlib.org/faq/usage_faq.html#parts-of-a-figure|More details]] | ||
| - | - some examples are more //pythonic// (ie object oriented) than others, some example mix different styles of coding, all this can be confusing. Try to [[http://matplotlib.org/faq/usage_faq.html#coding-styles|use an object oriented way of doing things]]! | ||
| - | - sometimes the results of the python/matplolib commands are displayed directly, sometimes not. It depends if you are in [[http://matplotlib.org/faq/usage_faq.html#what-is-interactive-mode|interactive or non-interactive]] mode | ||
| - | - the documentation may mention [[http://matplotlib.org/faq/usage_faq.html#what-is-a-backend|backends]]. What?? Basically, you use python commands to create a plot, and the backend is the //thing// that will render your plot on the screen or in a file (png, pdf, etc...) | ||
| - | - if you don't see a part of what you have plotted, maybe it's hidden behind other elements! Use the [[https://matplotlib.org/examples/pylab_examples/zorder_demo.html|zorder parameter]] to explicitly specify the plotting order/layers | ||
| - | - Read the [[http://www.labri.fr/perso/nrougier/teaching/matplotlib/|Matplotlib tutorial by Nicolas Rougier]] | ||
| - | - Download the [[http://matplotlib.org/contents.html|pdf version of the manual]]. **Do not print** the 2800+ pages of the manual! Read the beginner's guide (Chapter //FIVE// of //Part II//) and have a super quick look at the table of contents of the whole document. | ||
| ===== Graphics related resources ===== | ===== Graphics related resources ===== | ||
| * [[http://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1003833|Ten Simple Rules for Better Figures]] | * [[http://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1003833|Ten Simple Rules for Better Figures]] | ||
| + | * [[https://www.machinelearningplus.com/plots/top-50-matplotlib-visualizations-the-master-plots-python/|Top 50 matplotlib Visualizations]] | ||
| * [[http://seaborn.pydata.org/|Seaborn]] is a library for making attractive and informative statistical graphics in Python, built on top of matplotlib | * [[http://seaborn.pydata.org/|Seaborn]] is a library for making attractive and informative statistical graphics in Python, built on top of matplotlib | ||
| * See also: [[https://www.datacamp.com/community/tutorials/seaborn-python-tutorial| | * See also: [[https://www.datacamp.com/community/tutorials/seaborn-python-tutorial| | ||
| Python Seaborn Tutorial For Beginners]] | Python Seaborn Tutorial For Beginners]] | ||
| + | * Communicating/displaying/plotting your data (possibly for people not of your field): | ||
| + | * [[https://uxknowledgebase.com/introduction-to-designing-data-visualizations-part-1-31c056556133|Introduction to Designing Data Visualizations — Part 1]] | ||
| + | * [[https://uxknowledgebase.com/tables-other-charts-data-visualization-part-2-cfc582e4712c|Tables & Other Charts — Data Visualization Part 2]] | ||
| + | * [[https://uxknowledgebase.com/tables-other-charts-data-visualization-part-3-5bfab15ce525|Tables & Other Charts — Data Visualization Part 3]] | ||
| * Working with colors | * Working with colors | ||
| * [[https://matplotlib.org/users/colormaps.html|Choosing colormaps]] | * [[https://matplotlib.org/users/colormaps.html|Choosing colormaps]] | ||
| - | * [[https://matplotlib.org/cmocean/|Beautiful colormaps for oceanography: cmocean]] | + | * [[https://matplotlib.org/cmocean/|cmocean: Beautiful colormaps for oceanography]] |
| + | * [[https://jiffyclub.github.io/palettable/|Palettable: Color palettes for Python]] | ||
| * [[http://colorbrewer2.org|ColorBrewer 2.0]] is a tool that can help you understand, and experiment with //sequential//, //diverging// and //qualitative// colormaps | * [[http://colorbrewer2.org|ColorBrewer 2.0]] is a tool that can help you understand, and experiment with //sequential//, //diverging// and //qualitative// colormaps | ||
| Line 217: | Line 233: | ||
| Help on //stack overflow//: [[https://stackoverflow.com/questions/tagged/cartopy|cartopy help]] | Help on //stack overflow//: [[https://stackoverflow.com/questions/tagged/cartopy|cartopy help]] | ||
| + | |||
| + | ===== Maps and projections resources ===== | ||
| + | |||
| + | ==== About projections ==== | ||
| + | |||
| + | * [[https://egsc.usgs.gov/isb//pubs/MapProjections/projections.html|Map projections from USGS poster]] | ||
| + | * [[https://pubs.usgs.gov/pp/1395/report.pdf|Map projections - A working manual (USGS)]] | ||
| + | |||
| + | ==== Libraries ==== | ||
| + | |||
| + | * Projections in vcs | ||
| + | * [[http://matplotlib.org/basemap/users/mapsetup.html|Projections in basemap]] | ||
| + | * [[https://scitools.org.uk/cartopy/docs/latest/crs/projections.html|Projections in cartopy]] | ||
| + | |||
| ===== 3D resources ===== | ===== 3D resources ===== | ||
| Line 260: | Line 290: | ||
| Where: [[http://pandas.pydata.org|Pandas web site]] | Where: [[http://pandas.pydata.org|Pandas web site]] | ||
| - | JYP's comment: pandas is supposed to be quite good for loading, processing and plotting time series, without writing custom code. You should at least have a quick look at: | + | JYP's comment: pandas is supposed to be quite good for loading, processing and plotting time series, without writing custom code. It is **very convenient for processing tables in xlsx files** (or csv, etc...). You should at least have a quick look at: |
| - | * The [[http://www.scipy-lectures.org/packages/statistics/index.html|Statistics in Python]] tutorial that combines Pandas, [[http://statsmodels.sourceforge.net/|Statsmodels]] and [[http://seaborn.pydata.org/|Seaborn]] | + | |
| - | * the cheat sheet on the [[https://www.enthought.com/services/training/pandas-mastery-workshop/|Enthought workshops advertising page]] | + | * Some //Cheat Sheets// (in the following order): |
| - | * the cheat sheet on the [[https://github.com/pandas-dev/pandas/tree/master/doc/cheatsheet|github Pandas doc page]] | + | - Basics: [[http://datacamp-community-prod.s3.amazonaws.com/dbed353d-2757-4617-8206-8767ab379ab3|Pandas basics]] (associated with the [[https://www.datacamp.com/community/blog/python-pandas-cheat-sheet|Pandas Cheat Sheet for Data Science in Python]] pandas introduction page) |
| + | - Intermediate: [[https://github.com/pandas-dev/pandas/tree/master/doc/cheatsheet|github Pandas doc page]] | ||
| + | - Advanced: the cheat sheet on the [[https://www.enthought.com/services/training/pandas-mastery-workshop/|Enthought workshops advertising page]] | ||
| + | * Some tutorials: | ||
| + | * [[https://www.datacamp.com/community/blog/python-pandas-cheat-sheet|Pandas Cheat Sheet for Data Science in Python]] pandas introduction page | ||
| + | * The [[http://www.scipy-lectures.org/packages/statistics/index.html|Statistics in Python]] tutorial that combines Pandas, [[http://statsmodels.sourceforge.net/|Statsmodels]] and [[http://seaborn.pydata.org/|Seaborn]] | ||
| ===== Scipy Lecture Notes ===== | ===== Scipy Lecture Notes ===== | ||
| Line 273: | Line 308: | ||
| This is **a really nice and useful document** that is regularly updated and used for the [[https://www.euroscipy.org/|EuroScipy]] tutorials. You will learn more things about python, numpy and matplotlib, debugging and optimizing scripts, and also learn about using python for statistics, image processing, machine learning, washing dishes (this is just to check if you have read this page), etc... | This is **a really nice and useful document** that is regularly updated and used for the [[https://www.euroscipy.org/|EuroScipy]] tutorials. You will learn more things about python, numpy and matplotlib, debugging and optimizing scripts, and also learn about using python for statistics, image processing, machine learning, washing dishes (this is just to check if you have read this page), etc... | ||
| - | ===== Quick Reference ===== | + | ===== Quick Reference and cheat sheets ===== |
| * The nice and convenient Python 2.7 Quick Reference: [[http://rgruet.free.fr/PQR27/PQR2.7_printing_a4.pdf|pdf]] - [[http://rgruet.free.fr/PQR27/PQR2.7.html|html]] | * The nice and convenient Python 2.7 Quick Reference: [[http://rgruet.free.fr/PQR27/PQR2.7_printing_a4.pdf|pdf]] - [[http://rgruet.free.fr/PQR27/PQR2.7.html|html]] | ||
| Line 280: | Line 315: | ||
| * Python 3 [[https://perso.limsi.fr/pointal/python:abrege|Quick reference]] and [[https://perso.limsi.fr/pointal/python:memento|Cheat sheet]] | * Python 3 [[https://perso.limsi.fr/pointal/python:abrege|Quick reference]] and [[https://perso.limsi.fr/pointal/python:memento|Cheat sheet]] | ||
| + | * [[https://www.cheatography.com/weidadeyue/cheat-sheets/jupyter-notebook/pdf_bw/|Jupyter Notebook Keyboard Shortcuts]] | ||
| + | |||
| + | ===== Misc tutorials ===== | ||
| + | |||
| + | * [[https://pyformat.info/|PyFormat]]: //With this site we try to show you the most common use-cases covered by the old and new style string formatting API with practical examples// | ||
| ===== Some good coding tips ===== | ===== Some good coding tips ===== | ||
| Line 316: | Line 356: | ||
| * [[https://www.datacamp.com/community/tutorials/data-science-python-ide|Top 5 Python IDEs For Data Science]] | * [[https://www.datacamp.com/community/tutorials/data-science-python-ide|Top 5 Python IDEs For Data Science]] | ||
| * [[http://noeticforce.com/best-python-ide-for-programmers-windows-and-mac|Python IDE: The10 Best IDEs for Python Programmers]] | * [[http://noeticforce.com/best-python-ide-for-programmers-windows-and-mac|Python IDE: The10 Best IDEs for Python Programmers]] | ||
| + | * [[https://www.techbeamers.com/best-python-ide-python-programming/|Get the Best Python IDE]] | ||
| * [[https://wiki.python.org/moin/IntegratedDevelopmentEnvironments]] | * [[https://wiki.python.org/moin/IntegratedDevelopmentEnvironments]] | ||
other/python/jyp_steps.txt · Last modified: 2024/03/07 10:15 by jypeter
