other:python:jyp_steps
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
|
other:python:jyp_steps [2016/01/21 17:10] jypeter numpy, scipy... |
other:python:jyp_steps [2016/01/22 17:07] jypeter More changes |
||
|---|---|---|---|
| Line 1: | Line 1: | ||
| - | ====== Following JYP's recommended steps ====== | + | ====== JYP's recommended steps for learning python ====== |
| As can be expected, there is **a lot** of online python documentation available, and it's easy to get lost. You can always use google to find an answer to your problem, and you will probably end up looking at lots of answers on [[http://stackoverflow.com/questions/tagged/python|Stack Overflow]] or a similar site. But it's always better to know where you can find some good documentation... and to spend some time to read the documentation | As can be expected, there is **a lot** of online python documentation available, and it's easy to get lost. You can always use google to find an answer to your problem, and you will probably end up looking at lots of answers on [[http://stackoverflow.com/questions/tagged/python|Stack Overflow]] or a similar site. But it's always better to know where you can find some good documentation... and to spend some time to read the documentation | ||
| This page tries to list some //python for the scientist// related resources, in a suggested reading order. **Do not print anything** (or at least not everything), but it's a good idea to download all the //pdf// files in the same place, so that you can easily open and search the documents | This page tries to list some //python for the scientist// related resources, in a suggested reading order. **Do not print anything** (or at least not everything), but it's a good idea to download all the //pdf// files in the same place, so that you can easily open and search the documents | ||
| + | |||
| + | ===== JYP's introduction to python ===== | ||
| + | |||
| + | ==== Part 1 ==== | ||
| + | |||
| + | You can start using python by reading the {{:other:python:python_intro_ipsl_oct2013_v2.pdf|Bien démarrer avec python}} tutorial that was used during a 2013 IPSL python class: | ||
| + | * this tutorial is in French (my apologies for the lack of translation, but it should be easy to understand) | ||
| + | * it's an introduction to python (and programming) for the climate scientist: after reading this tutorial, you should be able to do most of the things you usually do in a shell script | ||
| + | * python types, tests, loops, reading a text file | ||
| + | * the tutorial is very detailed about string handling, because strings offer an easy way to practice working with indices (indexing and slicing), before indexing numpy arrays. And our usual pre/post-processing scripts often need to do a lot of string handling in order to generate the file/variable/experiment names | ||
| + | * after reading this tutorial, you should practice with the following: | ||
| + | * [[https://files.lsce.ipsl.fr/public.php?service=files&t=9731fdad4521ac5fa6e84b392d3a2e44|Basic python training test (ipython notebook version)]] | ||
| + | * {{:other:python:tp_intro_python_oct2013_no_solutions.pdf|Basic python training test (pdf version)}} | ||
| + | * {{:other:python:tp_intro_python_oct2013_full.pdf|Basic python training test (pdf version, with answers)}} | ||
| + | |||
| + | ==== Part 2 ==== | ||
| + | |||
| + | Once you have done your first steps, you should read [[http://www.lsce.ipsl.fr/Phocea/file.php?class=page&file=5/pythonCDAT_jyp_1sur2_070227.pdf|Plus loin avec Python]] (start at page 39, the previous pages are an old version of what was covered in //Part 1// above) | ||
| + | * this tutorial is in French (sorry again) | ||
| + | * after reading this tutorial, you will be able to do more than you can do in a shell script, in an easier way | ||
| + | * advanced string formatting | ||
| + | * creating functions and using modules | ||
| + | * working with file paths and handling files without calling external Linux programs\\ (e.g. using ''os.remove(file_name)'' instead of ''rm $file_name'') | ||
| + | * using command-line options for scripts, or using configuration files | ||
| + | * calling external programs | ||
| ===== The official python documentation ===== | ===== The official python documentation ===== | ||
| - | You do not need to read all the python documentation at this step, but it is really well made and you should at least have a look at it. The **Tutorial** is really well made, and you should have a look at the table of content of the **Python Standard Library**. There is a lot in the default library that can make your life easier | + | You do not need to read all the python documentation at this step, but it is really well made and you should at least have a look at it. The **Tutorial** is very good, and you should have a look at the table of content of the **Python Standard Library**. There is a lot in the default library that can make your life easier |
| ==== Python 2.7 ==== | ==== Python 2.7 ==== | ||
| - | [[https://docs.python.org/2.7/|html]] - [[https://docs.python.org/2.7/download.html|pdf]] | + | [[https://docs.python.org/2.7/|html]] - [[https://docs.python.org/2.7/download.html|pdf (in a zip file)]] |
| ==== Python 3 ==== | ==== Python 3 ==== | ||
| - | [[https://docs.python.org/3/|html]] - [[https://docs.python.org/3/download.html|pdf]] | + | [[https://docs.python.org/3/|html]] - [[https://docs.python.org/3/download.html|pdf (in a zip file)]] |
| ===== Numpy and Scipy ===== | ===== Numpy and Scipy ===== | ||
| - | Summary: Python provides //ordered// objects (e.g. lists, strings, ...) and some math operators, but you can't do real heavy computation with these. **Numpy** makes it possible to work with data arrays and using array syntax and masks (instead of explicit nested loops and tests) and the apropriate numpy functions will allow you to get performance similar to what you would get with a compiled program! **Scipy** adds more scientific functions | + | Summary: Python provides //ordered// objects (e.g. lists, strings, basic arrays, ...) and some math operators, but you can't do real heavy computation with these. **Numpy** makes it possible to work with multi-dimensional data arrays, and using array syntax and masks (instead of explicit nested loops and tests) and the apropriate numpy functions will allow you to get performance similar to what you would get with a compiled program! **Scipy** adds more scientific functions |
| + | |||
| + | Where: [[http://docs.scipy.org/doc/|html and pdf documentation]] | ||
| How to get started? | How to get started? | ||
| - | - always remember that indices start at ''0'' and that the last element of an array is at index ''-1''! Learn about indexing and slicing by manipulating a string (try '''This document by JY is awesome!'[::-1]'') | + | - always remember that indices start at ''0'' and that the last element of an array is at index ''-1''!\\ First learn about //indexing// and //slicing// by manipulating strings, as shown in [[#part1|Part 1]] above (try '''This document by JY is awesome!'[::-1]'' and '''This document by JY is awesome!'[slice(None, None, -1)]'') 8-) |
| - read the [[https://docs.scipy.org/doc/numpy-dev/user/quickstart.html|Quickstart tutorial]] | - read the [[https://docs.scipy.org/doc/numpy-dev/user/quickstart.html|Quickstart tutorial]] | ||
| - have a quick look at the full documentation to know where things are | - have a quick look at the full documentation to know where things are | ||
| - | - Numpy User Guide: | + | - Numpy User Guide |
| - Numpy Reference Guide | - Numpy Reference Guide | ||
| - Scipy Reference Guide | - Scipy Reference Guide | ||
| + | |||
| + | ===== cdms2 and netCDF4 ===== | ||
| + | |||
| + | There is a good chance that your input array data will come from a file in the [[http://www.unidata.ucar.edu/software/netcdf/|NetCDF]] format. Depending on which [[other:python:starting#some_python_distributions|python distribution]] you are using, you can use the //cdms2// or or //netCDF4// modules to read the data. | ||
| + | |||
| + | Note: the NetCDF file format is self-documented, and the metadata of climate date files often follows the [[http://cfconventions.org/|CF (Climate and Forecast) Metadata Conventions]] | ||
| + | |||
| + | ==== cdms2 ==== | ||
| + | |||
| + | Summary: cdms2 can read/write netCDF files (and read //grads// dat+ctl files) and provides a higher level interface than netCDF4. Unfortunately, cdms2 is only available in the UV-CDAT distribution, and distributions where somebody has installed some version of //cdat-lite//. When you can use cdms2, you also have access to //cdtime//, that is very useful for handling time axis data. | ||
| + | |||
| + | How to get started: | ||
| + | - read [[http://www.lsce.ipsl.fr/Phocea/file.php?class=page&file=5/pythonCDAT_jyp_2sur2_070306.pdf|JYP's cdms tutorial]], starting at page 54 | ||
| + | - the tutorial is in French (soooorry!) | ||
| + | - you have to replace //cdms// with **cdms2**, and //MV// with **MV2** (sooorry about that, the tutorial was written when CDAT was based on //Numeric// instead of //numpy// to handle array data) | ||
| + | - read the [[http://uv-cdat.llnl.gov/documentation/cdms/cdms.html|official cdms documentation]] | ||
| + | - ask questions and get answers on the [[http://uvcdat.askbot.com/questions/|UV-CDAT askbot]] | ||
| + | |||
| + | |||
| + | ==== netCDF4 ==== | ||
| + | |||
| + | Summary: netCDF4 can read/write netCDF files and is available in most python distributions | ||
| + | |||
| + | Where: [[http://unidata.github.io/netcdf4-python/]] | ||
| + | |||
| ===== Matplotlib ===== | ===== Matplotlib ===== | ||
| Line 42: | Line 94: | ||
| - a Matplotlib //Axis// is a plot inside a Figure... [[http://matplotlib.org/faq/usage_faq.html#parts-of-a-figure|More details]] | - a Matplotlib //Axis// is a plot inside a Figure... [[http://matplotlib.org/faq/usage_faq.html#parts-of-a-figure|More details]] | ||
| - some examples are more //pythonic// (ie object oriented) than others, some example mix different styles of coding, all this can be confusing. Try to [[http://matplotlib.org/faq/usage_faq.html#coding-styles|use an object oriented way of doing things]]! | - some examples are more //pythonic// (ie object oriented) than others, some example mix different styles of coding, all this can be confusing. Try to [[http://matplotlib.org/faq/usage_faq.html#coding-styles|use an object oriented way of doing things]]! | ||
| - | - sometimes the results of the python/matplolib commands are shown directly, sometimes not. It depends if you are in [[http://matplotlib.org/faq/usage_faq.html#what-is-interactive-mode|interactive or non-interactive]] mode | + | - sometimes the results of the python/matplolib commands are displayed directly, sometimes not. It depends if you are in [[http://matplotlib.org/faq/usage_faq.html#what-is-interactive-mode|interactive or non-interactive]] mode |
| - the documentation may mention [[http://matplotlib.org/faq/usage_faq.html#what-is-a-backend|backends]]. What?? Basically, you use python commands to create a plot, and the backend is the //thing// that will render your plot on the screen or in a file (png, pdf, etc...) | - the documentation may mention [[http://matplotlib.org/faq/usage_faq.html#what-is-a-backend|backends]]. What?? Basically, you use python commands to create a plot, and the backend is the //thing// that will render your plot on the screen or in a file (png, pdf, etc...) | ||
| - Read the [[http://www.labri.fr/perso/nrougier/teaching/matplotlib/|Matplotlib tutorial by Nicolas Rougier]] | - Read the [[http://www.labri.fr/perso/nrougier/teaching/matplotlib/|Matplotlib tutorial by Nicolas Rougier]] | ||
| Line 65: | Line 117: | ||
| Where: [[http://www.scipy-lectures.org/_downloads/ScipyLectures-simple.pdf|pdf]] - [[http://www.scipy-lectures.org/|html]] | Where: [[http://www.scipy-lectures.org/_downloads/ScipyLectures-simple.pdf|pdf]] - [[http://www.scipy-lectures.org/|html]] | ||
| - | This is a really nice document that is regularly updated and used for the [[https://www.euroscipy.org/|EuroScipy]] tutorials | + | This is a really nice document that is regularly updated and used for the [[https://www.euroscipy.org/|EuroScipy]] tutorials. You will learn more things about python, numpy and matplotlib, debugging and optimizing scripts, and also learn about using python for statistics, image processing, machine learning, washing dishes (this is just to check if you have read this page), etc... |
| ===== Quick Reference ===== | ===== Quick Reference ===== | ||
| - | * The nice Python 2.7 Quick Reference: [[http://rgruet.free.fr/PQR27/PQR2.7_printing_a4.pdf|pdf]] - [[http://rgruet.free.fr/PQR27/PQR2.7.html|html]] | + | * The nice and convenient Python 2.7 Quick Reference: [[http://rgruet.free.fr/PQR27/PQR2.7_printing_a4.pdf|pdf]] - [[http://rgruet.free.fr/PQR27/PQR2.7.html|html]] |
| ===== Some good coding tips ===== | ===== Some good coding tips ===== | ||
| Line 97: | Line 149: | ||
| The official [[https://docs.python.org/2.7/howto/pyporting.html|Porting Python 2 Code to Python 3]] page gives the required information to make the transition from python 2 to python 3. It is still safe to use Python 2.7, so there is no rush to change to Python 3. | The official [[https://docs.python.org/2.7/howto/pyporting.html|Porting Python 2 Code to Python 3]] page gives the required information to make the transition from python 2 to python 3. It is still safe to use Python 2.7, so there is no rush to change to Python 3. | ||
| + | |||
| + | ===== What now? ===== | ||
| + | |||
| + | You can do a lot more with python! But if you have read at least a part of this page, you should be able to find and use the modules you need. Make sure you do not reinvent the wheel! Use existing packages when possible, and make sure to report bugs or errors in the documentations when you find some | ||
| /* standard page footer */ | /* standard page footer */ | ||
other/python/jyp_steps.txt · Last modified: 2024/03/07 10:15 by jypeter
