PMIP3 wiki

User Tools

Site Tools

You are here: index » other » python » jyp_steps
other:python:jyp_steps

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
Next revisionBoth sides next revision
other:python:jyp_steps [2016/01/29 16:41] – Added the numpy for matlab users references jypeterother:python:jyp_steps [2016/05/20 15:41] – Added a link to 'Working with Python' at the top jypeter
Line 1: Line 1:
 ====== JYP's recommended steps for learning python ====== ====== JYP's recommended steps for learning python ======
 +
 +<note tip>If you don't know which python distribution to use and how to start the python interpreter, you should first read the [[starting|Working with Python]] page</note>
  
 As can be expected, there is **a lot** of online python documentation available, and it's easy to get lost. You can always use google to find an answer to your problem, and you will probably end up looking at lots of answers on [[http://stackoverflow.com/questions/tagged/python|Stack Overflow]] or a similar site. But it's always better to know where you can find some good documentation... and to spend some time to read the documentation As can be expected, there is **a lot** of online python documentation available, and it's easy to get lost. You can always use google to find an answer to your problem, and you will probably end up looking at lots of answers on [[http://stackoverflow.com/questions/tagged/python|Stack Overflow]] or a similar site. But it's always better to know where you can find some good documentation... and to spend some time to read the documentation
Line 49: Line 51:
 Where: [[http://docs.scipy.org/doc/|html and pdf documentation]] Where: [[http://docs.scipy.org/doc/|html and pdf documentation]]
  
-How to get started?+==== Getting started ==== 
   - always remember that indices start at ''0'' and that the last element of an array is at index ''-1''!\\ First learn about //indexing// and //slicing// by manipulating strings, as shown in [[#part1|Part 1]] above (try '''This document by JY is awesome!'[::-1]'' and '''This document by JY is awesome!'[slice(None, None, -1)]'') 8-)   - always remember that indices start at ''0'' and that the last element of an array is at index ''-1''!\\ First learn about //indexing// and //slicing// by manipulating strings, as shown in [[#part1|Part 1]] above (try '''This document by JY is awesome!'[::-1]'' and '''This document by JY is awesome!'[slice(None, None, -1)]'') 8-)
   - if you are a Matlab user (but the references are interesting for others as well), you can read the following:   - if you are a Matlab user (but the references are interesting for others as well), you can read the following:
Line 59: Line 62:
     - Numpy Reference Guide     - Numpy Reference Guide
     - Scipy Reference Guide     - Scipy Reference Guide
 +
 +==== Beware of the array view side effects ====
 +
 +<note warning>When you take a slice of an array, you get a **//View//** : an array that has a new shape but that still shares its data with the first array.
 +
 +That is not a problem when you only read the values, but **if you change the values of the //View//, you change the values of the first array** (and vice-versa)! If that is not what want, do not forget to **make a copy** of the data before working on it!
 +
 +//Views// are a good thing most of the time, so only make a copy of your data when needed, because otherwise copying a big array will just be a waste of CPU and computer memory. Anyway, it is always better to understand what you are doing... :-P
 +
 +Check the example below and the [[https://docs.scipy.org/doc/numpy-dev/user/quickstart.html#copies-and-views|copies and views]] part of the quickstart tutorial.
 +
 +<code python>
 +>>> import numpy as np
 +>>> a = np.arange(30).reshape((3,10))
 +>>> a
 +array([[ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9],
 +       [10, 11, 12, 13, 14, 15, 16, 17, 18, 19],
 +       [20, 21, 22, 23, 24, 25, 26, 27, 28, 29]])
 +
 +>>> b = a[1, :]
 +>>> b
 +array([10, 11, 12, 13, 14, 15, 16, 17, 18, 19])
 +
 +>>> b[3:7] = 0
 +>>> b
 +array([10, 11, 12,  0,  0,  0,  0, 17, 18, 19])
 +
 +>>> a
 +array([[ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9],
 +       [10, 11, 12,  0,  0,  0,  0, 17, 18, 19],
 +       [20, 21, 22, 23, 24, 25, 26, 27, 28, 29]])
 +       
 +>>> a[:, 2:4] = -1
 +>>> a
 +array([[ 0,  1, -1, -1,  4,  5,  6,  7,  8,  9],
 +       [10, 11, -1, -1,  0,  0,  0, 17, 18, 19],
 +       [20, 21, -1, -1, 24, 25, 26, 27, 28, 29]])
 +       
 +>>> b
 +array([10, 11, -1, -1,  0,  0,  0, 17, 18, 19])
 +
 +>>> c = a[1, :].copy()
 +>>> c
 +array([10, 11, -1, -1,  0,  0,  0, 17, 18, 19])
 +
 +>>> c[:] = 9
 +>>> c
 +array([9, 9, 9, 9, 9, 9, 9, 9, 9, 9])
 +
 +>>> b
 +array([10, 11, -1, -1,  0,  0,  0, 17, 18, 19])
 +
 +>>> a
 +array([[ 0,  1, -1, -1,  4,  5,  6,  7,  8,  9],
 +       [10, 11, -1, -1,  0,  0,  0, 17, 18, 19],
 +       [20, 21, -1, -1, 24, 25, 26, 27, 28, 29]])
 +</code></note>
  
 ===== cdms2 and netCDF4 ===== ===== cdms2 and netCDF4 =====
Line 131: Line 191:
  
   * [[http://blog.codinghorror.com/a-pragmatic-quick-reference/|A Pragmatic Quick Reference]]   * [[http://blog.codinghorror.com/a-pragmatic-quick-reference/|A Pragmatic Quick Reference]]
 +
 +===== Debugging your code =====
 +
 +There is only so much you can do with staring at your code in your favorite text editor, and adding ''print'' lines in your code (or using [[https://docs.python.org/2/howto/logging.html#logging-basic-tutorial|logging]] instead of ''print''). The next step is to **use the python debugger**!
 +
 +==== Debugging in text mode ====
 +
 +  - Start the script with: ''python -m pdb my_script.py''
 +  - Type ''run'' (or **r**) to go to the first line of the script
 +  - Type ''continue'' (or **c**) to execute the script to the end, or till the first breakpoint or error is reached
 +  - Use ''where'' (or **w**) to check the call stack that led to the current stop. Use ''up'' and ''down'' to navigate through the call stack and examine the values of the functions' parameters
 +  - Type ''break NNN'' to stop at line NNN
 +  - Use ''type(var)'' and ''print var'' to check the type and values of variables. You can also change the variables' values on the fly!
 +  - Type ''run'' (or **r**) to restart the script
 +  - Use ''next'' and ''step'' to execute some parts of the script line by line. If a code line calls a function:
 +    * ''next'' (or **n**) will execute a function and stop on the next line
 +    * ''step'' (or **s**) will stop at the first line **inside the function**  
 +  - Check the [[https://docs.python.org/2/library/pdb.html#debugger-commands|debugger commands]] for details, or type ''help'' in the debugger for using the built-in help
 +
 +==== Using pydebug ====
 +
 +Depending on the distribution, the editor and the programming environment you use, you may have access to a graphical version of the debugger. UV-CDAT users can use ''pydebug my_script.py''
  
 ===== Improving the performance of your code ===== ===== Improving the performance of your code =====
Line 143: Line 225:
  
 Hint: before optimizing your script, you should spent some time //profiling// it, in order to only spend time improving the slow parts of your script Hint: before optimizing your script, you should spent some time //profiling// it, in order to only spend time improving the slow parts of your script
 +
 +==== Useful packages ====
 +
 +  * [[https://github.com/pydata/numexpr|Numexpr]]: //Numexpr is a **fast numerical expression evaluator for NumPy**. With it, expressions that operate on arrays (like "3*a+4*b") are accelerated and use less memory than doing the same calculation in Python.//
 +  * [[http://www.pytables.org/|PyTables]]: //PyTables is a package for managing hierarchical datasets and designed to efficiently and **easily cope with extremely large amounts of data**//
  
 ==== Tutorials by Ian Osvald ==== ==== Tutorials by Ian Osvald ====
other/python/jyp_steps.txt · Last modified: 2025/02/26 11:40 by jypeter
Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki