This shows you the differences between two versions of the page.
Next revision | Previous revision Next revision Both sides next revision | ||
other:python:misc_by_jyp [2021/06/30 13:17] jypeter created |
other:python:misc_by_jyp [2021/08/18 14:04] jypeter Added pprint section |
||
---|---|---|---|
Line 2: | Line 2: | ||
<WRAP center round tip 60%> | <WRAP center round tip 60%> | ||
- | You will find on this page some useful, but unsorted, python tips and tricks that can't fit in a section of the main [[other:python:jyp_steps|JYP's recommended steps for learning python]] page | + | You will find on this page some **useful, but unsorted, python tips and tricks** that can't fit in a section of the main [[other:python:jyp_steps|JYP's recommended steps for learning python]] page |
</WRAP> | </WRAP> | ||
- | * Reading/setting environments variables\\ <code>>>> os.environ['TMPDIR'] | + | ==== Reading/setting environments variables ==== |
+ | |||
+ | |||
+ | <code>>>> os.environ['TMPDIR'] | ||
'/data/jypmce/climafcache' | '/data/jypmce/climafcache' | ||
>>> os.environ.get('SCRATCHDIR', '/data/jypmce/some_scratch_stuff') | >>> os.environ.get('SCRATCHDIR', '/data/jypmce/some_scratch_stuff') | ||
Line 14: | Line 17: | ||
</code> | </code> | ||
- | * Generating (aka //raising//) an error. This will stop the script, unless it is called in a function, and the code calling the function explicitely catches and deals with errors | + | ==== Generating (aka raising) an error ==== |
+ | |||
+ | This will stop the script, unless it is called in a function, and the code calling the function explicitely catches and deals with errors | ||
* <code>raise RuntimeError('\n\nOMG! An error! :-(\nAborting script...')</code> | * <code>raise RuntimeError('\n\nOMG! An error! :-(\nAborting script...')</code> | ||
* [[https://docs.python.org/3/tutorial/errors.html|Errors and Exceptions tutorial]] | * [[https://docs.python.org/3/tutorial/errors.html|Errors and Exceptions tutorial]] | ||
* [[https://docs.python.org/3/library/exceptions.html|Built-in Exceptions reference]] | * [[https://docs.python.org/3/library/exceptions.html|Built-in Exceptions reference]] | ||
- | * Stopping a script\\ <code>sys.exit('Some optional message about why we are stopping')</code> | + | |
+ | ==== Stopping a script ==== | ||
+ | |||
+ | A user can use ''CTRL-C'' or ''kill'' to stop a script, or ''CTRL-Z'' to suspend it temporarily (use ''fg'' to resume a suspended script). The code below can be used by the script itself to interrupt its execution, instead of raising an error | ||
+ | |||
+ | <code>sys.exit('Some optional message about why we are stopping')</code> | ||
+ | |||
+ | |||
+ | ==== Checking if a file/directory is writable by the current user ==== | ||
+ | |||
+ | <code>>>> os.access('/', os.W_OK) | ||
+ | False | ||
+ | >>> os.access('/home/jypmce/.bashrc', os.W_OK) | ||
+ | True</code> | ||
+ | |||
+ | ==== Working with paths and filenames ==== | ||
+ | |||
+ | If you are in a hurry, you can just use string functions to work with path and file names. But you will need some specific functions to check if a file exists, and similar operations. All these are available in 2 libraries that have similar functions. Both of these libraries can deal with Unix-type paths on Linux computers, and Windows-type paths on Windows computers | ||
+ | |||
+ | * [[https://docs.python.org/3/library/os.path.html|os.path]] //Common pathname manipulations// | ||
+ | * Available since... a long time! Use this if you want to avoid backward compatibility problems | ||
+ | * Some functions are directly in [[https://docs.python.org/3/library/os.html|os]] //Miscellaneous operating system interfaces//\\ e.g. [[https://docs.python.org/3/library/os.html#os.remove|os.remove]] and [[https://docs.python.org/3/library/os.html#os.rmdir|os.rmdir]] | ||
+ | * [[https://docs.python.org/3/library/pathlib.html|pathlib]] //Object-oriented filesystem paths// | ||
+ | * Available since Python version 3.4 | ||
+ | * [[https://docs.python.org/3/library/pathlib.html#correspondence-to-tools-in-the-os-module|Matching pathlib, and os or os.path functions]] | ||
+ | * [[https://docs.python.org/3/library/shutil.html|High-level file operations]] | ||
+ | |||
+ | === Example: getting the size(s) of all the files in a directory === | ||
+ | |||
+ | <code>$ cd /data/jypmce/TestDir | ||
+ | $ ls -l | ||
+ | total 72 | ||
+ | -rw-r--r-- 1 jypmce ipsl 18147 Jun 25 2012 get_TS_cmip5.py | ||
+ | -rw-r--r-- 1 jypmce ipsl 16152 Jun 21 2012 get_TS_cmip5.py~ | ||
+ | -rw-r--r-- 1 jypmce ipsl 13954 Jul 3 2012 get_TS_cmip5_regular.py | ||
+ | -rw-r--r-- 1 jypmce ipsl 16539 Jun 22 2012 get_TS_cmip5_regular.py~</code> | ||
+ | |||
+ | <code>>>> os.chdir('/data/jypmce/TestDir') | ||
+ | >>> print(os.getcwd()) | ||
+ | /data/jypmce/TestDir | ||
+ | >>> files_list = os.listdir() | ||
+ | >>> files_list | ||
+ | ['get_TS_cmip5.py~', 'get_TS_cmip5_regular.py', 'get_TS_cmip5_regular.py~', 'get_TS_cmip5.py'] | ||
+ | >>> files_sizes = list(map(os.path.getsize, files_list)) | ||
+ | >>> files_sizes | ||
+ | [16152, 13954, 16539, 18147] | ||
+ | >>> sum(files_sizes) | ||
+ | 64792</code> | ||
+ | ==== Using command-line arguments ==== | ||
+ | |||
+ | === The extremely easy but non-flexible way: sys.argv === | ||
+ | |||
+ | The name of a script, the number of arguments (including the name of the script), and the arguments (as strings) can be accessed through the ''sys.argv'' strings' list | ||
+ | |||
+ | Simple ''argv_test.py'' test script: | ||
+ | <code>#!/usr/bin/env python | ||
+ | import sys | ||
+ | nb_args = len(sys.argv) | ||
+ | print('Number of script arguments (including script name) =', nb_args) | ||
+ | for idx, val in enumerate(sys.argv): | ||
+ | print(idx, val)</code> | ||
+ | |||
+ | <code>$ python argv_test.py | ||
+ | Number of script arguments (including script name) = 1 | ||
+ | 0 argv_test.py | ||
+ | |||
+ | $ python argv_test.py tas tas_tes.nc | ||
+ | Number of script arguments (including script name) = 3 | ||
+ | 0 argv_test.py | ||
+ | 1 tas | ||
+ | 2 tas_tes.nc</code> | ||
+ | |||
+ | === The C-style way: getopt === | ||
+ | |||
+ | Use [[https://docs.python.org/3/library/getopt.html|getopt]] (//C-style parser for command line options//) | ||
+ | |||
+ | === The deprecated Python way: optparse === | ||
+ | |||
+ | [[https://docs.python.org/3/library/optparse.html|optparse]] (//parser for command line options//) is **deprecated since Python version 3.2**! You should now use argparse (check [[https://docs.python.org/3/library/argparse.html#upgrading-optparse-code|Upgrading optparse code]] for converting from ''optparse'' to ''argparse'') | ||
+ | |||
+ | === The current Python way: argparse === | ||
+ | |||
+ | [[https://docs.python.org/3/library/argparse.html|argparse]] (//parser for command-line options, arguments and sub-commands//) is available since Python version 3.2 | ||
+ | |||
+ | ==== Using ordered dictionaries ==== | ||
+ | |||
+ | Check the [[https://realpython.com/python-ordereddict/|OrderedDict vs dict in Python: The Right Tool for the Job]] tutorial | ||
+ | |||
+ | ==== Printing a readable version of long lists or dictionaries ==== | ||
+ | |||
+ | The [[https://docs.python.org/3/library/pprint.html|pprint]] module can be used for //pretty printing// objects (lists, dictionaries, ...). It will wrap long lines in a meaningful way | ||
+ | |||
+ | <code>>>> from collections import OrderedDict | ||
+ | |||
+ | >>> test_dic = OrderedDict([('AWI-ESM-1-1-LR_AWI', {'r1i1p1f1': {'grid': 'gn'}}), ('CESM2_NCAR', {'r1i1p1f1': {'grid': 'gn'}}), ('IPSL-CM6A-LR_IPSL', {'r1i1p1f1': {'grid': 'gr'}, 'r1i1p1f2': {'grid': 'gr'}, 'r1i1p1f3': {'grid': 'gr'}, 'r1i1p1f4': {'grid': 'gr'}})]) | ||
+ | |||
+ | >>> print(test_dic) | ||
+ | OrderedDict([('AWI-ESM-1-1-LR_AWI', {'r1i1p1f1': {'grid': 'gn'}}), ('CESM2_NCAR', {'r1i1p1f1': {'grid': 'gn'}}), ('IPSL-CM6A-LR_IPSL', {'r1i1p1f1': {'grid': 'gr'}, 'r1i1p1f2': {'grid': 'gr'}, 'r1i1p1f3': {'grid': 'gr'}, 'r1i1p1f4': {'grid': 'gr'}})]) | ||
+ | |||
+ | >>> pprint.pprint(test_dic) | ||
+ | OrderedDict([('AWI-ESM-1-1-LR_AWI', {'r1i1p1f1': {'grid': 'gn'}}), | ||
+ | ('CESM2_NCAR', {'r1i1p1f1': {'grid': 'gn'}}), | ||
+ | ('IPSL-CM6A-LR_IPSL', | ||
+ | {'r1i1p1f1': {'grid': 'gr'}, | ||
+ | 'r1i1p1f2': {'grid': 'gr'}, | ||
+ | 'r1i1p1f3': {'grid': 'gr'}, | ||
+ | 'r1i1p1f4': {'grid': 'gr'}})]) | ||
+ | </code> | ||
/* | /* | ||
- | * tip template\\ <code>Some code</code> | + | ==== Tip template ==== |
+ | |||
+ | <code>Some code</code> | ||
*/ | */ | ||