This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
other:python:misc_by_jyp [2021/07/06 11:25] jypeter Started the command-line args section |
other:python:misc_by_jyp [2021/10/26 11:27] jypeter Added sorting section |
||
---|---|---|---|
Line 2: | Line 2: | ||
<WRAP center round tip 60%> | <WRAP center round tip 60%> | ||
- | You will find on this page some useful, but unsorted, python tips and tricks that can't fit in a section of the main [[other:python:jyp_steps|JYP's recommended steps for learning python]] page | + | You will find on this page some **useful, but unsorted, python tips and tricks** that can't fit in a section of the main [[other:python:jyp_steps|JYP's recommended steps for learning python]] page |
</WRAP> | </WRAP> | ||
Line 39: | Line 39: | ||
True</code> | True</code> | ||
+ | ==== Working with paths and filenames ==== | ||
+ | |||
+ | If you are in a hurry, you can just use string functions to work with path and file names. But you will need some specific functions to check if a file exists, and similar operations. All these are available in 2 libraries that have similar functions. Both of these libraries can deal with Unix-type paths on Linux computers, and Windows-type paths on Windows computers | ||
+ | |||
+ | * [[https://docs.python.org/3/library/os.path.html|os.path]] //Common pathname manipulations// | ||
+ | * Available since... a long time! Use this if you want to avoid backward compatibility problems | ||
+ | * Some functions are directly in [[https://docs.python.org/3/library/os.html|os]] //Miscellaneous operating system interfaces//\\ e.g. [[https://docs.python.org/3/library/os.html#os.remove|os.remove]] and [[https://docs.python.org/3/library/os.html#os.rmdir|os.rmdir]] | ||
+ | * [[https://docs.python.org/3/library/pathlib.html|pathlib]] //Object-oriented filesystem paths// | ||
+ | * Available since Python version 3.4 | ||
+ | * [[https://docs.python.org/3/library/pathlib.html#correspondence-to-tools-in-the-os-module|Matching pathlib, and os or os.path functions]] | ||
+ | * [[https://docs.python.org/3/library/shutil.html|High-level file operations]] | ||
+ | |||
+ | |||
+ | === Example: getting the full path of the Python used === | ||
+ | |||
+ | Note: the actual python may be different from the default python! | ||
+ | |||
+ | <code>$ which python | ||
+ | /usr/bin/python | ||
+ | |||
+ | $ /modfs/modtools/miniconda3//envs/analyse_3.6_test/bin/python | ||
+ | >>> import sys, shutil | ||
+ | >>> shutil.which('python') | ||
+ | '/usr/bin/python' | ||
+ | >>> sys.executable | ||
+ | '/modfs/modtools/miniconda3//envs/analyse_3.6_test/bin/python'</code> | ||
+ | |||
+ | |||
+ | === Example: getting the full path of a script === | ||
+ | |||
+ | <code>>>> import os | ||
+ | >>> os.getcwd() | ||
+ | '/home/jypmce/PMIP4' | ||
+ | >>> os.path.exists('./argv_test.py') | ||
+ | True | ||
+ | >>> os.path.abspath('./argv_test.py') | ||
+ | '/home/jypmce/PMIP4/argv_test.py' | ||
+ | >>> os.path.exists('/home/jypmce/PMIP4/argv_test.py') | ||
+ | True | ||
+ | </code> | ||
+ | |||
+ | |||
+ | === Example: getting the size(s) of all the files in a directory === | ||
+ | |||
+ | <code>$ cd /data/jypmce/TestDir | ||
+ | $ ls -l | ||
+ | total 72 | ||
+ | -rw-r--r-- 1 jypmce ipsl 18147 Jun 25 2012 get_TS_cmip5.py | ||
+ | -rw-r--r-- 1 jypmce ipsl 16152 Jun 21 2012 get_TS_cmip5.py~ | ||
+ | -rw-r--r-- 1 jypmce ipsl 13954 Jul 3 2012 get_TS_cmip5_regular.py | ||
+ | -rw-r--r-- 1 jypmce ipsl 16539 Jun 22 2012 get_TS_cmip5_regular.py~</code> | ||
+ | |||
+ | <code>>>> os.chdir('/data/jypmce/TestDir') | ||
+ | >>> print(os.getcwd()) | ||
+ | /data/jypmce/TestDir | ||
+ | >>> files_list = os.listdir() | ||
+ | >>> files_list | ||
+ | ['get_TS_cmip5.py~', 'get_TS_cmip5_regular.py', 'get_TS_cmip5_regular.py~', 'get_TS_cmip5.py'] | ||
+ | >>> files_sizes = list(map(os.path.getsize, files_list)) | ||
+ | >>> files_sizes | ||
+ | [16152, 13954, 16539, 18147] | ||
+ | >>> sum(files_sizes) | ||
+ | 64792</code> | ||
+ | |||
+ | ==== Generating file names ==== | ||
+ | |||
+ | === Name depending on the current date/time === | ||
+ | |||
+ | <code>>>> import time | ||
+ | >>> plot_version = time.strftime('%Y%m%d_%H%M') | ||
+ | >>> f_name = 'test_%s.nc' % (plot_version,) | ||
+ | >>> f_name | ||
+ | 'test_20210827_1334.nc' | ||
+ | </code> | ||
+ | |||
+ | === Temporary file === | ||
+ | |||
+ | <code>>>> import tempfile, os | ||
+ | >>> f_tmp = tempfile.NamedTemporaryFile(mode='w', suffix='.nc', delete=False) | ||
+ | >>> f_tmp | ||
+ | <tempfile._TemporaryFileWrapper object at 0x2b5614743820> | ||
+ | >>> f_tmp.name | ||
+ | '/tmp/tmpi6uk9hre.nc' | ||
+ | >>> f_tmp.close() | ||
+ | >>> os.remove(f_tmp.name)</code> | ||
==== Using command-line arguments ==== | ==== Using command-line arguments ==== | ||
- | === The fast but non-flexible way === | + | === The extremely easy but non-flexible way: sys.argv === |
The name of a script, the number of arguments (including the name of the script), and the arguments (as strings) can be accessed through the ''sys.argv'' strings' list | The name of a script, the number of arguments (including the name of the script), and the arguments (as strings) can be accessed through the ''sys.argv'' strings' list | ||
Line 62: | Line 147: | ||
1 tas | 1 tas | ||
2 tas_tes.nc</code> | 2 tas_tes.nc</code> | ||
+ | |||
+ | === The C-style way: getopt === | ||
+ | |||
+ | Use [[https://docs.python.org/3/library/getopt.html|getopt]] (//C-style parser for command line options//) | ||
+ | |||
+ | === The deprecated Python way: optparse === | ||
+ | |||
+ | [[https://docs.python.org/3/library/optparse.html|optparse]] (//parser for command line options//) is **deprecated since Python version 3.2**! You should now use argparse (check [[https://docs.python.org/3/library/argparse.html#upgrading-optparse-code|Upgrading optparse code]] for converting from ''optparse'' to ''argparse'') | ||
+ | |||
+ | === The current Python way: argparse === | ||
+ | |||
+ | [[https://docs.python.org/3/library/argparse.html|argparse]] (//parser for command-line options, arguments and sub-commands//) is available since Python version 3.2 | ||
+ | |||
+ | ==== Using ordered dictionaries ==== | ||
+ | |||
+ | **Dictionary order is guaranteed to be insertion order**! Note that the [[https://docs.python.org/3/library/stdtypes.html#dict|usual Python dictionary]] also guarantees the order since version **3.6** | ||
+ | |||
+ | Check the [[https://docs.python.org/3/library/collections.html#collections.OrderedDict|OrderedDict class]] (''from collections import OrderedDict'') and the [[https://realpython.com/python-ordereddict/|OrderedDict vs dict in Python: The Right Tool for the Job]] tutorial | ||
+ | |||
+ | ==== Printing a readable version of long lists or dictionaries ==== | ||
+ | |||
+ | The [[https://docs.python.org/3/library/pprint.html|pprint]] module can be used for //pretty printing// objects (lists, dictionaries, ...). It will wrap long lines in a meaningful way | ||
+ | |||
+ | <code>>>> import pprint | ||
+ | |||
+ | >>> test_dic = {'AWI-ESM-1-1-LR_AWI':{'r1i1p1f1': {'grid': 'gn'}}, 'CESM2_NCAR':{'r1i1p1f1': {'grid': 'gn'}}, 'IPSL-CM6A-LR_IPSL':{'r1i1p1f1': {'grid': 'gr'}, 'r1i1p1f2': {'grid': 'gr'}, 'r1i1p1f3': {'grid': 'gr'}, 'r1i1p1f4': {'grid': 'gr'}}} | ||
+ | |||
+ | >>> print(test_dic) | ||
+ | {'AWI-ESM-1-1-LR_AWI': {'r1i1p1f1': {'grid': 'gn'}}, 'CESM2_NCAR': {'r1i1p1f1': {'grid': 'gn'}}, 'IPSL-CM6A-LR_IPSL': {'r1i1p1f1': {'grid': 'gr'}, 'r1i1p1f2': {'grid': 'gr'}, 'r1i1p1f3': {'grid': 'gr'}, 'r1i1p1f4': {'grid': 'gr'}}} | ||
+ | |||
+ | >>> pprint.pprint(test_dic) | ||
+ | {'AWI-ESM-1-1-LR_AWI': {'r1i1p1f1': {'grid': 'gn'}}, | ||
+ | 'CESM2_NCAR': {'r1i1p1f1': {'grid': 'gn'}}, | ||
+ | 'IPSL-CM6A-LR_IPSL': {'r1i1p1f1': {'grid': 'gr'}, | ||
+ | 'r1i1p1f2': {'grid': 'gr'}, | ||
+ | 'r1i1p1f3': {'grid': 'gr'}, | ||
+ | 'r1i1p1f4': {'grid': 'gr'}}} | ||
+ | |||
+ | >>> dir(test_dic) | ||
+ | ['__class__', '__contains__', '__delattr__', [... lots of unreadable stuff removed...] 'setdefault', 'update', 'values'] | ||
+ | |||
+ | >>> pprint.pprint(dir(test_dic)) | ||
+ | ['__class__', | ||
+ | '__contains__', | ||
+ | |||
+ | [... lots of lines removed in this example ] | ||
+ | |||
+ | 'setdefault', | ||
+ | 'update', | ||
+ | 'values'] | ||
+ | |||
+ | </code> | ||
+ | |||
+ | ==== Sorting ==== | ||
+ | |||
+ | <code>Some code</code> | ||
/* | /* |