User Tools

Site Tools


other:python:misc_by_jyp

This is an old revision of the document!


Useful python stuff

You will find on this page some useful, but unsorted, python tips and tricks that can't fit in a section of the main JYP's recommended steps for learning python page

Reading/setting environments variables

>>> os.environ['TMPDIR']
'/data/jypmce/climafcache'
>>> os.environ.get('SCRATCHDIR', '/data/jypmce/some_scratch_stuff')
'/data/jypmce/some_scratch_stuff'
>>> os.environ['temporary_env_var_for_THIS_script'] = 'some value'
>>> os.environ['temporary_env_var_for_THIS_script']
'some value'

Generating (aka raising) an error

This will stop the script, unless it is called in a function, and the code calling the function explicitely catches and deals with errors

Stopping a script

A user can use CTRL-C or kill to stop a script, or CTRL-Z to suspend it temporarily (use fg to resume a suspended script). The code below can be used by the script itself to interrupt its execution, instead of raising an error

sys.exit('Some optional message about why we are stopping')

Checking if a file/directory is writable by the current user

>>> os.access('/', os.W_OK)
False
>>> os.access('/home/jypmce/.bashrc', os.W_OK)
True

Working with paths and filenames

If you are in a hurry, you can just use string functions to work with path and file names. But you will need some specific functions to check if a file exists, and similar operations. All these are available in 2 libraries that have similar functions. Both of these libraries can deal with Unix-type paths on Linux computers, and Windows-type paths on Windows computers

Example: getting the size(s) of all the files in a directory

$ cd /data/jypmce/TestDir
$ ls -l
total 72
-rw-r--r-- 1 jypmce ipsl 18147 Jun 25  2012 get_TS_cmip5.py
-rw-r--r-- 1 jypmce ipsl 16152 Jun 21  2012 get_TS_cmip5.py~
-rw-r--r-- 1 jypmce ipsl 13954 Jul  3  2012 get_TS_cmip5_regular.py
-rw-r--r-- 1 jypmce ipsl 16539 Jun 22  2012 get_TS_cmip5_regular.py~
>>> os.chdir('/data/jypmce/TestDir')
>>> print(os.getcwd())
/data/jypmce/TestDir
>>> files_list = os.listdir()
>>> files_list
['get_TS_cmip5.py~', 'get_TS_cmip5_regular.py', 'get_TS_cmip5_regular.py~', 'get_TS_cmip5.py']
>>> files_sizes = list(map(os.path.getsize, files_list))
>>> files_sizes
[16152, 13954, 16539, 18147]
>>> sum(files_sizes)
64792

Using command-line arguments

The extremely easy but non-flexible way: sys.argv

The name of a script, the number of arguments (including the name of the script), and the arguments (as strings) can be accessed through the sys.argv strings' list

Simple argv_test.py test script:

#!/usr/bin/env python
import sys
nb_args = len(sys.argv)
print('Number of script arguments (including script name) =', nb_args)
for idx, val in enumerate(sys.argv):
    print(idx, val)
$ python argv_test.py
Number of script arguments (including script name) = 1
0 argv_test.py

$ python argv_test.py tas tas_tes.nc
Number of script arguments (including script name) = 3
0 argv_test.py
1 tas
2 tas_tes.nc

The C-style way: getopt

Use getopt (C-style parser for command line options)

The deprecated Python way: optparse

optparse (parser for command line options) is deprecated since Python version 3.2! You should now use argparse (check Upgrading optparse code for converting from optparse to argparse)

The current Python way: argparse

argparse (parser for command-line options, arguments and sub-commands) is available since Python version 3.2

Using ordered dictionaries

Printing a readable version of long lists or dictionaries

The pprint module can be used for pretty printing objects (lists, dictionaries, …). It will wrap long lines in a meaningful way

>>> from collections import OrderedDict

>>> test_dic = OrderedDict([('AWI-ESM-1-1-LR_AWI', {'r1i1p1f1': {'grid': 'gn'}}), ('CESM2_NCAR', {'r1i1p1f1': {'grid': 'gn'}}), ('IPSL-CM6A-LR_IPSL', {'r1i1p1f1': {'grid': 'gr'}, 'r1i1p1f2': {'grid': 'gr'}, 'r1i1p1f3': {'grid': 'gr'}, 'r1i1p1f4': {'grid': 'gr'}})])

>>> print(test_dic)
OrderedDict([('AWI-ESM-1-1-LR_AWI', {'r1i1p1f1': {'grid': 'gn'}}), ('CESM2_NCAR', {'r1i1p1f1': {'grid': 'gn'}}), ('IPSL-CM6A-LR_IPSL', {'r1i1p1f1': {'grid': 'gr'}, 'r1i1p1f2': {'grid': 'gr'}, 'r1i1p1f3': {'grid': 'gr'}, 'r1i1p1f4': {'grid': 'gr'}})])

>>> pprint.pprint(test_dic)
OrderedDict([('AWI-ESM-1-1-LR_AWI', {'r1i1p1f1': {'grid': 'gn'}}),
             ('CESM2_NCAR', {'r1i1p1f1': {'grid': 'gn'}}),
             ('IPSL-CM6A-LR_IPSL',
              {'r1i1p1f1': {'grid': 'gr'},
               'r1i1p1f2': {'grid': 'gr'},
               'r1i1p1f3': {'grid': 'gr'},
               'r1i1p1f4': {'grid': 'gr'}})])





[ PMIP3 Wiki Home ] - [ Help! ] - [ Wiki syntax ]

other/python/misc_by_jyp.1629295457.txt.gz · Last modified: 2021/08/18 14:04 by jypeter