Note: the former and outdated version (before the massive usage of conda) is still available for reference

In order to start working with Python, you need to have a Python distribution installed on your local computer or on the remote Linux server(s) you work on. A distribution provides a Python interpreter, and Python extensions (aka Python modules or packages). You may have several distributions installed on your computer and you need to know how to initialize them, and which one you are using at a given time (type which python on Linux to determine where the python executable is located)

If you are using a Linux computer or a Mac, you should already have a default python installed. The following example shows where the python interpreter is installed on the obelix LSCE servers (if it's in /usr/bin, it's the default python) and which version it is (example below: version 2.7.5 compiled in April 2019)

# Which is the current python (e.g. where is it located)?
 > which python
/usr/bin/python

# Where is it coming from ('rpm' works on a RedHat-like Linux machine)?
 > rpm -qf /usr/bin/python
python-2.7.5-77.el7_6.x86_64

# You also get some information when you start the interpreter
 > python
Python 2.7.5 (default, Apr  9 2019, 14:30:50)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-36)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>

1. Start a terminal on an obelix server
2. Type python to access the default Python 2 interpreter available on the servers…
3. …or follow the steps below to use the more complete CDAT distribution.
   Remember that you can exit the interpreter by typing CTRL-D (or quit() or exit())
   1. Determine if you are using a bash or tcsh shell, then type the following command to initialize conda
      • In a bash shell: source ~jypeter/.conda3_jyp.sh
      • In a tcsh shell: source ~jypeter/.conda3_jyp.csh
   2. Choose which flavor of Python you want by typing:
      • Python 2.7.x: conda activate cdatm_py2
      • Python 3.x: conda activate cdatm_py3
   3. Type python to start the interpreter
   4. Note: if you will have to use python regularly, you should add the source line above to your .cshrc /.login file (tcsh users) or .bashrc/.profile (bash users)
      • Do not add the conda activate cdatm_pyN line to the shell config files, because this will create potential side-effects!
        Only type this activation line in the terminals where you will need to use this specific version of Python

Note: Python is an interpreted language and we speak of Python scripts rather than Python programs.

• python: start the interpreter and start working interactively
• python script.py: execute script.py and exit
• python -i script: execute script.py and stay in the interpreter (-i = Interactive)

Type man python if you want to see what other command line options are available

You don't have to explicitly call the python interpreter, if the interpreter is specified in a #! shebang comment on the first line of the script with, and the script has its execution bit set (chmod +x my_script.py).

If you have a basic_script.py file with the following content in a directory…

#!/usr/bin/env python

import sys

script_name = sys.argv[0]

print('Hello world, I am the ' + script_name + ' script')

# The end

…and if you are in the same directory and have initialized the Python distribution you want, you can then run the script the following way:

 > chmod +x basic_script.py

 > ls -l basic_script.py
-rwxr-xr-x [...] basic_script.py

 > ./basic_script.py
Hello world, I am the ./basic_script.py script

1. Read the rest of this page to get a better understanding of what you have learned in this section, and use Python more efficiently
2. Learn Python by reading what you need on the JYP's recommended steps for learning python page

This section will help you choose a distribution in the big Python ecosystem (many distributions, python version 2.7.* or 3.*, …) on some of the servers used by LSCE users. You should use a distribution that is already available near your data (e.g. do the computation on a server near your data, do not move/duplicate the data!) and try to identify who is maintaining it, if you need help or additional packages.

Only install a distribution yourself if you need it on your local computer (desktop or laptop), or if you need to install some modules that can't be installed by the contacts listed below. A python distribution will require several Gb of disk space, so do not install it in a backed up home directory!

You can use either Python 2 or Python 3. Most packages are now available in both versions, but you should make sure that the most important package/s you need is/are available in the selected Python version. You can check the differences between both versions and try to write scripts that will work in both versions!

• Contact: the LSCE system administrators (help-lsce@lsce.ipsl.fr)
• Where: obelix interactive servers and cluster at LSCE
• Initialization type: module based + conda
• What's installed: type conda list after initializing the LSCE distribution

 > module avail
[...]
4ARTIC/3.6         grib_api/1.14      netcdf/3           python/2.7.5
batch_env          grib_api/1.14.0    netcdf/4           python/3.6
[...]
glost/0.3.1        ncview/2.1.7       python/2.7
[...]

 > module load python/2.7

 > which python
/usr/local/install/python-2.7/bin/python

 > python
Python 2.7.15 |Anaconda, Inc.| (default, Oct 10 2018, 21:32:13)
[GCC 7.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>

• Contact: the TGCC hotline (hotline.tgcc@cea.fr)
• Where: CEA TGCC
• Initialization type: module based
• What's installed: it depends…

# Get the default version of the Python based modules
$ module avail -t -d | egrep '(python|cdat)'
flavor/cdat/standard(default)
flavor/nest/python2(default)
flavor/pytorch/python2(default)
flavor/tensorflow/gpu_python2(default)
intelpython2/2019.0(default)
intelpython3/2019.0(default)
python/2.7.14(default)
python3/3.6.4(default)
cdat/8.0(default)

$ module load python
[...]
load module python/2.7.14 (Python)

$ which python
/ccc/products/python-2.7.14/intel--17.0.4.196__openmpi--2.0.2/default/bin/python

$ python
Python 2.7.14 (default, Jan 11 2018, 16:43:59)
[GCC 4.8.5] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>

CDAT (Community Data Analysis Tools) is a powerful and complete front-end to a rich set of visual-data exploration and analysis capabilities well suited for climate data analysis problems, and is recommended by JYP.

CDAT is available for Mac, Linux and Windows 10 (Windows 10 + Windows Subsystem for Linux, and Ubuntu).

• Contact: Jean-Yves Peterschmitt (JYP) @ LSCE
• Where: obelix interactive servers and cluster at LSCE, ciclad interactive servers and cluster at IPSL, irene @ TGCC
• Initialization type: conda based or module based
• What's installed: type conda list after initializing a specific CDAT distribution

The following versions are maintained by JYP, on the Linux servers where LSCE users have accounts

CDAT 8.0 is installed at TGCC and can be initialized with:

• Python 2: module load cdat
• Python 3: module load flavor/cdat/python3 cdat

The way you initialize CDAT depends on:

• which server you want to use it on
• which shell you are using in your terminals: tcsh or other shells
  • Read Which shell are you using? if you are not sure
  • Warning: the shell in a (batch) script may be different from your interactive shell!
• which version of CDAT you want to use…

Note: if you get an error when importing cdms2 or vcs it means that either you have forgotten to initialize CDAT, or that something went wrong during the initialization. In both cases, you are either still using the default python installed on your system, or another (non-CDAT) python distribution!

$ python -c 'import cdms2, vcs'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: No module named cdms2

# Am I using the correct python distribution?
$ which python
/usr/bin/python

Note: if you need to maintain a local version on your machine, you can read Installing and maintaining CDAT with conda and the versions specific notes.

You should add the following line at the end of your ~/.bashrc configuration file (create the ~/.bashrc file if it does not exist yet). This will basically make sure that your shell can find the conda executable it needs for initializing CDAT, and define a wp alias that you can use to find where the python executable is located

Note: when you add the specified line to your bashrc file, you will then always get the python supplied by conda, instead of the system's python, when you use bash. If you don't like this permanent behavior, just type source ~jyp_login_on_this_server/.conda_jyp.sh in the bash shell where you want to use conda+CDAT, rather than adding it to the .bashrc file…

1. If your shell is not bash, start a bash shell by typing bash

• You can later go back to your previous shell by typing exit or CTRL-D

1. Activate the conda environment that uses the version of UV-CDAT you want to use

• source activate conda_env_name
• Use the table below to determine which environments are available on which server

1. Use CDAT !

Enthought Canopy is a Scientific and Analytic Python Deployment with Integrated Analysis Environment provided by Enthought. It used to be called EPD (Enthought Python Distribution)

Canopy is available for Windows, Mac and Linux. You can download the free Canopy Express that will already provide many extensions. If you are entitled to use Canopy Academic, login from inside Canopy Express and download the extra modules you need

Anaconda is a distribution similar to canopy provided by Continuum Analytics.

Anaconda is available for Windows, Mac and Linux

Note: Anaconda provides and uses conda for its installation. Since UV-CDAT is now also installed and maintained with conda, you can read the Installing and maintaining UV-CDAT with conda page for more information, even if you are not going to use UV-CDAT

The ipython interpreter provides more options and commands than the standard python interpreter, but takes more time to load. If you are going to develop by starting and exiting the interpreter many times, it's faster to use the standard interpreter

Starting ipython: ipython

• Using IPython for interactive work
• Ipython-quick-ref-sheets

The ipython notebook is a way to interact with python (and other supported interpreted languages) inside a web browser. You can mix cells with python commands, cells with the output of the python commands (possibly graphics generated by the commands), and text (using some wiki-like rich text format). This interactive web page, aka notebook, can be saved in a my_notebook.ipynb file and re-used later (e.g. the notebook provided in the introduction to Python, part 1).

Starting the notebook server: ipython notebook

[ PMIP3 Wiki Home ] - [ Help! ] - [ Wiki syntax ]