Getting started in CLIM & ESTIMR

Welcome to the CLIM & ESTIMR teams! You are probably reading this page because you are a new intern student, visitor, etc... This page will help you survive your first steps at LSCE, at least survive the computers... Reading this page will save your life (and a lot of time)!

Miscellaneous information

A few useful links that have nothing to do with science and software resources!

LSCE-related public transport

Adding money online to your restaurant account

Survival kit and other information for CDD and interns (LSCE intranet)

CEA Saclay International Office Our multilingual staff is readily available to assist you through immigration procedures and to help you settle in France

e-International Welcome Office Paris-Saclay university help to prepare for your arrival in France and help you settle down in the best conditions

Trouver un logement dans une résidence universitaire (in French... Finding a room in a student dorm)

Science Accueil

Guide du Personnel UVSQ

Phone directory, and map of the building

Using existing information

Phone directory (intranet)

Dial 2 + NNNN (last 4 digits) to call somebody at LSCE (or CEA Saclay) from LSCE

701 and 714 map (intranet)

Updating the directory/map information

It's very important that you update your phone/office information when you arrive at LSCE, so that other people can find you using the tools above!

Finding your phone number

Use the following sequence of keys on the Mitel phone in your office:

Select MENU (use the downward arrow)

Select PARAMETRES (use the downward arrow)

Select GENERAL (use the downward arrow)

Select MON NUMERO

This will display your 2 + NNNN phone number

You can be called from outside with 01 69 08 NN NN

Updating your entry in the directory

Find your name in the phone directory

Click on Modifier mes coordonnées (in blue on the right side)

Click on Informations dans l'annuaire and update the different fields

Click on Envoyez votre demande

The LSCE mailing lists

It is important to join the correct mailing lists, if you want to get all the information you need, on time.

The 'all' list

When your account is created, your email address should automatically be added to the the all list, that can be used to send mails to all the people working at LSCE. You will also get IPSL information through this list

all@lsce.ipsl.fr	Everybody at LSCE
lsce-orme@lsce.ipsl.fr	People working at Orme
lsce-gif@lsce.ipsl.fr	People working in Gif

Mails sent to these lists will reach lots of people. Think well before sending a mail to these lists!

The 'clim' and 'estimr' lists

Ask your team leader or Jean-Yves to add you to your team's mailing list

List	Team	Archive
clim@lists.lsce.ipsl.fr	clim	https://lists.lsce.ipsl.fr/sympa/arc/clim
estimr@lists.lsce.ipsl.fr	estimr	https://lists.lsce.ipsl.fr/sympa/arc/estimr

Notes

You have to be identified on the server to access most lists's details. Enter you login and email password in the email address and pasword fields at the top right of any mailing list web page

You may want/need to join other mailing lists (project specific, sports, ...). Check the list of lists to find about all the existing lists

It is possible to use mails to interact with the LSCE lists server, rather than the web interface

Using your own computer at LSCE?

No need to try to connect your personal laptop to the LSCE wire network, it will not work!

The best you can do is use the Guest WiFi or eduroam network, if your office is close enough to one of the access points.

Warning! You can't connect directly to the obelix LSCE servers from the wifi network. You first have to connect to the ssh1.lsce.ipsl.fr gateway server, and then to obelix. More details in Accessing the LSCE servers from outside the LSCE

Accessing the LSCE intranet

If you try to access the intranet from outside the LSCE network (including eduroam), your browser will open a login/password popup window. Use your LSCE login (not your email address!) and password
Intranet web site: https://intranet.lsce.ipsl.fr/

There is a lot of information available on the LSCE intranet, but you can only access it from a computer connected to the LSCE network

Everything about the LSCE servers (and other software and hardware information): https://intranet.lsce.ipsl.fr/informatique/en/orme/index.php

Reading the mails with the webmail or Thunderbird

As soon as you have a LSCE e-mail address, make sure it is added to the clim or estimr mailing list

Printers

Accessing the CEA Saclay intranet and other CEA web sites with vpnssl

It used to be possible to access the CEA Saclay intranet by just clicking on http://www-saclay.cea.fr/ from any computer on the LSCE network. This is not possible anymore, now that LSCE is on the Paris-Saclay network (and not on the CEA network)
Use the following steps:

Start Chrome (or try another browser)

Open the following link: https://vpnssl.cea.fr/

ignore the security errors

this link will work from the Paris Saclay network, and ANY other location (e.g. home) OUTSIDE of CEA

Use your initials and badge number as the login (e.g. “James Bond” => “jb007007”) and the number supplied by the activIDentity keyring, followed by your 4 digits personal code, as your password

Select the CEA web site you want from the “Aventail WorkPlace” portal

Espace Sigma: that's where you specify when you will be on holidays

Intranet Saclay: where you can get information about the CEA buses, CEA directory, security training registration and information, practical information, etc...

Intranet du voyageur: if you need to make a plane/train reservation

...

Click on logout when you are done

Setting up your desktop computer

Everybody working for CLIM & ESTIMR gets a computer that can be connected to the LSCE network. Please take the time to read the instructions below, that may help make your life easier

Windows computer

Read the Windows 10 notes, especially the Configuring Windows 10 section

Note: there are also some Windows 7 notes, but you are supposed to use Windows 10 since January 2020!

Getting administrator's rights

Make sure you know what you are doing and that you are not executing a virus asking you suspicious access rights!

If an application requests administrator's rights to install something or make changes to the computer, and you are sure it's not a virus, use the .\admin local account and the admin password you received when your computer was configured, or ask the system administrators.

Linux computer

There are different window managers available, each one with different settings: you should the Cinnamon window manager (xfce is a bit too basic, and Gnome is not very convenient for doing actual work, and uses too much CPU). You can select the window manager by clicking on the little cogwheel below the password field, on the login screen.

This is Linux, you are using it for work, you should be efficient, so you should be able to do most of your work by typing commands in a terminal. The true power is in the correct use of the command-line interfaces, rather than having to spend time finding where to click in the windows. You will find some Linux documentation below.

Using the root acount

Do not use the root account or privileges if you don't know exactly what you are doing!

Depending on what you need to do:

run a single command with the root access rights
sudo command

become root in a terminal
sudo su -

use the root password when a program asks for it. Be sure the programs have legitimate reasons to ask for the root password!

Mac

Sorry, you are on your own, but it's soooo easy to use, right?

Accessing the Windows cluster from a Linux computer

Follow the instructions in the Windows servers page. If you need to use a program that is missing on this cluster, see the Help section

Accessing the LSCE Linux servers

You have to Use ssh to connect to the servers

If you are really in a hurry, go directly to the Connecting to the LSCE servers, IPSL servers, TGCC, ... section, but it is important that you read the full ssh documentation page at least once!

Which Linux servers should you use?

Working directly on your desktop

Always remember that your local LSCE desktop can access remote disks on the LSCE servers.

There are lots of things you can do directly on your local Windows 10 or Linux desktop (displaying pdf, images, using a text editor, ...), rather than on the remote servers. For example, if a script running on the servers generates a pdf file, it is more efficient to open this pdf file using Acrobat/evince on your desktop, than by using evince on the server.

Available servers

The LSCE interactive servers: obelixNN. Use ssh obelix to access these servers, and the load balancing system will send you to the server that has currently the smallest load.
Never forget that you are sharing these servers with other users! Do not use too much CPU and/or memory for a long time. Heavy computation should be done on the obelix cluster

$ ssh obelix
Last login: Mon Jun 3 08:49:53 2019 from somewhere
# You can use one of the following commands if you want to know on which obelix you are
$ echo $HOST
obelix5
$ hostname
obelix5

The LSCE cluster (aka the batch system). Use this for really heavy duty programs

The ciclad and climserv servers. In some cases, you may need an account to use the IPSL/ESPRI clusters, especially if you need to use data files that are already available/mirrored there (e.g. CMIPn data), rather than copying the data to LSCE

Requesting a new account

The asterixN servers: some wise LSCE elders may mention these servers, but they don't exist any more!

Which shell are you using?

It is important to know which shell you are using if you need to configure and tune your Linux account. The shell is basically the program that waits for you to type commands in a terminal and passes them to the computer

If you are not sure which shell is running in your terminals, you can use echo $SHELL to find out:

# Somewhere with tcsh
> echo $SHELL
/bin/tcsh
# Somewhere else with bash
$ echo $SHELL
/bin/bash

Determining the load of a Linux server

If the Linux server you are working on seems slow, or if you want to get an idea of the resources you are using, you should use the top command

>hostname
obelix3
>top
top - 17:48:51 up 8 days, 23:43, 29 users, load average: 1.22, 1.38, 1.98
Tasks: 324 total, 2 running, 320 sleeping, 2 stopped, 0 zombie
Cpu(s): 12.6%us, 0.0%sy, 0.0%ni, 87.4%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 32877756k total, 14706188k used, 18171568k free, 21372k buffers
Swap: 33554428k total, 1628972k used, 31925456k free, 14105940k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
10193 john 20 0 302m 4336 2272 R 100.0 0.0 7033:08 emacs
16504 john 20 0 110m 2008 1228 S 0.7 0.0 0:00.13 tcsh
26344 jane 20 0 1635m 20m 3880 S 0.3 0.1 0:44.90 ipython
1 root 20 0 21448 1108 892 S 0.0 0.0 0:01.61 init
[...]

The top lines give you some summary information about the system, but you should monitor the memory usage (VIRT and RES), CPU and TIME columns. By default, top will will put the processes using the more CPU at the top (as shown above). You can see above that the emacs text editor has clearly crashed, because it should not use 100% CPU for such a long time

If you type M, the processes will be sorted my memory usage, as shown below

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
16092 bob 20 0 8488m 8.1g 7912 R 99.6 25.9 1:41.38 idl
23256 john 20 0 232m 70m 2632 S 0.0 0.2 0:01.36 R
26344 jane 20 0 1635m 20m 3880 S 0.0 0.1 0:44.87 ipython
10619 bob 20 0 954m 9292 2664 S 0.0 0.0 0:40.67 gedit
1362 nslcd 20 0 442m 6056 2208 S 0.0 0.0 2:13.98 nslcd
10193 john 20 0 302m 4336 2272 R 99.9 0.0 7024:13 emacs
[...]

Other useful keys:

q	Quit
c	Display the command line options of the running processes
u	List only the processes of a specific user

Which disks should you use?

It is very important to use the correct directories and make it easy for your advisor to find things after you have left Put the source codes, scripts and papers you are writing on a backed up disk The backed up disk at LSCE is home directory You should have a few Gb available on home Use the quota -s command to find out the exact amount Put temporary files on a scratch disk (scratch01 at LSCE) Do NOT put (big) data files on a backed up disk Do not forget to clean your directories before you leave (ask your advisor what should be kept or cleaned)

Accessing remote disks

It is possible to access directly all the disks on the LSCE servers from your local Linux or Windows computer
(local = a computer on the LSCE ethernet/wired network)

Example path on the LSCE Linux server: /home/scratch01/your_lsce_login

Accessing the path above from a local computer:

Linux computer: /home/scratch01/your_lsce_login

Windows computer: \\dfshost\dfs\scratch01\your_lsce_login

The home directory

Location: the location depends on the machine you are connected to in the current terminal

On the LSCE servers (obelixN): /home/users/your_lsce_login

Non LSCE servers:

TGCC: /ccc/cont003/home/dsm/your_tgcc_login
You can find out the location of all your directories with ccc_home -a (use ccc_home -h to find out all the available options)

ciclad: /home/your_ciclad_login

On a local Linux desktop (lsceNNNN): /homel/your_lsce_login

The configuration files of your Linux account are stored in the home directory (use ls -a to see hidden files and directories). Ask your advisor if you should copy somebody else's configuration files

tcsh shell users: .cshrc and .login

bash shell users: .profile and .bashrc

emacs text editor: .emacs

You can go to your home directory by typing cd (no arguments),
and to the home directory of somebody else with cd ~login_name

At LSCE, you can't read the content of somebody else's home directory by default

Backup of the home directory on the LSCE servers: YES! Every night

More details on the LSCE backup system page

Put the most important sources/scripts/data/papers in this directory

Do not use the home directory to store:

temporary files (use scratch01)

big (or many) data files (use scratch01 or a project specific disk)

Your home directory will be archived when your account is closed, so that you or your advisor can access it later, if required

Backup of the TGCC home directory: YES!

Use the following command to access the backup: cd ~/.snapshot

Quota: YES!
There is a limit to what you can store in your home directory. If you reach this limit, you may experience some seemingly random errors (can't connect to the servers, the new files have a 0 size, ...)
The example below shows how to determine the quota on the LSCE disks: you can never go above the size displayed in the limit column, and you cannot stay more than 7 days above the limit displayed in the quota column! The grace column will display either None (you cannot write on the disk anymore, unless you get below the quota limit) or N days (you can use the disk, but you have N days to go below the quota limit)

> quota -s
Disk quotas for user johndoe (uid 6369):
Filesystem blocks quota limit grace files quota limit grace
prolix3:/users/ 4658M 4883M 4981M 23503 0 0

The scratch directory

Do not put anything in the scratch directory that you can't recreate with a program, or copy/download from somewhere else!

Location: /home/scratch01/your_lsce_login

Backup: NO

Use this directory for temporary and big data files

The files older than 6 months (and possibly older than 1 month) will be automatically deleted if there is not enough free space. See the Scratch area section for more details on the retention policy

In other words, the files are guaranteed to stay 1 month and possibly longer

Quota: NO

Project specific directories

The data on these disks is probably not backed up... You are not protected against file deletion by mistake, or a disk crash (a major disk crash is not likely, but can happen) You should store your source codes, scripts, notebooks, manuscripts, reports, figures, on a backed up directory, and only store on the project disks data downloaded from outside, or that you can regenerate with the backed up scripts

Location: /home/some_project_name/your_lsce_login

Ask your advisor if you should use a specific project directory and the technical details about it

Use this directory for storing the (big) data files that you can't store safely in the scratch directory, and should not store in the home directory

Backup: ask your advisor

Quota: ask your advisor

Determining the space used by directories

Your Linux environment may sometimes stop working correctly because you have exceeded the allowed quota on your disks (check your quotas with quota -s). You can use the following command to get the size of all the directories and files in the current directory, sorted by size: du -sh * | sort -rh

>cd /home/scratch01/johndoe
/home/scratch01/johndoe
>du -sh * | sort -rh
58M GLAC1DHiceF26.nc
47M dashboard
44M GLAC1DdrainagePointerF26ka.nc
15M octcdf
12M 1901.nc
3.2M cmake
...

Which programs should you use?

Your advisor will let you know which (version of which) programs you should use. Some programs are available by default, and you have to use the module command to access other programs.

Which program am I using?

At any time, you can use the which command to determine where a program is located. The option to get the version information of a program may vary (launch the program without arguments, with -v or --v, ...).

> which ncdump
/usr/bin/ncdump
> ncdump
ncdump [-c|-h] [-v ...] [[-b|-f] [c|f]] [-l len] [-n name] [-p n[,n]] [-k] [-x] [-s] [-t|-i] [-g ...] [-w] file
[...]
netcdf library version 4.3.3.1 of Dec 10 2015 16:44:18 $
> which python
/usr/bin/python
> python --version
Python 2.7.5

Using module to access optional programs

Main module options:

module list: list the currently loaded modules

module avail: list the available modules

module load module_a ... module_n: load modules (automatically including dependencies)

module load module_a: load default version of module_a

module load module_a/vvv: load vvv version of module_a

module purge: remove all modules

Detailed example:

> which ncview
ncview: Command not found.
> module list
No Modulefiles Currently Loaded.
> module avail
[...]
batch_env grib_api/1.14.0 netcdf/4 python/3.6
[...]
glost/0.3.1 ncview/2.1.7 python/2.7
> module load ncview netcdf/4
> module list
Currently Loaded Modulefiles:
1) ncview/2.1.7 2) netcdf/4
> which ncview
/usr/local/install/ncview-2.1.7/bin/ncview
> which ncdump
/usr/local/install/netcdf-4.3.2/bin/ncdump

Useful documentation for beginners

Linux/UNIX

UNIX_doc_JYP.pdf: some basic Linux documentation. Make sure you read it!

A list of the most useful commands

A tutorial

Useful Linux commands for working with (potentially big) text files (houchesa4_updated.pdf)

Using ssh on Linux and Windows

An annotated reference card of the emacs text editor (emacs_jyp.pdf)

JYP's .emacs file: for Linux - for Windows, with emacs for Windows

You can also check the Les fondamentaux du pôle de modélisation du climat de l'IPSL wiki page

NetCDF and file formats

What are binary (binaire_jyp.pdf) and NetCDF (netcdf_jyp.pdf) files?

Network Common Data Form (NetCDF) official page

The different netCDF formats (and do not forget to use ncdump -k data.nc if you need to know the format)

The NetCDF file format is self-documented, and the metadata of climate data files often follows the CF (Climate and Forecast) Metadata Conventions

The files in the CMIPn (n ≥ 5) database are written with the CMOR 3 library in order to make sure that they follow exactly the requested standards

You can use Panoply and ncview for visualizing quickly the NetCDF files

Climate Data Operators for manipulating NetCDF files (and some other formats)

netCDF Operator (NCO) for manipulating NetCDF files

NetCDF CF (Climate and Forecast) Metadata Conventions: this will help you understand the information you get when you do a ncdump -h some_climate_data_file.nc

Using Python to work with NetCDF data:

Using the cdms2 module if you work with the UV-CDAT distribution

Using the netCDF4 module

Using the Python language

Working with Python

JYP's recommended steps for learning python

Getting help (from the LSCE system administrators)

Send a mail to help-lsce

Describing your problem

When you need help from the administrators or other people, it will save a lot of time if you describe your problem as accurately as possible (do not just report XXXXX is not working as expected !)

You can copy-paste the error message(s). Finding the error log file(s) of the program/system (when available), or using the verbose mode of a program will make the problem resolution much easier and faster. The example below shows how to use the verbose mode of ssh

>ssh -v obelix
OpenSSH_7.4p1, OpenSSL 1.0.2k-fips 26 Jan 2017
debug1: Reading configuration data /home/users/johndoe/.ssh/config
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 58: Applying options for *
debug1: Connecting to obelix [157.136.72.75] port 22.
debug1: Connection established.
debug1: key_load_public: No such file or directory
debug1: identity file /home/users/johndoe/.ssh/id_rsa type -1
[...]
debug1: Entering interactive session.
debug1: pledge: exec
debug1: client_input_global_request: rtype hostkeys-00@openssh.com want_reply 0
debug1: Requesting X11 forwarding with authentication spoofing.
debug1: Sending environment.
debug1: Sending env LANG = en_US
Last login: Thu Mar 1 15:52:44 2018 from lsce4078.lsce.ipsl.fr
>

Taking a screenshot

Sometimes, it may also help to send a picture of what is wrong!

On Linux, you should find a screenshot application in the tools available on the system. See the example below for people using the xfce windows manager Click to get a larger version

On Windows, you can use the Capture screen option of XnView Click to get a larger version

What next?

Ask you advisor, who has probably already told you everything that was listed above, so you did not really learn anything new by reading this page

[ PMIP3 Wiki Home ] - [ Help! ] - [ Wiki syntax ]