Getting started in CLIM & ESTIMR

Welcome to the CLIM & ESTIMR teams! You are probably reading this page because you are a new intern student, visitor, etc… This page will help you survive your first steps at LSCE, at least survive the computers… Reading this page will save your life (and a lot of time)!

Using your own computer at LSCE?

No need to try to connect your laptop to the LSCE wire network, it will not work!

The best you can do is use the eduroam or guest wifi network, if your office is close enough to one of the access points.

Warning! You can't connect directly to the LSCE servers from the wifi network. If you need to do that, you will have to connect to a server outside LSCE first, and then back to LSCE…

Accessing the LSCE intranet

Web site: https://intranet.lsce.ipsl.fr/

There is a lot of information available on the LSCE intranet, but you can only access it from a computer connected to the LSCE network

Everything about the LSCE servers (and other software and hardware information): https://intranet.lsce.ipsl.fr/informatique/en/orme/index.php

Reading the mails with the webmail or Thunderbird

As soon as you have a LSCE e-mail address, make sure it is added to the clim or estimr mailing list

Printers

Accessing the CEA saclay intranet

Web site: http://www-saclay.cea.fr/

You can only access it from the LSCE and CEA network! This intranet is the place where you can get information about:

CEA buses

Security training registration

Setting up your desktop computer

Everybody working for CLIM & ESTIMR gets a computer that can be connected to the LSCE network. Please take the time to read the instructions below, that may help make your life easier

Windows 7 computer

Read the notes (and follow the instructions) in Windows 7 notes, especially the Basic Windows 7 Configuration section

Getting administrator's rights

Make sure you know what you are doing and that you are not executing a virus asking you suspicious access rights!

If an application requests administrator's rights to install something or make changes to the computer, and you are sure it's not a virus, use the .\adminuser account and the adminuser password you got when your computer was configured, or ask the system administrators.

Linux computer

There are different window managers available, each one with different settings: you should the Cinnamon window manager (xfce is a bit too basic, and Gnome is not very convenient for doing actual work, and uses too much CPU). You can select the window manager by clicking on the little cogwheel below the password field, on the login screen.

This is Linux, you are using it for work, you should be efficient, so you should be able to do most of your work by typing commands in a terminal. The true power is in the correct use of the command-line interfaces, rather than having to spend time finding where to click in the windows. You will find some Linux documentation below.

Using the root acount

Do not use the root account or privileges if you don't know exactly what you are doing!

Depending on what you need to do:

run a single command with the root access rights
sudo command

become root in a terminal
sudo su -

use the root password when a program asks for it. Be sure the programs have legitimate reasons to ask for the root password!

Mac

Sorry, you are on your own, but it's soooo easy to use, right?

Accessing the Linux servers from your LSCE deskop

Linux or mac desktop

Just open a terminal and use ssh to connect to the appropriate server

Windows desktop

There should be an Asterix shortcut on the left side of your desktop. Double-click on it in order to open a window environment on one of the asterix servers. Each terminal you will open in this environment will run on the same asterix server. Use ssh to connect to an obelix server if you need to run something that will require more computing resources

Which Linux servers should you use?

There are lots of things you can do directly on your local Win7 or Linux desktop (displaying pdf, images, using a text editor, …), rather than directly on the servers. Never forget that your local desktop has access to the remote disks on the Linux servers.

Otherwise, use the appropriate Linux servers. The load balancing system will send you to the server that has currently the smallest load

asterixN (ssh asterix, or ssh asterix.lscelb): use these servers for the usual day-to-day work (things that use less than 1 Gb, don't use 100% CPU for a long time)

obelixN (ssh obelix, or ssh obelix.lscelb): use these servers for interactive processes that will need more ram and/or CPU during a longer time

the batch system for really heavy duty programs

Determining the load of a Linux server

If the Linux server you are working on seems slow, or if you want to get an idea of the resources you are using, you should use the top command

jypeter@obelix4 - ...jypeter - 46 >top
top - 17:48:51 up 8 days, 23:43, 29 users, load average: 1.22, 1.38, 1.98
Tasks: 324 total, 2 running, 320 sleeping, 2 stopped, 0 zombie
Cpu(s): 12.6%us, 0.0%sy, 0.0%ni, 87.4%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 32877756k total, 14706188k used, 18171568k free, 21372k buffers
Swap: 33554428k total, 1628972k used, 31925456k free, 14105940k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
10193 mpeau 20 0 302m 4336 2272 R 100.0 0.0 7033:08 emacs
16504 mpeau 20 0 110m 2008 1228 S 0.7 0.0 0:00.13 tcsh
26344 dzhu 20 0 1635m 20m 3880 S 0.3 0.1 0:44.90 ipython
1 root 20 0 21448 1108 892 S 0.0 0.0 0:01.61 init
[...]

The top lines give you some summary information about the system, but you should monitor the memory usage (VIRT and RES), CPU and TIME columns. By default, top will will put the processes using the more CPU at the top (as shown above). You can see above that the emacs text editor has clearly crashed, because it should not use 100% CPU for such a long time

If you type M, the processes will be sorted my memory usage, as shown below

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
16092 jchang 20 0 8488m 8.1g 7912 R 99.6 25.9 1:41.38 idl
23256 pvait 20 0 232m 70m 2632 S 0.0 0.2 0:01.36 R
26344 dzhu 20 0 1635m 20m 3880 S 0.0 0.1 0:44.87 ipython
10619 astege 20 0 954m 9292 2664 S 0.0 0.0 0:40.67 gedit
1362 nslcd 20 0 442m 6056 2208 S 0.0 0.0 2:13.98 nslcd
16238 twang 20 0 128m 5068 2076 S 0.0 0.0 0:00.05 gconfd-2
10193 mpeau 20 0 302m 4336 2272 R 99.9 0.0 7024:13 emacs
[...]

Other useful keys:

q	Quit
c	Display the command line options of the running processes
u	List only the processes of a specific user

Which disks should you use?

It is very important to use the correct directories and make it easy for your advisor to find things after you have left Do not forget to clean your directories before you leave (ask your advisor what should be kept or cleaned)

Accessing remote disks

It is possible to access directly all the disks on the LSCE servers from your local Linux or Windows desktop

Example path on the LSCE Linux server: /home/scratch01/your_lsce_login

Accessing the same path from a local desktop:

Linux desktop: /home/scratch01/your_lsce_login

Windows desktop: \\dfshost\dfs\scratch01\your_lsce_login

The home directory

Location: the location depends on the machine you are connected to in the current terminal

On the LSCE servers (asterixN, obelixN): /home/users/your_lsce_login

On a local Linux desktop (lsceNNNN): /homel/your_lsce_login

The configuration files of your Linux account are stored in this directory (use ls -a to see hidden files and directories). Ask your advisor if you should copy somebody else's configuration files

tcsh shell users: .cshrc and .login

bash shell users: .profile and .bashrc

emacs text editor: .emacs

Backup of the home directory on the LSCE servers: YES! Every night

Put the most important sources/scripts/data/papers in this directory

Do not put temporary files, or big data files that are available somewhere else in this directory!

Your home directory will also be archived when your account is closed

Quota: YES!
There is a limit to what you can store in your home directory. If you reach this limit, you may experience some seemingly random errors (can't connect to the servers, the new files have a 0 size, …)

> quota -s
Disk quotas for user jypeter (uid 6369):
Filesystem blocks quota limit grace files quota limit grace
prolix3:/users/ 4658M 4883M 4981M 23503 0 0

The scratch directory

Location: /home/scratch01/your_lsce_login

Backup: NO

Use this directory for temporary and big data files

The files older than 6 months (and possibly older than 1 month) will be automatically deleted if there is not enough free space. See the Scratch area section for more details

Quota: NO

Project specific directories

Location: /home/some_project_name/your_lsce_login

Ask your advisor if you should use a specific project directory and the technical details about it

Use this directory for storing the (big) data files that you can't store in the home directory, and should not store in the scratch directory

Backup: ask your advisor

Quota: ask your advisor

Useful documentation for beginners

Linux/UNIX

UNIX_doc_JYP.pdf: some basic Linux documentation. Make sure you read it!

A list of the most useful commands

A tutorial

Useful Linux commands for working with (potentially big) text files (houchesa4_updated.pdf)

Using ssh on Linux and Windows

An annotated reference card of the emacs text editor (emacs_jyp.pdf)

You can also check the Les fondamentaux du pôle de modélisation du climat de l'IPSL wiki page

NetCDF and file formats

What are binary (binaire_jyp.pdf) and NetCDF (netcdf_jyp.pdf) files?

Network Common Data Form (NetCDF) official page

The different netCDF formats (and do not forget to use ncdump -k data.nc if you need to know the format)

Climate Data Operators for manipulating NetCDF files (and some other formats)

netCDF Operator (NCO) for manipulating NetCDF files

NetCDF CF (Climate and Forecast) Metadata Conventions: this will help you understand the information you get when you do a ncdump -h some_climate_data_file.nc

Using Python to work with NetCDF data:

Using the cdms2 module if you work with the UV-CDAT distribution

Using the netCDF4 module

Getting help from the LSCE system administrators

Send a mail to: help-lsce@lsce.ipsl.fr

Accessing the LSCE servers from outside the LSCE

The only way to access the LSCE servers from outside LSCE is to connect first to the idefix1.extra.cea.fr gateway server. Unless you are on a machine that is outside LSCE, but still on the CEA network (e.g. one of the machines of the CEA computing center)

Send a mail to help-lsce@lsce.ipsl.fr, and request an access to the idefix1 server

Use ssh to connect to idefix1

From a Linux/mac desktop: ssh -A -X your_lsce_login@idefix1.extra.cea.fr

From a Windows desktop:

Text only: use Putty

Text & graphics. This is more complex to set up, and you need to use one of the options below:

use Putty and cygwin/X

use VirtualBox

There is only a limited number of programs that you can run on idefix1, so you have to go from idefix1 to asterix, obelix or curie with on of the ssh commands below:

ssh asterix.lscelb

ssh obelix.lscelb

ssh your_tgcc_login@curie-ccrt.ccc.cea.fr

It's possible to use only one line to to both ssh commands

asterix: ssh -A -X your_lsce_login@idefix1.extra.cea.fr -t ssh -A asterix.lscelb

obelix: ssh -A -X your_lsce_login@idefix1.extra.cea.fr -t ssh -A obelix.lscelb

TGCC: ssh -A -X your_lsce_login@idefix1.extra.cea.fr -t ssh -A your_tgcc_login@curie-ccrt.ccc.cea.fr

You can define an alias in order to connect more easily to the machines:

tcsh users: add the following line to your ~/.cshrc configuration file
alias iasterix 'ssh -A -X your_lsce_login@idefix1.extra.cea.fr -t ssh -A asterix.lscelb'

bash users:

You should create and configure an ssh key, if you don't want to type your password each time you use ssh or scp: https://intranet.lsce.ipsl.fr/informatique/orme/ssh.php

If you want more information about ssh

Linux users: http://ciclad-web.ipsl.jussieu.fr/data/documents/ssh-utilisateurs-unix.pdf

Windows users: http://ciclad-web.ipsl.jussieu.fr/data/documents/ssh_putty_v2_2011-1.pdf

Copying files to/from LSCE

If you are on a computer outside LSCE and you need to transfer a file to/from LSCE, you need to make the transfer between your current machine and idefix1!

Example from a Linux/mac desktop: scp -p file_outside_lsce your_lsce_login@idefix1.extra.cea.fr:/home/scratch01/your_lsce_login/

From a Windows desktop: use winscp

What next?

Ask you advisor, who has probably already told you everything that was listed above, so you did not really learn anything new by reading this page

[ PMIP3 Wiki Home ] - [ Help! ] - [ Wiki syntax ]