Linux servers used by LSCE users

• You can do what you want on your local laptop/desktop (except get viruses)

• Be very careful when you use remote interactive servers shared with other users
  • Others should not even notice that you(r processes) are there!
    Yes, you can use computing resources cleanly and efficiently…
  • Note: notebook servers are shared servers!
  • If you have some heavy duty processing to do:
    • Use shared interactive servers only for quick tests (ideally with a subset of your huge dataset)
    • Use dedicated batch/computing clusters for heavy processing
      • heavy =
        • will run for a long time
        • and/or use a lot of CPU (one or many CPUs)
        • and/or use a lot of RAM
        • and/or do a lot of disk reading/writing,
        • …
      • resources you use in High Performance Computing centers will probably be deducted from a (shared) project

• Disk space (and backup space) is also an important resource!
  • When possible, avoid duplicating data, and run your computations near the data
    • e.g. use the mirror data stored in /bdd/ on the spirit[x] servers
  • Use scratch disks, when available, for temporary data and do not forget to remove temporary data
    • LSCE: /home/scratch01/$USER
    • spiritx: /scratchx/$USER
    • TGCC: $SCRATCH
  • Do not store temporary data, or replicated data, or data that can be easily re-created, on backed-up disks

• Ask your experienced colleagues when you are not sure about something (sometimes even if you are sure…)
  • you can get better domain specific answers from your colleagues than from chatbots
  • There are no stupid questions (especially for beginners)
  • There are always different ways to do things, some are much less efficient than others

• Learn how to monitor resources usage:
  • CPU and RAM: top
  • Disk: quota, du, df
  • Listing and killing processes: ps, kill
  • …

• Read (some of) the local documentation
  RTFM (Read The Fine Manual)
  • Some Linux related documents (by JYP)
    • Learn at least how to use the man(ual) command !!
  • LSCE documentation
  • IPSL Computing and Data Center documentation
  • TGCC documentation
  • Jean Zay @ IDRIS documentation
  • …

• We use ssh and ssh clients to connect securely to the servers we work with
  • Use the PuTTY ssh client to easily open a terminal on a remote server from a Windows computer

• Some computing resources can also be accessed with a web browser and a JupyterHub server

• Servers can usually only access data on local (in the same computer room) disks. If you need to work locally on data available on a remote server, you first have to download the remote data
  • Notable exceptions :
    • spiritx servers in Palaiseau campus can access spirit servers disks at Sorbonne Université, in read-only mode, and the other way around (details)
    • some of the data available in /bdd on the spirit[x] servers are actually distributed from HPC centers
  • Reminder: when possible, avoid duplicating data, and perform the data analysis near the (original) data
  • When possible, reduce the data before download, in order to transfer only the subset you need: e.g. select only the regions and variables you need, pre-compute spatial or temporal means, etc…
    You can easily reduce NetCDF data with:
    • cdo (Climate Data Operators)
    • NCO (netCDF Operators)
  • It is of course a good idea to document (in a simple text file) where the files come from, how they were pre-processed, …

• We use transfer programs based on the ssh protocol to move data securely
  • scp (secure copy): easily copy files and directories recursively
    • Use the WinSCP scp client to easily copy files to/from a Windows computer
  • rsync: mirror directories (more complex)

If you need to check that a (potentially big) file is identical to its remote version, you can first check that the file sizes are identical, and then check data integrity by comparing the checksum (with sha256sum or md5sum) of the original and copied files. It is more efficient and secure to compare this unique fingerprint than to transfer a file again (and use diff)

e.g. with a 14 Gb file on spirit:

$ cd /bdd/CMIP6/CMIP/IPSL/IPSL-CM6A-LR/piControl/r1i1p1f1/Omon/thetao/gn/latest

$ ls -l thetao_Omon_IPSL-CM6A-LR_piControl_r1i1p1f1_gn_185001-194912.nc
[...] 14293466154 [...] thetao_Omon_IPSL-CM6A-LR_piControl_r1i1p1f1_gn_185001-194912.nc

$ ls -lh thetao_Omon_IPSL-CM6A-LR_piControl_r1i1p1f1_gn_185001-194912.nc
[...] 14G [...] thetao_Omon_IPSL-CM6A-LR_piControl_r1i1p1f1_gn_185001-194912.nc

$ md5sum thetao_Omon_IPSL-CM6A-LR_piControl_r1i1p1f1_gn_185001-194912.nc
5378634e97136d75e4517cb72e0cda9a  thetao_Omon_IPSL-CM6A-LR_piControl_r1i1p1f1_gn_185001-194912.nc

$ sha256sum thetao_Omon_IPSL-CM6A-LR_piControl_r1i1p1f1_gn_185001-194912.nc
90cf48c540216897bf4d9841acd7034f85cc10e233eed74b1aa27fd77a8da1e6  thetao_Omon_IPSL-CM6A-LR_piControl_r1i1p1f1_gn_185001-194912.nc

ssh1.lsce.ipsl.fr

• Use ssh ssh1.lsce.ipsl.fr when you are outside the LSCE wired network, before accessing any other LSCE server:
  • By outside LSCE, we mean that your computer is:
    • connected to the LSCE WiFi network (instead of the wired network)
    • or physically outside LSCE
• You can also use ssh1 when you are copying files to/from outside LSCE with scp or ssh-based tools
• The ssh1 gateway can only be used for copying files or accessing the LSCE interactive servers!
  • You have to connect to an obelixNN interactive server before doing any real work

obelixNN (obelix2 to obelix5)

• Use ssh obelix to access the LSCE servers from inside LSCE.
  The load balancing system will send you to the obelixNN server that has currently the smallest load.
  • By inside LSCE, we mean that:
    • your computer is connected to the LSCE wired network
    • or you are using the LSCE VPN
    • or you have a terminal connected to the ssh1 gateway
• Never forget that you are sharing these interactive servers with other users!
  • Do not use too much CPU and/or memory for a long time. Heavy computation should be done on the LSCE cluster
  • Learn how to use the top command to determine the current load of a server and the CPU/memory usage of your processes !
• You can use one of the following commands if you want to know on which obelixNN you are

  • $ ssh obelix
    Last login: Mon Jun  3 08:49:53 2019 from somewhere
    $ echo $HOST
    obelix5
    $ hostname
    obelix5

• In a python script, you can use the following to get some information

  • >>> import os
    >>> os.getlogin(), os.getenv('HOST')
    ('mylogin', 'obelix5')

• The LSCE cluster (aka the batch system)
  Use this cluster for heavy duty programs, rather than killing the multi-users interactive servers
  • Heavy duty programs are programs that will prevent other users from working on the same server because these programs use a lot of CPU and/or memory for more than a few minutes. Ask your advisor, if you are not sure

• Read the Using JupyterHub servers section of the Working with Python page

• The IPSL Computing and Data Center, aka Mésocentre ESPRI (Spirit / SpiritX / HAL)
  • Note: the Spirit/X clusters have replaced the ciclad/climserv cluster (in 2022)
  • You will need a specific account (different from your LSCE account) to access these servers and use data files that are already available/mirrored there (e.g. CMIPn data).
    Do not duplicate at LSCE data already available on IPSL servers and that you can process there!
  • Requesting an IPSL account
  • IPSL ESPRI Mesocenter documentation
    • spirit clusters
  • Getting help

[ PMIP3 Wiki Home ] - [ Help! ] - [ Wiki syntax ]