Differences

This shows you the differences between two versions of the page.

--- other:newppl:starting [2022/11/23 16:54]
jypeter [Text editors] Added xkcd credit for emacs cartoon
+++ other:newppl:starting [2024/04/03 11:33]
jypeter [Available servers] Added getlogin and getenv Python example
@@ Line 13: / Line 13: @@
   * [[other:newppl:riealgo|Adding money online to your restaurant account]]
   * [[https://intranet.lsce.ipsl.fr/Phocea/Page/index.php?id=97|Survival kit and other information for CDD and interns]] (LSCE intranet)
-  * [[http://internationaloffice.ceasaclay.com/?lang=en|CEA Saclay International Office]] //Our multilingual staff is readily available to assist you through immigration procedures and to help you settle in France//
+  * [[https://www.universite-paris-saclay.fr/en/campus-life/international-welcome-desk|International welcome desk]] //Université Paris-Saclay supports international students, PhD students and researchers as soon as they arrive in France to ensure their stay take place under the best conditions//
-  * [[https://www.universite-paris-saclay.fr/en/campus-life/international-welcome-desk|International welcome desk]] //Université Paris-Saclay provides you with a variety of services and documentation to welcome international students and researchers under the best conditions//
   * [[https://logement.campus-paris-saclay.fr/| Trouver un logement dans une résidence universitaire]] (in French... //Finding a room in a student dorm//)
   * [[https://www.science-accueil.org/en/|Science Accueil]]
@@ Line 25: / Line 24: @@
   * [[https://intranet.lsce.ipsl.fr/en/Phocea/Membres/Annuaire/index.php|Phone directory]] (intranet)
-    * Dial **2** + ''NNNN'' (last 4 digits) to call somebody at LSCE (or CEA Saclay) from LSCE
+    * Dial **2** + ''NNNN'' (last 4 digits) to **call somebody at LSCE** (or CEA Saclay) from LSCE
+    * Dial **0** + ''NNNNNNNNNNNNNNNNNNNN''  to **call somebody outside LSCE** from LSCE
+    * <wrap hi>Dial **18** to call FLS (CEA security & firemen)</wrap>
+      * [[https://intranet.lsce.ipsl.fr/Phocea/Page/index.php?id=312|Other security related phone numbers]]
+        * <wrap hi>Recommended</wrap>: add the FLS full phone number to your cell phone
   * [[https://intranet.lsce.ipsl.fr/PlanLabo/index.html|701 and 714 map]] (intranet)
@@ Line 106: / Line 109: @@
   * [[https://intranet.lsce.ipsl.fr/informatique/en/orme/imprimantes.php|Printers]]
-===== Accessing the CEA Saclay intranet and other CEA web sites with vpnssl =====
+===== Accessing the CEA Saclay intranet and other CEA web sites from a browser =====
 <WRAP center round important 60%>
-It used to be possible to access the CEA Saclay intranet by just clicking on http://www-saclay.cea.fr/ from any computer on the LSCE network.
+LSCE is on the Paris-Saclay network (and **not on the CEA network**)
-This is **not possible** anymore, now that LSCE is on the Paris-Saclay network (and not on the CEA network)
+It is therefore **not possible** to access the //CEA Saclay// intranet by just clicking on http://www-saclay.cea.fr/ from a computer on the LSCE wired network (or //eduroam//, or any network outside CEA)
 </WRAP>
 Use the following steps:
   - Start Chrome (or try another browser)
-  - Open the following link: https://vpnssl.cea.fr/
+    * it is even better if you use a [[https://support.google.com/chrome/answer/95464?hl=en|Private/Incognito window]] to do this
-    * ignore the security errors
+  - Open the following link: https://vpn.cea.fr/
+    * ignore the possible security/certificate errors
     * this link will work from the Paris Saclay network, and ANY other location (e.g. home) OUTSIDE of CEA
   - Use your initials and badge number as the **login** (e.g. "//James Bond//" => "**jb007007**") and the number supplied by the //activIDentity// keyring, followed by your 4 digits personal code, as your **password**
@@ Line 126: / Line 130: @@
     * ...
   - Click on //logout// when you are done
+    * ... and close your browser //Incognito// window if you used one
 ===== Setting up your desktop computer =====
@@ Line 190: / Line 195: @@
 ==== Available servers ====
-  * **The LSCE interactive servers**: ''obelix//NN//''\\ Use ''ssh obelix'' to access these servers, and the //load balancing// system will send you to the server that has currently the smallest load.
+  * **The LSCE //gateway//**: ''ssh1.lsce.ipsl.fr''
-    * <wrap hi>Never forget that you are sharing these servers with other users!</wrap>
+    * Use ''ssh ssh1.lsce.ipsl.fr'' when you are **outside the LSCE wired network**, before accessing any other LSCE server:
-    * <wrap hi>Do not use too much CPU and/or memory for a long time</wrap>. Heavy computation should be done on the [[https://intranet.lsce.ipsl.fr/informatique/en/calcul/batch.php|LSCE cluster]]
+      * By **//outside LSCE//**, we mean that your computer is:
+        * connected to the LSCE WiFi network (instead of the wired network)
+        * or physically outside LSCE
+    * You can also use ''ssh1'' when you are **copying files to/from outside LSCE** with ''scp'' or ssh-based tools
+    * <wrap hi>The ''ssh1'' gateway can **only** be used for copying files or accessing the LSCE interactive servers</wrap>!
+      * **You have to** connect to an ''obelix//NN//'' interactive server before doing any real work
+  * **The LSCE interactive servers**: ''obelix//NN//'' (''obelix2'' to ''obelix5'')
+    * Use ''ssh obelix'' to access the LSCE servers from **inside LSCE**.\\ The //load balancing// system will send you to the ''obelix//NN//'' server that has currently the smallest load.
+      * By **//inside LSCE//**, we mean that:
+        * your computer is connected to the LSCE **wired** network
+        * or you are using the [[https://intranet.lsce.ipsl.fr/informatique/fr/vpn-forticlient.php#vpnlsce|LSCE VPN]]
+        * or you have a //terminal// connected to the ''ssh1'' gateway
+    * <wrap hi>Never forget that you are **sharing** these interactive servers with other users!</wrap>
+      * <wrap hi>Do not use too much CPU and/or memory for a long time</wrap>. Heavy computation should be done on the [[https://intranet.lsce.ipsl.fr/informatique/en/calcul/batch.php|LSCE cluster]]
       * Learn how to [[other:newppl:starting#determining_the_load_of_a_linux_server|use the top command]] to determine the current load of a server and the CPU/memory usage of your processes !
-    * <code>$ ssh obelix
+    * You can use one of the following commands if you want to know on which ''obelixNN'' you are
+      * <code>$ ssh obelix
 Last login: Mon Jun  3 08:49:53 2019 from somewhere
-# You can use one of the following commands if you want to know on which obelix you are
 $ echo $HOST
 obelix5
 $ hostname
 obelix5</code>
+    * In a python script, you can use the following to get some information
+      * <code>>>> import os
+>>> os.getlogin(), os.getenv('HOST')
+('mylogin', 'obelix5')</code>
-  * **[[https://intranet.lsce.ipsl.fr/informatique/en/calcul/batch.php|The LSCE cluster]]** (aka the //batch system//)\\ Use this cluster for really heavy duty programs, rather than killing the multi-users interactive servers
+  * **[[https://intranet.lsce.ipsl.fr/informatique/en/calcul/batch.php|The LSCE cluster]]** (aka the //batch system//)\\ Use this cluster for **//heavy duty// programs**, rather than killing the multi-users interactive servers
+    * //Heavy duty// programs are programs that will prevent other users from working on the same server because these programs use a lot of CPU and/or memory for more than a few minutes. Ask your advisor, if you are not sure
-  * **The [[http://mesocentre.ipsl.fr/|IPSL/ESPRI Mesocenter servers/clusters]]** (''ciclad'' / ''climserv'' / etc...)\\ You will need an account (different from your //LSCE// account) to access these servers and use data files that are already available/mirrored there (e.g. **''CMIPn'' data**). **Do not duplicate** at LSCE data that is already available on IPSL servers and that you can process there!
+  * **The [[https://documentations.ipsl.fr/spirit/|IPSL Computing and Data Center]]**, aka //Mésocentre ESPRI// (''Spirit'' / ''SpiritX'' / ''HAL'')
-    * [[http://mesocentre.ipsl.fr/account-opening/|Requesting a new account]]
+    * Note: the ''Spirit''/''X'' clusters have replaced the ''ciclad''/''climserv'' cluster (in 2022)
-    * [[https://documentations.ipsl.fr/MESO_User/|IPSL ESPRI Mesocenter documentation]] (//English//)
+    * You will need a specific account (different from your //LSCE// account) to access these servers and use data files that are already available/mirrored there (e.g. **''CMIPn'' data**).\\ **Do not duplicate** at LSCE data  already available on IPSL servers and that you can process there!
-    * [[https://mesocentre.ipsl.fr/quick-start/|Utilisation du Mésocentre IPSL]] (//French//, the //English// documentation link above has more information)
+    * [[https://mesocentre.ipsl.fr/account-opening/|Requesting an IPSL account]]
+    * [[https://documentations.ipsl.fr/spirit/|IPSL ESPRI Mesocenter documentation]]
+      * [[https://documentations.ipsl.fr/spirit/spirit_clusters/head_nodes.html|spirit clusters]]
+    * [[https://documentations.ipsl.fr/spirit/getting_started/support.html|Getting help]]
   * **The ''asterix//NN//'' servers**\\ Some wise LSCE elders may mention these servers, but they don't exist any more!
@@ Line 249: / Line 276: @@
 </code>
-The top lines give you some summary information about the system, but you should monitor the memory usage (''VIRT'' and ''RES''), ''CPU'' and ''TIME'' columns. By default, ''top'' will will put the processes using the more CPU at the top (as shown above). You can see above that the ''emacs'' text editor has clearly crashed, because it should not use 100% CPU for such a long time
+The top lines give you some summary information about the system, but you should monitor the memory usage (''VIRT'' and ''RES''), ''CPU'' and ''TIME+'' columns. By default, ''top'' will will put the processes using the more CPU at the top (as shown above). You can see above that the ''emacs'' text editor has clearly crashed, because it should not use 100% CPU for such a long time
+  * The ''TIME+'' information is in //minutes// (e.g. ''5432:01'' means //5432 minutes and 1 second//, ''25:15.20'' means //25 minutes, 15 seconds and 20% of 1 second//)
+  * Memory usage:
+    * ''%MEM'' (Kb): percentage of the total available **physical memory** used by a process. This is based on ''RES'', the **non-swapped physical memory** used by a process
+    * ''VIRT'' (Kb): total  amount  of **virtual memory** (i.e. memory temporarily swapped/saved to disk) used by a process.  It includes all code, data and shared libraries plus pages that have been swapped out and pages that have been mapped but not used
+    * The ''free'' command can display the memory usage of a server at a given time. ''available'' is an estimation of how much memory is available for starting new applications, without swapping
+      * <code> % free -ht
+              total        used        free      shared  buff/cache   available
+Mem:            62G        1.5G         56G         11M        5.0G         60G
+Swap:           62G        1.7G         60G
+Total:         125G        3.3G        116G</code>
 If you type ''M'', the processes will be sorted my memory usage, as shown below
@@ Line 265: / Line 304: @@
 |  **q**  | Quit |
-|  **c**  | Display the command line options of the running processes|
+|  **c**  | Display the command line options of the running processes |
 |  **u**  | List only the processes of a specific user |
+|  **M**  | Sort by memory usage  (''%MEM'' columun)  |
+|  **P**  | Sort by CPU usage  (''%CPU'' columun) (Default) |
+==== Using the kill command to terminate processes ====
+FIXME
+''kill proc_number''
+''kill -9 -1''
+''qdel''
 ===== Which disks should you use? =====
@@ Line 306: / Line 356: @@
       * tcsh shell users: ''.cshrc'' and ''.login''
       * bash shell users: ''.profile'' and ''.bashrc''
-      * emacs text editor: ''.emacs''
+      * [[other:emacs_doc|emacs text editor]]: ''.emacs''
     * You can go to your home directory by typing ''cd'' (no arguments),\\ and to the home directory of somebody else with ''cd ~login_name''
       * At LSCE, you can't read the content of somebody else's //home// directory by default
@@ Line 373: / Line 423: @@
 ...</code>
+FIXME Add something about ''df''
 ===== Which programs should you use? =====
@@ Line 445: / Line 496: @@
 <note important>A //jupyter notebook// is **not** a text editor!
-You need a correct text editor to efficiently work with scripts and programs</note>
+**You need a correct text editor to efficiently work with scripts and programs**
-There are **lots** of text editors used on Linux computers/servers (vi, vim, gvim, emacs, nano, ...), that you can use:
+{{ :other:newppl:real_programmers.png?direct&600 |}} [[https://www.explainxkcd.com/wiki/index.php/378:_Real_Programmers|xkcd & Real Programmers]]</note>
-  * in basic text terminals (if you know the basic commands and shortcuts)
-  * or in graphics mode (you may need to have an [[other:x_conf|X server configured]])
+There are **lots of text editors available** on Linux computers/servers (''emacs'', ''nano'', ''vi'', ''vim'', ''gvim'', ''gedit'', ...). You can use them:
+  * In basic **text terminals**
+    * **If you only need to read a text file**, use a //pager// like [[http://www.greenwoodsoftware.com/less|less]] (or ''more''): ''less my_text_file.txt''
+      * ''SPACE'' goes down one page
+      * ''b'' (//back//) goes back one page
+      * ''/STRING'' looks for ''STRING'' in the text (then ''n''ext, ''p''revious, ''q''uit)
+      * ''q'' exits
+    * **[[#gnu_nano|nano]] is probably the easiest editor in text mode** to use!
+    * Useful if you are in a hurry, and/or working on a remote server with a bad network connection or have no local [[other:x_conf|X server]] running
+    * You should know the basic commands and shortcuts!
+  * With a **full Graphical User Interface** (aka //GUI//) allowing you to use the mouse+menus/buttons/etc...
+    * you will need to have a //local// [[other:x_conf|X server]] running, if you want to use the text editor GUI on a //remote// server
 === emacs ===
@@ Line 455: / Line 519: @@
 <note important>Read the **dedicated [[other:emacs_doc|Using emacs]] page**, and become a power user!</note>
-{{ :other:newppl:real_programmers.png?direct&600 |}} [[https://www.explainxkcd.com/wiki/index.php/378:_Real_Programmers|xkcd & Real Programmers]]
+=== Gnu nano ===
+<note tip>Gnu [[https://www.nano-editor.org/|nano]] is probably the easiest text editor in text mode to use!</note>
+You can easily view/edit a file by just typing ''nano my_text_file.txt'' in a terminal, and looking at the instructions displayed at the bottom of the screen (e.g. ''^X Exit'' means that you can exit the editor by typing ''CTRL-X'')
 === vi (vim, gvim) ===
@@ Line 465: / Line 532: @@
 === Notepad++ ===
-[[other:win10apps#notepad|Notepad++]] seems like a nice and powerful text editor for **Windows**, if you don't have  the time and resource to install WSL+emacs
+[[other:win10apps#notepad|Notepad++]] is a nice and powerful text editor for **Windows**, but it is also very easy to [[other:emacs_doc#windows_installation|install emacs on Windows]]
-==== NetCDF and file formats ====
+==== NetCDF and related Conventions ====
+=== Super-short introduction to NetCDF ===
   * What are binary ({{:other:newppl:binaire_jyp.pdf}}) and NetCDF ({{:other:newppl:netcdf_jyp.pdf}}) files?
@@ Line 473: / Line 542: @@
   * The NetCDF file format is **self-documented**, and the metadata of climate data files often follows the [[http://cfconventions.org/|CF (Climate and Forecast) Metadata Conventions]]
     * The files in the [[https://esgf-node.ipsl.upmc.fr/projects/esgf-ipsl/|CMIPn (n ≥ 5) database]] are written with the [[https://cmor.llnl.gov/|CMOR 3 library]] in order to make sure that they follow exactly the requested standards
-  * You can use [[other:win7apps#panoply|Panoply]] and [[http://meteora.ucsd.edu/~pierce/ncview_home_page.html|ncview]] for visualizing quickly the NetCDF files
-  * [[https://code.zmaw.de/projects/cdo/wiki/Cdo#Documentation|Climate Data Operators]] for manipulating NetCDF files (and some other formats)
-  * [[http://nco.sourceforge.net/#RTFM|netCDF Operator (NCO)]] for manipulating NetCDF files
+=== Some ways of working with NetCDF without programming ===
-  * [[http://cfconventions.org/|NetCDF CF (Climate and Forecast) Metadata Conventions]]: this will help you understand the information you get when you do a ''ncdump -h some_climate_data_file.nc''
-  * Using Python to work with NetCDF data:
+  * Displaying the content of a NetCDF file in a text format\\ Check the ''ncdump'' [[https://docs.unidata.ucar.edu/nug/current/netcdf_utilities_guide.html#ncdump_guide|options]]. People usually want to quickly check the metadata, and use other programs to work with the actual data
-    * Using the [[http://uv-cdat.llnl.gov/documentation/cdms/cdms.html|cdms2]] module if you work with the [[other:python:starting#uv-cdat|UV-CDAT distribution]]
+    * ''ncdump some_climate_data_file.nc'': dump the full content to text
-    * Using the [[http://unidata.github.io/netcdf4-python/|netCDF4]] module
+    * ''ncdump -h some_climate_data_file.nc'': only display the metadata
+  * You can use [[other:win7apps#panoply|Panoply]] and [[http://meteora.ucsd.edu/~pierce/ncview_home_page.html|ncview]] for quickly visualizing NetCDF files
+  * Manipulating NetCDF files:
+    * [[https://code.zmaw.de/projects/cdo/wiki/Cdo#Documentation|Climate Data Operators]] (''cdo'')
+    * [[http://nco.sourceforge.net/#RTFM|netCDF Operator]] (//NCO// operators)
+=== NetCDF related Conventions ===
+  * [[https://arxiv.org/abs/1911.08638|Gridspec: A standard for the description of grids used in Earth System models]]
+    * A nice document for discovering the different type of (ir)regular grids used in climate models!
+  * [[http://cfconventions.org/|NetCDF CF (Climate and Forecast) Metadata Conventions]]
+  * [[https://ugrid-conventions.github.io/ugrid-conventions/|UGRID Conventions]] for storing unstructured (or flexible mesh) model data
+  * [[http://sgrid.github.io/sgrid/|SGRID Conventions]] for storing staggered data on structured grids that are consistent with the UGRID conventions
+=== Using NetCDF with Python ===
+Check the [[other:python:jyp_steps#using_netcdf_files_with_python|dedicated section]] of the //Python// page
 ==== Programming languages ====

PMIP3 wiki

User Tools

Site Tools

Differences

Page Tools