This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
other:newppl:starting [2021/08/16 08:17] jypeter [Miscellaneous information] Updated link for Paris-Saclay welcome desk |
other:newppl:starting [2023/09/11 08:14] jypeter [Miscellaneous information] |
||
---|---|---|---|
Line 13: | Line 13: | ||
* [[other:newppl:riealgo|Adding money online to your restaurant account]] | * [[other:newppl:riealgo|Adding money online to your restaurant account]] | ||
* [[https://intranet.lsce.ipsl.fr/Phocea/Page/index.php?id=97|Survival kit and other information for CDD and interns]] (LSCE intranet) | * [[https://intranet.lsce.ipsl.fr/Phocea/Page/index.php?id=97|Survival kit and other information for CDD and interns]] (LSCE intranet) | ||
- | * [[http://internationaloffice.ceasaclay.com/?lang=en|CEA Saclay International Office]] //Our multilingual staff is readily available to assist you through immigration procedures and to help you settle in France// | + | * [[https://www.universite-paris-saclay.fr/en/campus-life/international-welcome-desk|International welcome desk]] //Université Paris-Saclay supports international students, PhD students and researchers as soon as they arrive in France to ensure their stay take place under the best conditions// |
- | * [[https://www.universite-paris-saclay.fr/en/campus-life/international-welcome-desk|International welcome desk]] //Université Paris-Saclay provides you with a variety of services and documentation to welcome international students and researchers under the best conditions// | + | |
* [[https://logement.campus-paris-saclay.fr/| Trouver un logement dans une résidence universitaire]] (in French... //Finding a room in a student dorm//) | * [[https://logement.campus-paris-saclay.fr/| Trouver un logement dans une résidence universitaire]] (in French... //Finding a room in a student dorm//) | ||
* [[https://www.science-accueil.org/en/|Science Accueil]] | * [[https://www.science-accueil.org/en/|Science Accueil]] | ||
+ | * [[https://www.science-accueil.org/en/request-services/|Service request page]] | ||
* {{:other:newppl:uvsq-guide-du-personnel-nov2019.pdf|Guide du Personnel UVSQ}} | * {{:other:newppl:uvsq-guide-du-personnel-nov2019.pdf|Guide du Personnel UVSQ}} | ||
Line 80: | Line 80: | ||
===== Using your own computer at LSCE? ===== | ===== Using your own computer at LSCE? ===== | ||
- | No need to try to connect your personal laptop to the LSCE wire network, it will **not** work! | + | <wrap em>You can't connect your personal laptop to the LSCE wired network</wrap> (aka //ethernet//), it will **not** work! |
The best you can do is use the [[https://intranet.lsce.ipsl.fr/informatique/en/wifi/index.php|Guest WiFi]] or [[https://intranet.lsce.ipsl.fr/informatique/en/wifi/eduroam.php|eduroam]] network, if your office is close enough to one of the access points. | The best you can do is use the [[https://intranet.lsce.ipsl.fr/informatique/en/wifi/index.php|Guest WiFi]] or [[https://intranet.lsce.ipsl.fr/informatique/en/wifi/eduroam.php|eduroam]] network, if your office is close enough to one of the access points. | ||
- | **Warning!** You **can't** connect directly to the //obelix// LSCE servers from the wifi network. You first have to connect to the ''ssh1.lsce.ipsl.fr'' //gateway// server, and then to ''obelix''. More details in [[#accessing_the_lsce_servers_from_outside_the_lsce|Accessing the LSCE servers from outside the LSCE]] | + | **Warning!** You **can't** connect directly to the //obelix// LSCE servers from the wifi network. You first have to connect to the ''ssh1.lsce.ipsl.fr'' //gateway// server, and then to ''obelix''. More details in [[other:ssh#lsce_servers|LSCE servers]] |
===== Accessing the LSCE intranet ===== | ===== Accessing the LSCE intranet ===== | ||
<WRAP center round tip 60%> | <WRAP center round tip 60%> | ||
- | If you try to access the intranet **from outside the LSCE** network (including //eduroam//), your browser will open a ''login''/''password'' popup window. Use your LSCE login (not your email address!) and password | + | If you try to access the intranet **from outside the LSCE** network (including //eduroam//), your browser will open a ''login''/''password'' popup window. |
+ | |||
+ | Use your LSCE login (not your email address!) and password\\ e.g. use ''jdoe'' and not ''jane.doe@lsce.ipsl.fr'' | ||
</WRAP> | </WRAP> | ||
Line 131: | Line 133: | ||
Read the [[other:index#windows_10_notes|Windows 10 notes]], especially the //Configuring Windows 10// section | Read the [[other:index#windows_10_notes|Windows 10 notes]], especially the //Configuring Windows 10// section | ||
- | |||
- | Note: there are also some [[other:index#windows_7_notes|Windows 7 notes]], but you are supposed to use Windows 10 since January 2020! | ||
=== Getting administrator's rights === | === Getting administrator's rights === | ||
Line 158: | Line 158: | ||
==== Mac ==== | ==== Mac ==== | ||
- | Sorry, you are on your own, but it's soooo easy to use, right? | + | Sorry, you are (almost) on your own, but a Mac is soooo easy to use, right? 8-) |
+ | |||
+ | Some useful mac links: | ||
+ | * [[other:ssh#a_recommended_terminal_for_mac|A recommended terminal for Mac]] | ||
+ | * [[other:x_conf#using_an_x_server_on_a_mac|Using an X server on a Mac]] | ||
+ | * [[other:ssh#mac_ssh_agent|Mac ssh agent]] | ||
===== Accessing the Windows cluster from a Linux computer ===== | ===== Accessing the Windows cluster from a Linux computer ===== | ||
- | Follow the instructions in the [[https://intranet.lsce.ipsl.fr/informatique/en/winsrv.php|Windows servers]] page. If you need to use a program that is missing on this cluster, see the [[#getting_help_from_the_lsce_system_administrators|Help]] section | + | If you have a Linux computer, but need to use a Windows application, follow the instructions in the [[https://intranet.lsce.ipsl.fr/informatique/en/winsrv.php|Windows servers]] page. |
+ | |||
+ | If you need to use a program that is missing on the Windows cluster, see the [[#getting_help_from_the_lsce_system_administrators|Help]] section. | ||
===== Accessing the LSCE Linux servers ===== | ===== Accessing the LSCE Linux servers ===== | ||
- | You have to [[other:ssh|Use ssh to connect to the servers]] | + | You have to [[other:ssh|use ssh]] to connect to the [[other:ssh#lsce_servers|LSCE Linux servers]], and [[other:ssh#copying_files_with_scp|use scp to copy files between servers]] |
- | If you are really in a hurry, go directly to the [[other:ssh#connecting_to_the_lsce_servers_ipsl_servers_tgcc|Connecting to the LSCE servers, IPSL servers, TGCC, ...]] section, but it is important that you **read the full //ssh documentation// page at least once**! | + | If you are really in a hurry, go directly to the [[other:ssh#connecting_to_servers_commonly_used_by_lsce_users|Connecting to servers commonly used by LSCE users]] section, but it is important that you **read the full [[other:ssh|ssh documentation]] page at least once**! |
Line 176: | Line 183: | ||
==== Working directly on your desktop ==== | ==== Working directly on your desktop ==== | ||
- | Always remember that **your local LSCE desktop can [[#accessing_remote_disks|access remote disks on the LSCE servers]]**. | + | Always remember that **your local LSCE desktop/laptop can [[#accessing_remote_disks|access remote disks on the LSCE servers]]**. |
- | There are lots of things you can do directly on your **local** Windows 10 or Linux desktop (displaying pdf, images, using a text editor, ...), rather than on the remote servers. For example, if a script running on the servers generates a pdf file, it is more efficient to open this pdf file using ''Acrobat''/''evince'' on your desktop, than by using ''evince'' on the server. | + | There are **lots** of things you can do directly on your **local** computer (displaying pdf, images, using a text editor, ...), rather than on the remote servers. For example, if a script running on the servers generates a pdf file, it is **more efficient** to open this pdf file using ''Acrobat''/''evince'' on your local computer, than by using ''evince'' on the server. |
==== Available servers ==== | ==== Available servers ==== | ||
- | * **The LSCE interactive servers**: ''obelix//NN//''. Use ''ssh obelix'' to access these servers, and the //load balancing// system will send you to the server that has currently the smallest load.\\ Never forget that you are sharing these servers with other users! Do not use too much CPU and/or memory for a long time. Heavy computation should be done on the obelix cluster | + | * **The LSCE //gateway//**: ''ssh1.lsce.ipsl.fr'' |
- | * <code>$ ssh obelix | + | * Use ''ssh ssh1.lsce.ipsl.fr'' when you are **outside the LSCE wired network**, before accessing any other LSCE server: |
+ | * By **//outside LSCE//**, we mean that your computer is: | ||
+ | * connected to the LSCE WiFi network (instead of the wired network) | ||
+ | * or physically outside LSCE | ||
+ | * You can also use ''ssh1'' when you are **copying files to/from outside LSCE** with ''scp'' or ssh-based tools | ||
+ | * <wrap hi>The ''ssh1'' gateway can **only** be used for copying files or accessing the LSCE interactive servers</wrap>! | ||
+ | * **You have to** connect to an ''obelix//NN//'' interactive server before doing any real work | ||
+ | |||
+ | * **The LSCE interactive servers**: ''obelix//NN//'' (''obelix2'' to ''obelix5'') | ||
+ | * Use ''ssh obelix'' to access the LSCE servers from **inside LSCE**.\\ The //load balancing// system will send you to the ''obelix//NN//'' server that has currently the smallest load. | ||
+ | * By **//inside LSCE//**, we mean that: | ||
+ | * your computer is connected to the LSCE **wired** network | ||
+ | * or you are using the [[https://intranet.lsce.ipsl.fr/informatique/fr/vpn-forticlient.php#vpnlsce|LSCE VPN]] | ||
+ | * or you have a //terminal// connected to the ''ssh1'' gateway | ||
+ | * <wrap hi>Never forget that you are **sharing** these interactive servers with other users!</wrap> | ||
+ | * <wrap hi>Do not use too much CPU and/or memory for a long time</wrap>. Heavy computation should be done on the [[https://intranet.lsce.ipsl.fr/informatique/en/calcul/batch.php|LSCE cluster]] | ||
+ | * Learn how to [[other:newppl:starting#determining_the_load_of_a_linux_server|use the top command]] to determine the current load of a server and the CPU/memory usage of your processes ! | ||
+ | * You can use one of the following commands if you want to know on which ''obelixNN'' you are | ||
+ | * <code>$ ssh obelix | ||
Last login: Mon Jun 3 08:49:53 2019 from somewhere | Last login: Mon Jun 3 08:49:53 2019 from somewhere | ||
- | # You can use one of the following commands if you want to know on which obelix you are | ||
$ echo $HOST | $ echo $HOST | ||
obelix5 | obelix5 | ||
Line 191: | Line 215: | ||
obelix5</code> | obelix5</code> | ||
- | * **[[https://intranet.lsce.ipsl.fr/informatique/en/calcul/batch.php|The LSCE cluster]]** (aka the //batch system//). Use this for really heavy duty programs | + | * **[[https://intranet.lsce.ipsl.fr/informatique/en/calcul/batch.php|The LSCE cluster]]** (aka the //batch system//)\\ Use this cluster for **//heavy duty// programs**, rather than killing the multi-users interactive servers |
- | * **[[http://mesocentre.ipsl.fr/|The ciclad and climserv servers]]**. In some cases, you may need an account to use the [[http://mesocentre.ipsl.fr/|IPSL/ESPRI clusters]], especially if you need to use data files that are already available/mirrored there (e.g. **''CMIPn'' data**), rather than copying the data to LSCE | + | * //Heavy duty// programs are programs that will prevent other users from working on the same server because these programs use a lot of CPU and/or memory for more than a few minutes. Ask your advisor, if you are not sure |
- | * [[http://mesocentre.ipsl.fr/account-opening/|Requesting a new account]] | + | |
- | * **The ''asterix//N//'' servers**: some wise LSCE elders may mention these servers, but they don't exist any more! | + | * **The [[https://documentations.ipsl.fr/spirit/|IPSL Computing and Data Center]]**, aka //Mésocentre ESPRI// (''Spirit'' / ''SpiritX'' / ''HAL'') |
+ | * Note: the ''Spirit''/''X'' clusters have replaced the ''ciclad''/''climserv'' cluster (in 2022) | ||
+ | * You will need a specific account (different from your //LSCE// account) to access these servers and use data files that are already available/mirrored there (e.g. **''CMIPn'' data**).\\ **Do not duplicate** at LSCE data already available on IPSL servers and that you can process there! | ||
+ | * [[https://mesocentre.ipsl.fr/account-opening/|Requesting an IPSL account]] | ||
+ | * [[https://documentations.ipsl.fr/spirit/|IPSL ESPRI Mesocenter documentation]] | ||
+ | * [[https://documentations.ipsl.fr/spirit/spirit_clusters/head_nodes.html|spirit clusters]] | ||
+ | * [[https://documentations.ipsl.fr/spirit/getting_started/support.html|Getting help]] | ||
+ | |||
+ | * **The ''asterix//NN//'' servers**\\ Some wise LSCE elders may mention these servers, but they don't exist any more! | ||
==== Which shell are you using? ==== | ==== Which shell are you using? ==== | ||
Line 268: | Line 300: | ||
==== Accessing remote disks ==== | ==== Accessing remote disks ==== | ||
- | It is possible to access directly all the disks on the LSCE servers from your //local// Linux or Windows computer\\ (//local// = a computer on the LSCE ethernet/wired network) | + | It is possible to **directly** access some disks on the LSCE servers from: |
- | * Example path on the LSCE Linux server: ''/home/scratch01/your_lsce_login'' | + | * a Linux or Windows computer connected directly to the LSCE ethernet/wired network |
- | * Accessing the path above from a //local// computer: | + | * or a remote computer using the [[https://intranet.lsce.ipsl.fr/informatique/fr/vpn-forticlient.php#vpnlsce|LSCE VPN]] |
- | * Linux computer: ''/home/scratch01/your_lsce_login'' | + | |
- | * Windows computer: ''\\dfshost\dfs\scratch01\your_lsce_login''\\ {{ :other:newppl:dfs_on_win.png?nolink |}} | + | For security reasons, only backed up disks and ''scratch01'' can be accessed from Windows with //dfs// |
+ | |||
+ | e.g. if you have a ''/home/scratch01/your_lsce_login'' path on a Linux server, you can access it the following way from your desktop/laptop: | ||
+ | |||
+ | * Linux computer: ''cd /home/scratch01/your_lsce_login'' | ||
+ | * Windows computer explorer: ''\\dfshost\dfs\scratch01\your_lsce_login''\\ {{ :other:newppl:dfs_on_win.png?nolink |}} | ||
==== The home directory ==== | ==== The home directory ==== | ||
Line 286: | Line 323: | ||
* tcsh shell users: ''.cshrc'' and ''.login'' | * tcsh shell users: ''.cshrc'' and ''.login'' | ||
* bash shell users: ''.profile'' and ''.bashrc'' | * bash shell users: ''.profile'' and ''.bashrc'' | ||
- | * emacs text editor: ''.emacs'' | + | * [[other:emacs_doc|emacs text editor]]: ''.emacs'' |
* You can go to your home directory by typing ''cd'' (no arguments),\\ and to the home directory of somebody else with ''cd ~login_name'' | * You can go to your home directory by typing ''cd'' (no arguments),\\ and to the home directory of somebody else with ''cd ~login_name'' | ||
* At LSCE, you can't read the content of somebody else's //home// directory by default | * At LSCE, you can't read the content of somebody else's //home// directory by default | ||
Line 419: | Line 456: | ||
- A tutorial | - A tutorial | ||
* Useful Linux commands for working with (potentially big) text files ({{:other:newppl:houchesa4_updated.pdf}}) | * Useful Linux commands for working with (potentially big) text files ({{:other:newppl:houchesa4_updated.pdf}}) | ||
- | * Using ssh on [[http://ciclad-web.ipsl.jussieu.fr/data/documents/ssh-utilisateurs-unix.pdf|Linux]] and [[http://ciclad-web.ipsl.jussieu.fr/data/documents/ssh_putty_v2_2011-1.pdf|Windows]] | + | * Using [[other:ssh|ssh]] and [[other:ssh#copying_files_with_scp|scp]] |
- | * An annotated reference card of the emacs text editor ({{:other:newppl:emacs_jyp.pdf}}) | + | |
- | * JYP's .emacs file: {{:other:emacs_linux_150806.tar|for Linux}} - {{:other:emacs_win_7_64_140408.zip|for Windows}}, with [[other:win7apps#gnu_emacs_gnuwin_diffutils|emacs for Windows]] | + | |
- | You can also check the [[http://wiki.ipsl.jussieu.fr/FondamentauxpoleModelisationClimat|Les fondamentaux du pôle de modélisation du climat de l'IPSL]] wiki page | + | ==== Text editors ==== |
- | ==== NetCDF and file formats ==== | + | <note important>A //jupyter notebook// is **not** a text editor! |
+ | |||
+ | **You need a correct text editor to efficiently work with scripts and programs** | ||
+ | |||
+ | {{ :other:newppl:real_programmers.png?direct&600 |}} [[https://www.explainxkcd.com/wiki/index.php/378:_Real_Programmers|xkcd & Real Programmers]]</note> | ||
+ | |||
+ | There are **lots of text editors available** on Linux computers/servers (''emacs'', ''nano'', ''vi'', ''vim'', ''gvim'', ''gedit'', ...). You can use them: | ||
+ | * In basic **text terminals** | ||
+ | * useful if you are in a hurry, and/or working on a remote server with a bad network connection or have no local [[other:x_conf|X server]] running | ||
+ | * you should know the basic commands and shortcuts! Some text editors (e.g ''nano'') fortunately display some help at the bottom of the screen | ||
+ | |||
+ | * With a **full Graphical User Interface** (aka //GUI//) allowing you to use the mouse+menus/buttons/etc... | ||
+ | * you will need to have a //local// [[other:x_conf|X server]] running, if you want to use the text editor GUI on a //remote// server | ||
+ | |||
+ | === emacs === | ||
+ | |||
+ | <note important>Read the **dedicated [[other:emacs_doc|Using emacs]] page**, and become a power user!</note> | ||
+ | |||
+ | |||
+ | |||
+ | === vi (vim, gvim) === | ||
+ | |||
+ | * [[https://www.atmos.albany.edu/daes/atmclasses/atm350/vi_cheat_sheet.pdf|vi cheat sheet]] | ||
+ | * [[https://web.mit.edu/merolish/Public/vi-ref.pdf|vi reference card]] | ||
+ | |||
+ | === Notepad++ === | ||
+ | |||
+ | [[other:win10apps#notepad|Notepad++]] is a nice and powerful text editor for **Windows**, but it is also very easy to [[other:emacs_doc#windows_installation|install emacs on Windows]] | ||
+ | ==== NetCDF and related Conventions ==== | ||
+ | |||
+ | === Super-short introduction to NetCDF === | ||
* What are binary ({{:other:newppl:binaire_jyp.pdf}}) and NetCDF ({{:other:newppl:netcdf_jyp.pdf}}) files? | * What are binary ({{:other:newppl:binaire_jyp.pdf}}) and NetCDF ({{:other:newppl:netcdf_jyp.pdf}}) files? | ||
Line 432: | Line 497: | ||
* The NetCDF file format is **self-documented**, and the metadata of climate data files often follows the [[http://cfconventions.org/|CF (Climate and Forecast) Metadata Conventions]] | * The NetCDF file format is **self-documented**, and the metadata of climate data files often follows the [[http://cfconventions.org/|CF (Climate and Forecast) Metadata Conventions]] | ||
* The files in the [[https://esgf-node.ipsl.upmc.fr/projects/esgf-ipsl/|CMIPn (n ≥ 5) database]] are written with the [[https://cmor.llnl.gov/|CMOR 3 library]] in order to make sure that they follow exactly the requested standards | * The files in the [[https://esgf-node.ipsl.upmc.fr/projects/esgf-ipsl/|CMIPn (n ≥ 5) database]] are written with the [[https://cmor.llnl.gov/|CMOR 3 library]] in order to make sure that they follow exactly the requested standards | ||
- | * You can use [[other:win7apps#panoply|Panoply]] and [[http://meteora.ucsd.edu/~pierce/ncview_home_page.html|ncview]] for visualizing quickly the NetCDF files | ||
- | * [[https://code.zmaw.de/projects/cdo/wiki/Cdo#Documentation|Climate Data Operators]] for manipulating NetCDF files (and some other formats) | ||
- | * [[http://nco.sourceforge.net/#RTFM|netCDF Operator (NCO)]] for manipulating NetCDF files | ||
- | * [[http://cfconventions.org/|NetCDF CF (Climate and Forecast) Metadata Conventions]]: this will help you understand the information you get when you do a ''ncdump -h some_climate_data_file.nc'' | ||
- | * Using Python to work with NetCDF data: | ||
- | * Using the [[http://uv-cdat.llnl.gov/documentation/cdms/cdms.html|cdms2]] module if you work with the [[other:python:starting#uv-cdat|UV-CDAT distribution]] | ||
- | * Using the [[http://unidata.github.io/netcdf4-python/|netCDF4]] module | ||
- | ==== Using the Python language ==== | ||
- | - [[other:python:starting|Working with Python]] | + | === Some ways of working with NetCDF without programming === |
- | - [[other:python:jyp_steps|JYP's recommended steps for learning python]] | + | |
+ | * Displaying the content of a NetCDF file in a text format\\ Check the ''ncdump'' [[https://docs.unidata.ucar.edu/nug/current/netcdf_utilities_guide.html#ncdump_guide|options]]. People usually want to quickly check the metadata, and use other programs to work with the actual data | ||
+ | * ''ncdump some_climate_data_file.nc'': dump the full content to text | ||
+ | * ''ncdump -h some_climate_data_file.nc'': only display the metadata | ||
+ | * You can use [[other:win7apps#panoply|Panoply]] and [[http://meteora.ucsd.edu/~pierce/ncview_home_page.html|ncview]] for quickly visualizing NetCDF files | ||
+ | * Manipulating NetCDF files: | ||
+ | * [[https://code.zmaw.de/projects/cdo/wiki/Cdo#Documentation|Climate Data Operators]] (''cdo'') | ||
+ | * [[http://nco.sourceforge.net/#RTFM|netCDF Operator]] (//NCO// operators) | ||
+ | |||
+ | |||
+ | === NetCDF related Conventions === | ||
+ | |||
+ | * [[https://arxiv.org/abs/1911.08638|Gridspec: A standard for the description of grids used in Earth System models]] | ||
+ | * A nice document for discovering the different type of (ir)regular grids used in climate models! | ||
+ | |||
+ | * [[http://cfconventions.org/|NetCDF CF (Climate and Forecast) Metadata Conventions]] | ||
+ | * [[https://ugrid-conventions.github.io/ugrid-conventions/|UGRID Conventions]] for storing unstructured (or flexible mesh) model data | ||
+ | * [[http://sgrid.github.io/sgrid/|SGRID Conventions]] for storing staggered data on structured grids that are consistent with the UGRID conventions | ||
+ | |||
+ | === Using NetCDF with Python === | ||
+ | |||
+ | |||
+ | Check the [[other:python:jyp_steps#using_netcdf_files_with_python|dedicated section]] of the //Python// page | ||
+ | |||
+ | ==== Programming languages ==== | ||
+ | |||
+ | * [[other:index#python_notes|JYP's Python notes]] | ||
===== Getting help (from the LSCE system administrators) ===== | ===== Getting help (from the LSCE system administrators) ===== | ||
Line 483: | Line 566: | ||
{{ :other:newppl:screenshot_xfce.jpg?200 |Click to get a larger version}} | {{ :other:newppl:screenshot_xfce.jpg?200 |Click to get a larger version}} | ||
- | On Windows, you can use the //Capture screen// option of [[other:win7apps#xnview|XnView]] | + | On Windows, you can use [[other:win10apps#snip_sketch|Snip & Sketch]], or the //Capture screen// option of [[other:win10apps#xnview_classic|XnView]] |
{{ :other:newppl:xnview_capture.jpg?200 |Click to get a larger version}} | {{ :other:newppl:xnview_capture.jpg?200 |Click to get a larger version}} | ||
+ | ===== Getting help, when using the IPSL Mesocenter ===== | ||
+ | Check the [[https://mesocentre.ipsl.fr/contacts//|Contacts]] page | ||
===== What next? ===== | ===== What next? ===== | ||