User Tools

Site Tools


other:newppl:starting

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
Next revision Both sides next revision
other:newppl:starting [2022/08/23 09:38]
jypeter [Accessing the Windows cluster from a Linux computer] Improved
other:newppl:starting [2023/02/01 16:38]
jypeter [NetCDF and file formats] Improved the section
Line 134: Line 134:
  
 Read the [[other:​index#​windows_10_notes|Windows 10 notes]], especially the //​Configuring Windows 10// section Read the [[other:​index#​windows_10_notes|Windows 10 notes]], especially the //​Configuring Windows 10// section
- 
-Note: there are also some [[other:​index#​windows_7_notes|Windows 7 notes]], but you are supposed to use Windows 10 since January 2020! 
  
 === Getting administrator'​s rights === === Getting administrator'​s rights ===
Line 176: Line 174:
 ===== Accessing the LSCE Linux servers ===== ===== Accessing the LSCE Linux servers =====
  
-You have to [[other:​ssh|use ssh to connect to the servers]], and [[other:​ssh#​copying_files_with_scp|use scp to copy files between servers]]+You have to [[other:​ssh|use ssh]] to connect to the [[other:​ssh#​lsce_servers|LSCE Linux servers]], and [[other:​ssh#​copying_files_with_scp|use scp to copy files between servers]]
  
-If you are really in a hurry, go directly to the [[other:​ssh#​connecting_to_the_lsce_servers_ipsl_servers_tgcc|Connecting to the LSCE servers, IPSL servers, TGCC, ...]] section, but it is important that you **read the full [[other:​ssh|ssh documentation]] page at least once**!+If you are really in a hurry, go directly to the [[other:​ssh#​connecting_to_servers_commonly_used_by_lsce_users|Connecting to servers commonly used by LSCE users]] section, but it is important that you **read the full [[other:​ssh|ssh documentation]] page at least once**!
  
  
Line 186: Line 184:
 ==== Working directly on your desktop ==== ==== Working directly on your desktop ====
  
-Always remember that **your local LSCE desktop can [[#​accessing_remote_disks|access remote disks on the LSCE servers]]**.+Always remember that **your local LSCE desktop/​laptop ​can [[#​accessing_remote_disks|access remote disks on the LSCE servers]]**.
  
-There are lots of things you can do directly on your **local** ​Windows 10 or Linux desktop ​(displaying pdf, images, using a text editor, ...), rather than on the remote servers. For example, if a script running on the servers generates a pdf file, it is more efficient to open this pdf file using ''​Acrobat''/''​evince''​ on your desktop, than by using ''​evince''​ on the server.+There are **lots** of things you can do directly on your **local** ​computer ​(displaying pdf, images, using a text editor, ...), rather than on the remote servers. For example, if a script running on the servers generates a pdf file, it is **more efficient** to open this pdf file using ''​Acrobat''/''​evince''​ on your local computer, than by using ''​evince''​ on the server.
  
 ==== Available servers ==== ==== Available servers ====
  
-  * **The LSCE interactive servers**: ''​obelix//​NN//''​\\ Use ''​ssh obelix''​ to access these servers, and the //load balancing// system will send you to the server that has currently the smallest load.\\ Never forget that you are sharing these servers with other users! Do not use too much CPU and/or memory for a long time. Heavy computation should be done on the //LSCE cluster//+  * **The LSCE interactive servers**: ''​obelix//​NN//''​\\ Use ''​ssh obelix''​ to access these servers, and the //load balancing// system will send you to the server that has currently the smallest load. 
 +    * <wrap hi>Never forget that you are sharing these servers with other users!</​wrap>​ 
 +    * <wrap hi>Do not use too much CPU and/or memory for a long time</​wrap>​. Heavy computation should be done on the [[https://intranet.lsce.ipsl.fr/​informatique/​en/​calcul/​batch.php|LSCE cluster]] 
 +      * Learn how to [[other:​newppl:​starting#​determining_the_load_of_a_linux_server|use the top command]] to determine the current load of a server and the CPU/memory usage of your processes !
     * <​code>​$ ssh obelix     * <​code>​$ ssh obelix
 Last login: Mon Jun  3 08:49:53 2019 from somewhere Last login: Mon Jun  3 08:49:53 2019 from somewhere
Line 439: Line 440:
   * Useful Linux commands for working with (potentially big) text files ({{:​other:​newppl:​houchesa4_updated.pdf}})   * Useful Linux commands for working with (potentially big) text files ({{:​other:​newppl:​houchesa4_updated.pdf}})
   * Using [[other:​ssh|ssh]] and [[other:​ssh#​copying_files_with_scp|scp]]   * Using [[other:​ssh|ssh]] and [[other:​ssh#​copying_files_with_scp|scp]]
-  * Text editors: ​there are **lots** of text editors ​used on Linux computers/​servers (vi, vim, gvim, emacs, nano, ...). You will find some basic information below + 
-    * ''​vi'':​ [[https://​www.atmos.albany.edu/​daes/​atmclasses/​atm350/​vi_cheat_sheet.pdf|vi cheat sheet]] ​and [[https://​web.mit.edu/​merolish/​Public/​vi-ref.pdf|vi reference card]] +==== Text editors ​==== 
-    * An annotated reference card of the ''​emacs''​ text editor ({{:​other:​newppl:​emacs_jyp.pdf}}) + 
-      * JYP's .emacs file: {{:other:emacs_linux_150806.tar|for Linux}} - {{:​other:​emacs_win_7_64_140408.zip|for Windows}}with [[other:win7apps#gnu_emacs_gnuwin_diffutils|emacs ​for Windows]] +<note important>​A //jupyter notebook// is **not** a text editor! 
-==== NetCDF and file formats ​====+ 
 +**You need a correct text editor to efficiently work with scripts and programs** 
 + 
 +{{ :other:​newppl:​real_programmers.png?​direct&​600 |}} [[https://​www.explainxkcd.com/​wiki/​index.php/​378:​_Real_Programmers|xkcd & Real Programmers]]</​note>​  
 + 
 +There are **lots of text editors ​available** ​on Linux computers/​servers (''​emacs'',​ ''​nano'',​ ''​vi''​''​vim''​''​gvim''​''​gedit''​, ...). You can use them: 
 +  * In basic **text terminals** 
 +    * useful if you are in a hurry, and/or working on a remote server with a bad network connection or have no local [[other:​x_conf|X server]] running 
 +    * you should know the basic commands and shortcuts! Some text editors (e.g ''​nano''​) fortunately display some help at the bottom of the screen 
 + 
 +  * With a **full Graphical User Interface** (aka //GUI//) allowing you to use the mouse+menus/​buttons/​etc... 
 +    * you will need to have a //local// [[other:x_conf|X server]] running, if you want to use the text editor GUI on a //remote// server 
 + 
 +=== emacs === 
 + 
 +<note important>​Read the **dedicated [[other:​emacs_doc|Using emacs]] page**, and become a power user!</​note>​ 
 + 
 + 
 + 
 +=== vi (vim, gvim) === 
 + 
 +  * [[https://​www.atmos.albany.edu/​daes/​atmclasses/​atm350/​vi_cheat_sheet.pdf|vi cheat sheet]] 
 +  * [[https://​web.mit.edu/​merolish/​Public/​vi-ref.pdf|vi reference card]] 
 + 
 +=== Notepad++ === 
 + 
 +[[other:win10apps#​notepad|Notepad++]] is a nice and powerful text editor ​for **Windows**but it is also very easy to [[other:emacs_doc#windows_installation|install ​emacs on Windows]] 
 +==== NetCDF and related Conventions ​==== 
 + 
 +=== Super-short introduction to NetCDF ​===
  
   * What are binary ({{:​other:​newppl:​binaire_jyp.pdf}}) and NetCDF ({{:​other:​newppl:​netcdf_jyp.pdf}}) files?   * What are binary ({{:​other:​newppl:​binaire_jyp.pdf}}) and NetCDF ({{:​other:​newppl:​netcdf_jyp.pdf}}) files?
Line 450: Line 480:
   * The NetCDF file format is **self-documented**,​ and the metadata of climate data files often follows the [[http://​cfconventions.org/​|CF (Climate and Forecast) Metadata Conventions]]   * The NetCDF file format is **self-documented**,​ and the metadata of climate data files often follows the [[http://​cfconventions.org/​|CF (Climate and Forecast) Metadata Conventions]]
     * The files in the [[https://​esgf-node.ipsl.upmc.fr/​projects/​esgf-ipsl/​|CMIPn (n ≥ 5) database]] are written with the [[https://​cmor.llnl.gov/​|CMOR 3 library]] in order to make sure that they follow exactly the requested standards     * The files in the [[https://​esgf-node.ipsl.upmc.fr/​projects/​esgf-ipsl/​|CMIPn (n ≥ 5) database]] are written with the [[https://​cmor.llnl.gov/​|CMOR 3 library]] in order to make sure that they follow exactly the requested standards
-  * You can use [[other:​win7apps#​panoply|Panoply]] and [[http://​meteora.ucsd.edu/​~pierce/​ncview_home_page.html|ncview]] for visualizing quickly the NetCDF files 
-  * [[https://​code.zmaw.de/​projects/​cdo/​wiki/​Cdo#​Documentation|Climate Data Operators]] for manipulating NetCDF files (and some other formats) 
-  * [[http://​nco.sourceforge.net/#​RTFM|netCDF Operator (NCO)]] for manipulating NetCDF files 
-  * [[http://​cfconventions.org/​|NetCDF CF (Climate and Forecast) Metadata Conventions]]:​ this will help you understand the information you get when you do a ''​ncdump -h some_climate_data_file.nc''​ 
-  * Using Python to work with NetCDF data: 
-    * Using the [[http://​uv-cdat.llnl.gov/​documentation/​cdms/​cdms.html|cdms2]] module if you work with the [[other:​python:​starting#​uv-cdat|UV-CDAT distribution]] 
-    * Using the [[http://​unidata.github.io/​netcdf4-python/​|netCDF4]] module 
  
-==== Using the Python language ==== 
  
-  ​- [[other:python:starting|Working with Python]] +=== Some ways of working with NetCDF without programming === 
-  - [[other:​python:​jyp_steps|JYP'​s ​recommended steps for learning python]]+ 
 +  * Displaying the content of a NetCDF file in a text format\\ Check the ''​ncdump''​ [[https://​docs.unidata.ucar.edu/​nug/​current/​netcdf_utilities_guide.html#​ncdump_guide|options]]. People usually want to quickly check the metadata, and use other programs to work with the actual data 
 +    * ''​ncdump some_climate_data_file.nc'':​ dump the full content to text 
 +    * ''​ncdump ​-h some_climate_data_file.nc'':​ only display the metadata 
 +  * You can use [[other:win7apps#​panoply|Panoply]] and [[http://​meteora.ucsd.edu/​~pierce/​ncview_home_page.html|ncview]] for quickly visualizing NetCDF files 
 +  ​* Manipulating NetCDF files: 
 +    * [[https://​code.zmaw.de/​projects/​cdo/​wiki/​Cdo#​Documentation|Climate Data Operators]] (''​cdo''​) 
 +    * [[http://​nco.sourceforge.net/#​RTFM|netCDF Operator]] (//NCO// operators) 
 + 
 + 
 +=== NetCDF related Conventions === 
 + 
 +  * [[http://​cfconventions.org/​|NetCDF CF (Climate and Forecast) Metadata Conventions]] 
 +  * [[https://​ugrid-conventions.github.io/​ugrid-conventions/​|UGRID Conventions]] for storing unstructured (or flexible mesh) model data 
 +  * [[http://​sgrid.github.io/​sgrid/​|SGRID Conventions]] for storing staggered data on structured grids that are consistent with the UGRID conventions 
 + 
 +=== Using NetCDF with Python === 
 + 
 + 
 +Check the [[other:​python:​jyp_steps#​using_netcdf_files_with_python|dedicated section]] of the //Python// page 
 + 
 +==== Programming languages ==== 
 + 
 +  * [[other:​index#​python_notes|JYP'​s ​Python notes]]
  
 ===== Getting help (from the LSCE system administrators) ===== ===== Getting help (from the LSCE system administrators) =====
Line 501: Line 546:
 {{ :​other:​newppl:​screenshot_xfce.jpg?​200 |Click to get a larger version}} {{ :​other:​newppl:​screenshot_xfce.jpg?​200 |Click to get a larger version}}
  
-On Windows, you can use the //Capture screen// option of [[other:win7apps#xnview|XnView]]+On Windows, you can use [[other:​win10apps#​snip_sketch|Snip & Sketch]], or the //Capture screen// option of [[other:win10apps#xnview_classic|XnView]]
 {{ :​other:​newppl:​xnview_capture.jpg?​200 |Click to get a larger version}} {{ :​other:​newppl:​xnview_capture.jpg?​200 |Click to get a larger version}}
  
other/newppl/starting.txt · Last modified: 2024/07/02 10:22 by jypeter