Differences

This shows you the differences between two versions of the page.

--- other:python:starting [2019/06/04 18:08] – Re-ordered and improved jypeter
+++ other:python:starting [2025/07/03 16:18] (current) – [Using a non-standard kernel] jypeter
@@ Line 1: / Line 1: @@
 ====== Working with Python ======
-<note tip>This page will tell you how to select a python distribution and start (and exit!) the python interpreter
+<note tip>This page will tell you how to select a //python distribution// and start (and exit!) the python interpreter
-You can then read the [[jyp_steps|JYP's recommended steps for learning python]] for really working with python
+After reading this page, you can read the [[jyp_steps|JYP's recommended steps for learning python]] for really working with python
 </note>
@@ Line 30: / Line 30: @@
 </code>
-===== Ultra quick-start on the LSCE servers =====
+===== Ultra quick-start on the interactive LSCE servers =====
 ==== Initialization ====
@@ Line 40: / Line 40: @@
       * In a //bash// shell: ''source ~jypeter/.conda3_jyp.sh''
       * In a //tcsh// shell: ''source ~jypeter/.conda3_jyp.csh''
-    - Choose which //flavor// of Python you want by typing the activation command:
+    - Choose which environment you want by typing the activation command:
-      * Python 2.7.x: ''conda activate cdatm_py2''
       * Python 3.x: ''conda activate cdatm_py3''
+      * You can use ''conda env list'' to find out which environments are available, but the one you most likely want is ''cdatm_py3''
+    - If you need an (older) Python 2.7 environment, replace the steps above with:
+      * ''source ~jypeter/.conda3_19-06_jyp.sh''
+      * ''conda activate cdatm_py2''
     - Type ''python'' to start the interpreter
+    - If you need a list of the installed packages (and their version), use
+      * ''conda list''
+      * A partial list of what is installed is available in the [[other:uvcdat:cdat_conda:cdat_8_2_1#extra_packages_list|Extra packages list]]
     - Note: if you will have to use python regularly, you should add the ''source'' line above to your ''.cshrc'' /''.login'' file (//tcsh// users) or ''.bashrc''/''.profile'' (//bash// users)
       * **Do not** add the ''conda activate cdatm_pyN'' line to the shell config files, because this will create potential side-effects!\\ Only type the activation command in the terminal(s) where you will need to use this specific version of Python
@@ Line 50: / Line 56: @@
 <code> >ssh obelix
-Last login: Mon Jun  3 08:49:53 2019 from somewhere
+Last login: Wed Feb  1 09:56:29 2023 from somewhere
 bash-4.2$ which python
 /usr/bin/python
 bash-4.2$ source ~jypeter/.conda3_jyp.sh
 bash-4.2$ which python
 /usr/bin/python
-bash-4.2$ conda activate cdatm_py2
-(cdatm_py2) bash-4.2$ which python
+bash-4.2$ conda activate cdatm_py3
-/home/share/unix_files/cdat/miniconda3/envs/cdatm_py2/bin/python
-(cdatm_py2) bash-4.2$ python
+(cdatm_py3) bash-4.2$ which python
-Python 2.7.15 | packaged by conda-forge | (default, Feb 28 2019, 04:00:11)
+/home/share/unix_files/cdat/miniconda3_21-02/envs/cdatm_py3/bin/python
-[GCC 7.3.0] on linux2
+(cdatm_py3) bash-4.2$ python
+Python 3.8.8 | packaged by conda-forge | (default, Feb 20 2021, 16:22:27)
+[GCC 9.3.0] on linux
 Type "help", "copyright", "credits" or "license" for more information.
 >>> print('hello!')
@@ Line 123: / Line 135: @@
   - Learn Python by reading what you need on the [[other:python:jyp_steps|JYP's recommended steps for learning python]] page
+===== Using JupyterHub servers =====
+==== JupyterHub servers available for LSCE users ====
+  * [[https://intranet.lsce.ipsl.fr/informatique/en/jupyter/index.php|LSCE JupyterHub server information]]
+  * [[https://documentations.ipsl.fr/spirit/jupyterhub/documentation.html|IPSL JupyterHub server information]]
+  * You will **have to** be on the LSCE wired network, or use the [[https://intranet.lsce.ipsl.fr/informatique/en/intra-cea.php|LSCE VPN]], if you want to use the LSCE or IPSL JupyterHub servers !
+    * Otherwise, check the IPSL server documentation, if you want to try **accessing the IPSL server using an ssh tunnel**...
+==== Notebooks and jupyter servers documentation ====
+FIXME
+==== Using a non-standard kernel ====
+Note: a //kernel// is a specific python environment/distribution
+The LSCE jupyterhub server can be used to open //Notebooks// using pre-configured Python or R //kernels//. It is also possible to define additional per-user kernels by adding [[https://jupyter-client.readthedocs.io/en/stable/kernels.html#kernel-specs|kernel configuration directories]] in the user's home directory
+The steps below show how to add a specific [[other:python:starting#cdat|CDAT environment]] kernel, but can be easily adapted for using other kernels
+  - Initialize your (favorite) Python environment in a terminal, and (as usual) check which Python executable you are using\\ <code> >which python
+/usr/bin/python
+ >source ~jypeter/.conda3_21-02_jyp.csh
+ >conda activate cdatm_py3
+(cdatm_py3)  >which python
+/home/share/unix_files/cdat/miniconda3_21-02/envs/cdatm_py3/bin/python</code>
+  - Determine where the kernel configuration files of the **active** python are located\\ <code> >jupyter kernelspec list
+Available kernels:
+  python3     /home/share/unix_files/cdat/miniconda3_21-02/envs/cdatm19_nompi_py3/share/jupyter/kernels/python3</code>
+  - Copy the existing kernel directory that you have found to your home directory (more precisely to the ''~/.local/share/jupyter/kernels'' directory). You can use the existing kernel name (probably ''python3''), or use a new kernel name (using only ASCII letters, numbers,  ''-'' hyphen, ''.'' period and ''_'' underscore)\\ <code> >mkdir -p ~/.local/share/jupyter/kernels
+ >cp -pr /home/share/unix_files/cdat/miniconda3_21-02/envs/cdatm19_nompi_py3/share/jupyter/kernels/python3 ~/.local/share/jupyter/kernels
+ >cp -pr /home/share/unix_files/cdat/miniconda3_21-02/envs/cdatm19_nompi_py3/share/jupyter/kernels/python3 ~/.local/share/jupyter/kernels/my_favorite_kernel
+ >ls ~/.local/share/jupyter/kernels/
+my_favorite_kernel/    python3/
+</code>
+  - Check that the ''kernelspec list'' command now finds the new kernel(s)\\ <code> >jupyter kernelspec list
+Available kernels:
+  my_favorite_kernel    your_home_dir/.local/share/jupyter/kernels/my_favorite_kernel
+  python3               your_home_dir/.local/share/jupyter/kernels/python3</code>
+  - You can optionally change the value of the ''display_name'' parameter in the ''kernel.json'' configuration file if you want to customize the name that will appear in the jupyterhub kernel menus\\ <code> >ls your_home_dir/.local/share/jupyter/kernels/my_favorite_kernel
+kernel.json  logo-32x32.png  logo-64x64.png
+ >cat ~/.local/share/jupyter/kernels/my_favorite_kernel/kernel.json
+{
+ "argv": [
+  "/home/share/unix_files/cdat/miniconda3_21-02/envs/cdatm19_nompi_py3/bin/python",
+  "-m",
+  "ipykernel_launcher",
+  "-f",
+  "{connection_file}"
+ ],
+ "display_name": "My favorite Python kernel",
+ "language": "python"
+}
+</code>
+  - Refresh the ''https://jupyter.lsce.ipsl.fr/'' (if you are using the LSCE notebook server) page and the new kernel(s) should appear in the kernels' related menus
 ===== Python distributions available for LSCE users =====
@@ Line 133: / Line 207: @@
 ==== LSCE distribution ====
-  * Contact: the LSCE system administrators ([[help-lsce@lsce.ipsl.fr]])
+  * Contact: the [[other:newppl:starting#getting_help_from_the_lsce_system_administrators|LSCE system administrators]]
   * Where: //obelix// interactive servers and //obelix// cluster at LSCE
   * Initialization type: [[other:newppl:starting#using_module_to_access_optional_programs|module based]] + //conda//
-  * What's installed: type ''conda list'' after initializing the LSCE distribution
+  * What's installed: type ''conda list'' after initializing on of the LSCE distributions
 <code>
- > module avail
+$ module avail python
-[...]
+python/2.7      python/2-ramces python/3.9      python/intel
-ARTIC/3.6         grib_api/1.14      netcdf/3           python/2.7.5
+python/2.7.5    python/3        python/4artic
-batch_env          grib_api/1.14.0    netcdf/4           python/3.6
-[...]
-glost/0.3.1        ncview/2.1.7       python/2.7
-[...]
- > module load python/2.7
+$ module load python/3.9
- > which python
+$ which python
-/usr/local/install/python-2.7/bin/python
+/usr/local/install/python-3.9/bin/python
- > python
+$ python
-Python 2.7.15 |Anaconda, Inc.| (default, Oct 10 2018, 21:32:13)
+Python 3.9.15 (main, Nov 24 2022, 14:31:59)
-[GCC 7.3.0] on linux2
+[GCC 11.2.0] :: Anaconda, Inc. on linux
 Type "help", "copyright", "credits" or "license" for more information.
 >>>
@@ Line 161: / Line 231: @@
 ==== CDAT ====
-[[https://uvcdat.llnl.gov/|CDAT]] (//Community Data Analysis Tools//) is a powerful and complete front-end to a rich set of visual-data exploration and analysis capabilities well suited for **climate** data analysis problems, and is recommended by JYP.
+[[https://cdat.llnl.gov/|CDAT]] (//Community Data Analysis Tools//) is a powerful and complete front-end to a rich set of visual-data exploration and analysis capabilities well suited for **climate** data analysis problems. CDAT is **recommended by JYP**.
 CDAT is **available for Mac, Linux and Windows 10 (Windows 10 + [[other:win10wsl|Windows Subsystem for Linux, and Ubuntu]])**.
@@ Line 167: / Line 237: @@
   * Contact: Jean-Yves Peterschmitt (//JYP//) @ LSCE
   * Where: //obelix// interactive servers and cluster at LSCE, //ciclad// interactive servers and cluster at IPSL, irene @ TGCC
-  * Initialization type: conda based or [[other:newppl:starting#using_module_to_access_optional_programs|module based]]
+  * Initialization type: conda based
   * What's installed: type ''conda list'' after initializing a specific CDAT distribution
@@ Line 175: / Line 245: @@
 ^  CDAT version  ^  JYP\\ version  ^  python\\ version  ^  Availability?  ^  Available packages ^  Installation notes  ^
-|  **8.1**  |  18  |  2.7.15\\ 3.6.7  |  LSCE, ciclad  |  {{ :other:uvcdat:cdat_conda:cdat-8.1_py2_list_190307.txt |Default CDAT packages}}\\ [[other:uvcdat:cdat_conda:cdat_8_1#extra_packages_list|Extra packages]]  |  [[other:uvcdat:cdat_conda:cdat_8_1|8.1 notes]]  |
+|  **8.2.1**  |  19  |  3.8.8  |  LSCE  |  {{  :other:uvcdat:cdat_conda:cdat_8.2.1_nompi_21-03-05.txt |Default CDAT 8.2.1 packages}}\\ [[other:uvcdat:cdat_conda:cdat_8_2_1#extra_packages_list|Extra packages]]  |  [[other:uvcdat:cdat_conda:cdat_8_2_1|8.2.1 notes]]  |
+|  **8.1**  |  18  |  2.7.15\\ 3.6.7  |  LSCE, ciclad  |  {{:other:uvcdat:cdat_conda:cdat-8.1_py2_list_190307.txt|Default CDAT 8.1 packages}}\\ [[other:uvcdat:cdat_conda:cdat_8_1#extra_packages_list|Extra packages]]  |  [[other:uvcdat:cdat_conda:cdat_8_1|8.1 notes]]  |
 == Initialization ==
-Read the [[https://wiki.lsce.ipsl.fr/pmip3/doku.php/other:python:starting#ultra_quick-start_on_the_lsce_servers|Ultra quick-start on the LSCE servers]] above
+Read the [[https://wiki.lsce.ipsl.fr/pmip3/doku.php/other:python:starting#ultra_quick-start_on_the_interactive_lsce_servers|Ultra quick-start on the LSCE servers]] above
 Note: on the ciclad cluster, use ''source ~jypmce/.conda3_jyp.sh'' to initialize conda
 === CDAT at TGCC ===
+Note: TGCC also supports its own [[#tgcc_distribution|TGCC distribution]], that may be more up-to-date, if you don't need specific CDAT modules
 CDAT **8.0** is installed at TGCC and can be initialized with:
@@ Line 189: / Line 262: @@
   * Python 3: ''module load flavor/cdat/python3 cdat''
-=== A common error ===
+=== A common CDAT-related error ===
 Note: if you get an error when importing //cdms2// or //vcs// it means that either you have forgotten to initialize CDAT, or that something went wrong during the initialization. In both cases, you are either still using the default python installed on your system, or another (non-CDAT) python distribution!
@@ Line 235: / Line 308: @@
 </code>
-==== Canopy ====
-[[https://www.enthought.com/products/canopy/|Enthought Canopy]] is //a Scientific and Analytic Python Deployment with Integrated Analysis Environment// provided by [[https://www.enthought.com/|Enthought]]. It used to be called EPD (//Enthought Python Distribution//)
-Canopy is **available for Windows, Mac and Linux**. You can download the free //Canopy Express// that will already provide many extensions. If you are entitled to use //Canopy Academic//, login from inside Canopy Express and download the extra modules you need
-==== Anaconda ====
-[[https://www.continuum.io/why-anaconda|Anaconda]] is a distribution similar to [[#canopy]] provided by [[https://www.continuum.io/|Continuum Analytics]].
-Anaconda is **available for Windows, Mac and Linux**
-Note: Anaconda provides and uses ''conda'' for its installation. Since CDAT is now also installed and maintained with ''conda'', you can read the [[other:uvcdat:conda_notes|Installing and maintaining UV-CDAT with conda]] page for more information, even if you are not going to use CDAT
 ===== ipython =====
@@ Line 261: / Line 320: @@
   * [[https://damontallen.github.io/IPython-quick-ref-sheets/|Ipython-quick-ref-sheets]]
+==== LSCE jupyter notebook server ====
+The different Python distributions available on the LSCE servers (interactive servers and cluster) can also be used on the [[https://intranet.lsce.ipsl.fr/informatique/en/jupyter/index.php|LSCE jupyter notebook server]]:
+  * If your computer is on the LSCE wired network
+  * Or if you are using the [[https://intranet.lsce.ipsl.fr/informatique/fr/vpn-forticlient.php|LSCE VPN]]
 ==== ipython notebook ====
+<WRAP center round tip 60%>
+If you want to use ipython notebooks at LSCE, you should **use the [[other:python:starting#lsce_jupyter_notebook_server|LSCE notebook server]]** rather than trying to use (and probably overloading) the interactive servers!
+</WRAP>
 <note warning>FIXME Add a link to the new notebook page</note>
-The ipython notebook is a way to interact with python (and other supported interpreted languages) inside a web browser. You can mix //cells// with python commands, cells with the output of the python commands (possibly graphics generated by the commands), and text (using some wiki-like rich text format). This interactive web page, aka notebook, can be saved in a ''my_notebook.ipynb'' file and re-used later (e.g. the notebook provided in the [[other:python:jyp_steps#part_1|introduction to Python, part 1]]).
+[[https://www.dataquest.io/blog/jupyter-notebook-tutorial/|Jupyter Notebook for Beginners: A Tutorial]]
+The **ipython notebook is a way to interact with python (and other supported interpreted languages) inside a web browser**. You can mix //cells// with python commands, cells with the output of the python commands (possibly graphics generated by the commands), and text (using some wiki-like rich text format). This interactive web page, aka notebook, can be saved in a ''my_notebook.ipynb'' file and re-used later (e.g. the notebook provided in the [[other:python:jyp_steps#part_1|introduction to Python, part 1]]).
 Starting the notebook server: ''ipython notebook''
-<note warning>**Warning!** When you start the notebook server, you will start a python process and a web browser, and a new python process will be started each time you open a new notebook.
+<note warning>
+  * **Warning!** When you start the notebook server, you will start a python process and a web browser, and a new python process will be started each time you open a new notebook.
-Please make sure that you shutdown cleanly each notebook and the server when you are finished, in order not to clutter the local computer or remote server with lots of python processes (especially a server shared by many users!).
+  * Please make sure that you **shutdown cleanly each notebook and the server when you are finished**, in order not to clutter the local computer or remote server with lots of python processes (especially an interactive server shared by many users!).
-You may want to [[other:newppl:starting#determining_the_load_of_a_linux_server|use the 'top' command]] in order to monitor what is happening</note>
+  * You may want to [[other:newppl:starting#determining_the_load_of_a_linux_server|use the 'top' command]] in order to monitor what is happening on the interactive server where you are using your notebook server
+</note>
 /* standard page footer */