Differences

This shows you the differences between two versions of the page.

--- other:python:starting [2019/05/17 08:56]
jypeter [Initialization] Improved
+++ other:python:starting [2019/06/17 07:03]
jypeter [CDAT] improved
@@ Line 12: / Line 12: @@
 In order to start working with Python, you need to have a //Python distribution// installed on your local computer or on the remote Linux server(s) you work on. A //distribution// provides a //Python interpreter//, and Python extensions (aka Python //modules// or //packages//). You may have several distributions installed on your computer and you need to know how to initialize them, and which one you are using at a given time (type ''which python'' on Linux to determine where the python executable is located)
-If you are using a Linux computer or a Mac, you should already have a **default python** installed. The following example shows where the python interpreter is installed on the //obelix// LSCE servers (if it's in ''/usr/bin'', it's the default python) and which version it is (example below: version //2.7.5// compiled in April 2019)
+If you are using a Linux computer or a Mac, you should already have a **default python** installed. The following example shows where the python interpreter is installed on the //obelix// LSCE servers (if it's in ''/usr/bin'', it's the //default// python) and which version it is (example below: version //2.7.5// compiled in April 2019)
 <code># Which is the current python (e.g. where is it located)?
@@ Line 34: / Line 34: @@
 ==== Initialization ====
-  - Start a terminal on an //obelix// server
+  - Start a terminal on an //obelix// server ([[https://wiki.lsce.ipsl.fr/pmip3/doku.php/other:newppl:starting#which_linux_servers_should_you_use|more info]])
   - Type ''python'' to access the //default Python 2// interpreter available on the servers...
-  - ...or follow the steps below to use the more complete //CDAT// distribution.\\ Remember that **you can exit the interpreter by typing CTRL-D** (or ''quit()'' or ''exit()'')
+  - ...or **follow the steps below to use the more complete //CDAT// distribution maintained by JYP**.\\ Remember that **you can exit the interpreter by typing CTRL-D** (or ''quit()'' or ''exit()'')
-    - Determine if you are using a [[other:newppl:starting#which_shell_are_you_using|bash or tcsh shell]], then type the following command to initialize //conda//
+    - Determine if you are using a [[other:newppl:starting#which_shell_are_you_using|bash or tcsh shell]] (all the new LSCE accounts use the //bash// shell), then type the following command to //initialize conda// (//conda// helps you manage your python environment)
       * In a //bash// shell: ''source ~jypeter/.conda3_jyp.sh''
       * In a //tcsh// shell: ''source ~jypeter/.conda3_jyp.csh''
-    - Choose which //flavor// of Python you want by typing:
+    - Choose which //flavor// of Python you want by typing the activation command:
       * Python 2.7.x: ''conda activate cdatm_py2''
       * Python 3.x: ''conda activate cdatm_py3''
     - Type ''python'' to start the interpreter
     - Note: if you will have to use python regularly, you should add the ''source'' line above to your ''.cshrc'' /''.login'' file (//tcsh// users) or ''.bashrc''/''.profile'' (//bash// users)
-      * **Do not** add the ''conda activate cdatm_pyN'' line to the shell config files, because this will create potential side-effects!\\ Only type this activation line in the terminals where you will need to use this specific version of Python
+      * **Do not** add the ''conda activate cdatm_pyN'' line to the shell config files, because this will create potential side-effects!\\ Only type the activation command in the terminal(s) where you will need to use this specific version of Python
+=== Example ===
+<code> >ssh obelix
+Last login: Mon Jun  3 08:49:53 2019 from somewhere
+bash-4.2$ which python
+/usr/bin/python
+bash-4.2$ source ~jypeter/.conda3_jyp.sh
+bash-4.2$ which python
+/usr/bin/python
+bash-4.2$ conda activate cdatm_py2
+(cdatm_py2) bash-4.2$ which python
+/home/share/unix_files/cdat/miniconda3/envs/cdatm_py2/bin/python
+(cdatm_py2) bash-4.2$ python
+Python 2.7.15 | packaged by conda-forge | (default, Feb 28 2019, 04:00:11)
+[GCC 7.3.0] on linux2
+Type "help", "copyright", "credits" or "license" for more information.
+>>> print('hello!')
+hello!
+>>>   # Type CTRL-D to exit the python interpreter
+</code>
 ==== Useful keyboard shortcuts ====
@@ Line 106: / Line 127: @@
 This section will help you choose a distribution in the big Python ecosystem (many distributions, python version 2.7.* or 3.*, ...) on some of the servers used by LSCE users. You should use a distribution that is already available near your data (e.g. do the computation on a server near your data, do not move/duplicate the data!) and try to identify who is maintaining it, if you need help or additional packages.
-Only install a distribution yourself if you need it on your local computer (desktop or laptop), or if you need to install some modules that can't be installed by the contacts listed below. A python distribution will require several Gb of disk space, so do not install it in a backed up //home// directory!
+Only install a distribution yourself if you need it on your local computer (desktop or laptop), or if you need to install some modules that can't be installed by the contacts listed below. A python distribution will require several Gb of disk space, so do not install it in your backed up //home// directory!
 You can use either //Python 2// or //Python 3//. Most packages are now available in both versions, but you should make sure that the most important package/s you need is/are available in the selected Python version. You can check the [[https://wiki.lsce.ipsl.fr/pmip3/doku.php/other:python:jyp_steps#python_27_vs_python_3|differences between both versions]] and try to write scripts that will work in both versions!
@@ Line 113: / Line 134: @@
   * Contact: the LSCE system administrators ([[help-lsce@lsce.ipsl.fr]])
-  * Where: //obelix// interactive servers and cluster at LSCE
+  * Where: //obelix// interactive servers and //obelix// cluster at LSCE
   * Initialization type: [[other:newppl:starting#using_module_to_access_optional_programs|module based]] + //conda//
   * What's installed: type ''conda list'' after initializing the LSCE distribution
@@ Line 134: / Line 155: @@
 Python 2.7.15 |Anaconda, Inc.| (default, Oct 10 2018, 21:32:13)
 [GCC 7.3.0] on linux2
-Type "help", "copyright", "credits" or "license" for more information.
->>>
-</code>
-==== TGCC distribution ====
-  * Contact: the TGCC hotline ([[hotline.tgcc@cea.fr]])
-  * Where: CEA TGCC
-  * Initialization type: [[other:newppl:starting#using_module_to_access_optional_programs|module based]]
-  * What's installed: it depends...
-<code># Get the default version of the Python based modules
-$ module avail -t -d | egrep '(python|cdat)'
-flavor/cdat/standard(default)
-flavor/nest/python2(default)
-flavor/pytorch/python2(default)
-flavor/tensorflow/gpu_python2(default)
-intelpython2/2019.0(default)
-intelpython3/2019.0(default)
-python/2.7.14(default)
-python3/3.6.4(default)
-cdat/8.0(default)
-$ module load python
-[...]
-load module python/2.7.14 (Python)
-$ which python
-/ccc/products/python-2.7.14/intel--17.0.4.196__openmpi--2.0.2/default/bin/python
-$ python
-Python 2.7.14 (default, Jan 11 2018, 16:43:59)
-[GCC 4.8.5] on linux2
 Type "help", "copyright", "credits" or "license" for more information.
 >>>
@@ Line 188: / Line 176: @@
 ^  CDAT version  ^  JYP\\ version  ^  python\\ version  ^  Availability?  ^  Available packages ^  Installation notes  ^
 |  **8.1**  |  18  |  2.7.15\\ 3.6.7  |  LSCE, ciclad  |  {{ :other:uvcdat:cdat_conda:cdat-8.1_py2_list_190307.txt |Default CDAT packages}}\\ [[other:uvcdat:cdat_conda:cdat_8_1#extra_packages_list|Extra packages]]  |  [[other:uvcdat:cdat_conda:cdat_8_1|8.1 notes]]  |
+== Initialization ==
+Read the [[https://wiki.lsce.ipsl.fr/pmip3/doku.php/other:python:starting#ultra_quick-start_on_the_lsce_servers|Ultra quick-start on the LSCE servers]] above
+Note: on the ciclad cluster, use ''source ~jypmce/.conda3_jyp.sh'' to initialize conda
 === CDAT at TGCC ===
+Note: TGCC also supports its own [[#tgcc_distribution|TGCC distribution]], that may be more up-to-date, if you don't need specific CDAT modules
 CDAT **8.0** is installed at TGCC and can be initialized with:
@@ Line 195: / Line 191: @@
   * Python 3: ''module load flavor/cdat/python3 cdat''
-=== The basics of CDAT initialization ===
+=== A common CDAT-related error ===
-The way you initialize CDAT depends on:
-  * which **server** you want to use it on
-  * which **shell** you are using in your terminals: //tcsh// or other shells
-    * Read [[other:newppl:starting#which_shell_are_you_using|Which shell are you using?]] if you are not sure
-    * Warning: the shell in a (batch) script may be different from your interactive shell!
-  * which **version** of CDAT you want to use...
 Note: if you get an error when importing //cdms2// or //vcs// it means that either you have forgotten to initialize CDAT, or that something went wrong during the initialization. In both cases, you are either still using the default python installed on your system, or another (non-CDAT) python distribution!
@@ Line 215: / Line 204: @@
 /usr/bin/python</code>
-=== conda-based versions of CDAT ===
+==== TGCC distribution ====
-<WRAP center round important 60%>
+  * Contact: the TGCC hotline ([[hotline.tgcc@cea.fr]])
-\\ The steps detailed in this section are for [[other:uvcdat:cdat_conda:|UV-CDAT 2.8.0 and later]]
+  * Where: CEA TGCC
-</WRAP>
+  * Initialization type: [[other:newppl:starting#using_module_to_access_optional_programs|module based]]
+  * What's installed: it depends...
-Note: if you need to maintain a **local** version on **your** machine, you can read [[other:uvcdat:conda_notes|Installing and maintaining CDAT with conda]] and the [[other:uvcdat:cdat_conda:index#versions|versions specific notes]].
+<code># Get the default version of the Python based modules
+$ module avail -t -d | egrep '(python|cdat)'
+flavor/cdat/standard(default)
+flavor/nest/python2(default)
+flavor/pytorch/python2(default)
+flavor/tensorflow/gpu_python2(default)
+intelpython2/2019.0(default)
+intelpython3/2019.0(default)
+python/2.7.14(default)
+python3/3.6.4(default)
+cdat/8.0(default)
-== Configuring .bashrc ==
+$ module load python
+[...]
-You should **add** the following line at the end of your ''~/.bashrc'' configuration file (create the ''~/.bashrc'' file if it does not exist yet). This will basically make sure that your shell can find the ''conda'' executable it needs for initializing CDAT, and define a ''wp'' alias that you can use to find where the ''python'' executable is located
+load module python/2.7.14 (Python)
-^  Server  ^  Line to add to ''.bashrc''  ^
-|  **LSCE**  | ''source ~jypeter/.conda_jyp.sh''\\ \\ Note:{{ :other:python:conda_jyp.txt |conda_jyp}} content |
-|  **ciclad**  | ''source ~jypmce/.conda_jyp.sh'' |
-Note: when you add the specified line to your ''bashrc'' file, you will then always get the python supplied by conda, instead of the system's python, when you use ''bash''. If you don't like this permanent behavior, just type ''source ~jyp_login_on_this_server/.conda_jyp.sh'' in the bash shell where you want to use conda+CDAT, rather than adding it to the ''.bashrc'' file...
-== Initializing CDAT ==
-<WRAP center round tip 60%>
-When you initialize CDAT, it will be available only in the **current** window/shell. In the other windows, you still get whatever your default python is. This should minimize potential side-effects
-</WRAP>
-  - If [[other:newppl:starting#which_shell_are_you_using|your shell]] is not //bash//, start a bash shell by typing ''bash''
-    * You can later go back to your previous shell by typing ''exit'' or ''CTRL-D''
-  - Activate the //conda environment// that uses the version of UV-CDAT you want to use
-    * ''source activate conda_env_name''
-    * Use the table below to determine which environments are available on which server
-  - Use CDAT !
-^  UV-CDAT version  ^  Server  ^  Environment name\\ and activation line  ^  Packages\\ (''conda list'')  ^
-|  [[:other:uvcdat:cdat_conda:cdat_2_8_0|2.8.0]]  |  **LSCE**\\ **ciclad**  |**cdatm14** (''source activate cdatm14'')  |  {{ :other:uvcdat:cdat_conda:conda_list_lsce_cdatm14_170310.txt |cdatm14 installed packages}}  |
-|  [[:other:uvcdat:cdat_conda:cdat_2_10|2.10]]  |  **LSCE**  |**cdatm15** (''source activate cdatm15'')  |  {{ :other:uvcdat:cdat_conda:uvcdat-2.10_list_170721.txt |cdatm14 installed packages}}  |
-|  **cdatm** will always point to the latest stable version  ||||
-|  //latest stable version//  |  **LSCE**  |**cdatm** (''source activate cdatm'')  |  ''conda list''  |
+$ which python
+/ccc/products/python-2.7.14/intel--17.0.4.196__openmpi--2.0.2/default/bin/python
+$ python
+Python 2.7.14 (default, Jan 11 2018, 16:43:59)
+[GCC 4.8.5] on linux2
+Type "help", "copyright", "credits" or "license" for more information.
+>>>
+</code>
 ==== Canopy ====
@@ Line 269: / Line 250: @@
 Anaconda is **available for Windows, Mac and Linux**
-Note: Anaconda provides and uses ''conda'' for its installation. Since UV-CDAT is now also installed and maintained with ''conda'', you can read the [[other:uvcdat:conda_notes|Installing and maintaining UV-CDAT with conda]] page for more information, even if you are not going to use UV-CDAT
+Note: Anaconda provides and uses ''conda'' for its installation. Since CDAT is now also installed and maintained with ''conda'', you can read the [[other:uvcdat:conda_notes|Installing and maintaining UV-CDAT with conda]] page for more information, even if you are not going to use CDAT
 ===== ipython =====
@@ Line 275: / Line 256: @@
 ==== ipython interpreter ====
-The //ipython// interpreter provides more options and commands than the standard python interpreter, but takes more time to load. If you are going to develop by starting and exiting the interpreter many times, it's faster to use the standard interpreter
+The //ipython// interpreter provides more options and commands than the standard python interpreter, but takes more time to start. If you are going to develop by starting and exiting the interpreter many times, it's faster to use the standard interpreter
 Starting ipython: ''ipython''
@@ Line 283: / Line 264: @@
 ==== ipython notebook ====
+<note warning>FIXME Add a link to the new notebook page</note>
 The ipython notebook is a way to interact with python (and other supported interpreted languages) inside a web browser. You can mix //cells// with python commands, cells with the output of the python commands (possibly graphics generated by the commands), and text (using some wiki-like rich text format). This interactive web page, aka notebook, can be saved in a ''my_notebook.ipynb'' file and re-used later (e.g. the notebook provided in the [[other:python:jyp_steps#part_1|introduction to Python, part 1]]).

PMIP3 wiki

User Tools

Site Tools

Differences

Page Tools