PMIP3 wiki

User Tools

Site Tools

You are here: index » other » uvcdat » cdat_conda » cdat_8_0_py2
other:uvcdat:cdat_conda:cdat_8_0_py2

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Next revision		Previous revision
Next revisionBoth sides next revision
other:uvcdat:cdat_conda:cdat_8_0_py2 [2018/11/07 18:08] – created jypeterother:uvcdat:cdat_conda:cdat_8_0_py2 [2018/11/08 17:10] – jypeter
Line 16: Line 16:
 ===== Installation with Miniconda3 ===== ===== Installation with Miniconda3 =====
  
-We have installed a new version of Miniconda3, based on the old [[other:uvcdat:conda_notes#installing_miniconda|installing miniconda]] instructions+==== Installing Miniconda3 ==== 
  
- * Follow the [[https://conda.io/docs/install/quick.html|Quick install]] steps to install Miniconda+We have installed a new version of Miniconda3, by following the existing [[other:uvcdat:conda_notes#installing_miniconda|installing miniconda]] instructions 
 + 
 +  * Follow the [[https://conda.io/docs/install/quick.html|Quick install]] steps to install Miniconda
     * Get the [[https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh|Python 3.7 Linux 64-bit (bash installer)]] (unless there is a [[https://conda.io/miniconda.html|more recent version?]])     * Get the [[https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh|Python 3.7 Linux 64-bit (bash installer)]] (unless there is a [[https://conda.io/miniconda.html|more recent version?]])
       * ''cd /home/scratch01/jypeter'' (some temporary storage space for the installer)       * ''cd /home/scratch01/jypeter'' (some temporary storage space for the installer)
Line 44: Line 46:
 12K     x86_64-conda_cos6-linux-gnu</code> 12K     x86_64-conda_cos6-linux-gnu</code>
         * Accept the automatic update of the //PATH// variable in ''.bashrc'' (note: the installer automatically creates a backup of ''.bashrc'')         * Accept the automatic update of the //PATH// variable in ''.bashrc'' (note: the installer automatically creates a backup of ''.bashrc'')
-          * ''# added by Miniconda2 4.3.21 installer''\\ ''export PATH="/home/share/unix_files/cdat/miniconda2/bin:$PATH"'' +          * ''# added by Miniconda3 installer''\\ ''export PATH="/home/share/unix_files/cdat/miniconda3/bin:$PATH"'' 
-    * Open a new terminal (if you use bash) or type ''bash'' and check if the python from the conda distribution has been correctly installed +    * Open a new terminal (if you use bash) or open a new terminal and type ''bash'' 
-      * Update and clean the new installation +      * check if the python from the conda distribution has been correctly installed\\ <code> > bash 
-        * ''conda update %%--%%all'' +$ which python 
-        * ''conda clean %%--%%all'' +/home/share/unix_files/cdat/miniconda3/bin/python 
-      * <code>bash-4.1$ which python +$ which conda 
-/home/share/unix_files/cdat/miniconda2/bin/python +/home/share/unix_files/cdat/miniconda3/bin/conda</code> 
-bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2 +      * Update and clean the new installation\\ <code>$ conda update --all 
-114M    /home/share/unix_files/cdat/miniconda2 +$ du -sh /home/share/unix_files/cdat/miniconda3 
-bash-4.1$ conda list +515M    /home/share/unix_files/cdat/miniconda3 
-# packages in environment at /home/share/unix_files/cdat/miniconda2:+$ conda clean --all 
 +$ du -sh /home/share/unix_files/cdat/miniconda3 
 +270M    /home/share/unix_files/cdat/miniconda3 
 +$ conda list 
 +# packages in environment at /home/share/unix_files/cdat/miniconda3:
 # #
-cffi                      1.9.1                    py27_0 +# Name                    Version                   Build  Channel
-conda                     4.3.13                   py27_0 +
-conda-env                 2.6.0                         0 +
-cryptography              1.7.1                    py27_0+
 [...] [...]
 +python                    3.7.1                h0371630_3
 +readline                  7.0                  h7b6447c_5
 +requests                  2.20.0                   py37_0
 +[...]</code>
 +  * Remove the miniconda3 initialization lines that were automatically added at the end of ''.bashrc'' and put them (and other useful commands) in a special initialization file (see ''~jypeter/.conda3_jyp.sh'' below), that can be sourced only when we actually want to use conda and CDAT\\ <code>bash-4.2$ which python
 +/usr/bin/python
 +$ cat ~jypeter/.conda3_jyp.sh
 +# Conda initialization by JYP
 +#
 +# Use this for working with conda and CDAT centrally managed by JYP
 +#
 +# More details in:
 +#   https://wiki.lsce.ipsl.fr/pmip3/doku.php/other:python:starting#conda-based_v                                    ersions_of_uv-cdat
 +#   https://wiki.lsce.ipsl.fr/pmip3/doku.php/other:uvcdat:conda_notes
 +#
 +# Jean-Yves Peterschmitt - LSCE - 11/2018
  
-<note important>\\ We used the [[other:uvcdat:cdat_conda:cdat_2_8_0#downgrading_conda|recommended downgraded version of conda]] that was available after installing UV-CDAT 2.8</note>+export PATH="/home/share/unix_files/cdat/miniconda3/bin:$PATH"
  
-DIsk space used before 2.10 installation (after running ''conda clean --all'')+# Use the alias below to easily determine where your python 
 +# interpreter is located 
 +alias wp="which python"
  
-<code> > du -sh /home/share/unix_files/cdat/miniconda2 +# Where are ALL the python interpreters in the search path 
-6.5G /home/share/unix_files/cdat/miniconda2+alias wpa="which -a python"
  
- > du -sh /home/share/unix_files/cdat/miniconda2/*  +# The end 
-4.4M /home/share/unix_files/cdat/miniconda2/bin +$ source ~jypeter/.conda3_jyp.sh 
-4.0K /home/share/unix_files/cdat/miniconda2/conda-bld +$ wp 
-272K /home/share/unix_files/cdat/miniconda2/conda-meta +/home/share/unix_files/cdat/miniconda3/bin/python 
-6.4G /home/share/unix_files/cdat/miniconda2/envs +</code>
-20K /home/share/unix_files/cdat/miniconda2/etc +
-3.8M /home/share/unix_files/cdat/miniconda2/include +
-99M /home/share/unix_files/cdat/miniconda2/lib +
-4.0K /home/share/unix_files/cdat/miniconda2/LICENSE.txt +
-76K /home/share/unix_files/cdat/miniconda2/pkgs +
-376K /home/share/unix_files/cdat/miniconda2/share +
-364K /home/share/unix_files/cdat/miniconda2/ssl +
- +
- >du -sh /home/share/unix_files/cdat/miniconda2/envs/* +
-2.8G /home/share/unix_files/cdat/miniconda2/envs/cdatm14 +
-1.7G /home/share/unix_files/cdat/miniconda2/envs/keras +
-2.0G /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0</code> +
- +
-Actual installation+
  
-<code>bash-4.1$ conda create -n uvcdat-2.10 -c conda-forge -c uvcdat uvcdat=2.10+==== Installing CDAT 8.0 ====
  
 +<code>conda create -n cdat-8.0_py2 -c cdat/label/v80 -c conda-forge -c cdat python=2.7 cdat
 # Generate the list of installed packages # Generate the list of installed packages
-conda list -n uvcdat-2.10 > /home/scratch01/jypeter/uvcdat-2.10_list_170721.txt</code>+conda list -n cdat-8.0_py2 > /home/scratch01/jypeter/cdat-8.0_py2_list_181108.txt</code>
  
-List of installed packages: {{:other:uvcdat:cdat_conda:uvcdat-2.10_list_170721.txt}}+List of installed packages: {{:other:uvcdat:cdat_conda:cdat-8.0_py2_list_181108.txt}}
  
-Disk space after 2.10 installation+Disk space after 8.0 installation
  
-<code>bash-4.1$ conda env list +<code>$ du -sh /home/share/unix_files/cdat/miniconda3 
-cdatm14                  /home/share/unix_files/cdat/miniconda2/envs/cdatm14 +4.3G    /home/share/unix_files/cdat/miniconda3 
-keras                    /home/share/unix_files/cdat/miniconda2/envs/keras +$ du -sh /home/share/unix_files/cdat/miniconda3/* 
-uvcdat-2.10              /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.10 +15M     /home/share/unix_files/cdat/miniconda3/bin 
-uvcdat-2.8.0             /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0 +2.4M    /home/share/unix_files/cdat/miniconda3/compiler_compat 
-root                  *  /home/share/unix_files/cdat/miniconda2+4.0K    /home/share/unix_files/cdat/miniconda3/conda-bld 
 +3.7M    /home/share/unix_files/cdat/miniconda3/conda-meta 
 +2.8G    /home/share/unix_files/cdat/miniconda3/envs 
 +28K     /home/share/unix_files/cdat/miniconda3/etc 
 +4.9M    /home/share/unix_files/cdat/miniconda3/include 
 +156M    /home/share/unix_files/cdat/miniconda3/lib 
 +8.0K    /home/share/unix_files/cdat/miniconda3/LICENSE.txt 
 +1.4G    /home/share/unix_files/cdat/miniconda3/pkgs 
 +1.3M    /home/share/unix_files/cdat/miniconda3/share 
 +24K     /home/share/unix_files/cdat/miniconda3/ssl 
 +12K     /home/share/unix_files/cdat/miniconda3/x86_64-conda_cos6-linux-gnu 
 +$ du -sh /home/share/unix_files/cdat/miniconda3/envs/* 
 +2.8G    /home/share/unix_files/cdat/miniconda3/envs/cdat-8.0_py2</code>
  
- > du -sh /home/share/unix_files/cdat/miniconda2 
-11G /home/share/unix_files/cdat/miniconda2 
  
- > du -sh /home/share/unix_files/cdat/miniconda2/*  +===== Cloning cdat to add specific packages for LSCE =====
-4.4M /home/share/unix_files/cdat/miniconda2/bin +
-4.0K /home/share/unix_files/cdat/miniconda2/conda-bld +
-272K /home/share/unix_files/cdat/miniconda2/conda-meta +
-9.2G /home/share/unix_files/cdat/miniconda2/envs +
-20K /home/share/unix_files/cdat/miniconda2/etc +
-3.8M /home/share/unix_files/cdat/miniconda2/include +
-99M /home/share/unix_files/cdat/miniconda2/lib +
-4.0K /home/share/unix_files/cdat/miniconda2/LICENSE.txt +
-1.4G /home/share/unix_files/cdat/miniconda2/pkgs +
-376K /home/share/unix_files/cdat/miniconda2/share +
-364K /home/share/unix_files/cdat/miniconda2/ssl +
- +
- > du -sh /home/share/unix_files/cdat/miniconda2/envs/* +
-2.8G /home/share/unix_files/cdat/miniconda2/envs/cdatm14 +
-1.7G /home/share/unix_files/cdat/miniconda2/envs/keras +
-2.8G /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.10 +
-2.0G /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0 +
-</code> +
- +
-Disk space after removing the keras environment that we don't need any more +
- +
-<code>jypeter@asterix2 - ...cdat - 67 >du -sh /home/share/unix_files/cdat/miniconda2 +
-9.0G /home/share/unix_files/cdat/miniconda2 +
-jypeter@asterix2 - ...cdat - 68 >du -sh /home/share/unix_files/cdat/miniconda2/* +
-4.4M /home/share/unix_files/cdat/miniconda2/bin +
-4.0K /home/share/unix_files/cdat/miniconda2/conda-bld +
-272K /home/share/unix_files/cdat/miniconda2/conda-meta +
-7.5G /home/share/unix_files/cdat/miniconda2/envs +
-20K /home/share/unix_files/cdat/miniconda2/etc +
-3.8M /home/share/unix_files/cdat/miniconda2/include +
-99M /home/share/unix_files/cdat/miniconda2/lib +
-4.0K /home/share/unix_files/cdat/miniconda2/LICENSE.txt +
-1.4G /home/share/unix_files/cdat/miniconda2/pkgs +
-376K /home/share/unix_files/cdat/miniconda2/share +
-364K /home/share/unix_files/cdat/miniconda2/ssl +
-jypeter@asterix2 - ...cdat - 69 >du -sh /home/share/unix_files/cdat/miniconda2/envs/* +
-2.8G /home/share/unix_files/cdat/miniconda2/envs/cdatm14 +
-2.8G /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.10 +
-2.0G /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0</code> +
- +
-Disk space used in <wrap em>a brand new directory hierarchy</wrap> (''cdat/miniconda2b'', on the obelix servers) +
- +
-<code>bash-4.2$ conda env list +
-uvcdat-2.10              /home/share/unix_files/cdat/miniconda2b/envs/uvcdat-2.10 +
-root                  *  /home/share/unix_files/cdat/miniconda2b +
- +
-bash-4.2$ du -sh /home/share/unix_files/cdat/miniconda2b +
-4.3G /home/share/unix_files/cdat/miniconda2b +
- +
-bash-4.2$ du -sh /home/share/unix_files/cdat/miniconda2b/* +
-4.4M /home/share/unix_files/cdat/miniconda2b/bin +
-4.0K /home/share/unix_files/cdat/miniconda2b/conda-bld +
-280K /home/share/unix_files/cdat/miniconda2b/conda-meta +
-2.8G /home/share/unix_files/cdat/miniconda2b/envs +
-20K /home/share/unix_files/cdat/miniconda2b/etc +
-3.8M /home/share/unix_files/cdat/miniconda2b/include +
-100M /home/share/unix_files/cdat/miniconda2b/lib +
-8.0K /home/share/unix_files/cdat/miniconda2b/LICENSE.txt +
-1.4G /home/share/unix_files/cdat/miniconda2b/pkgs +
-16K /home/share/unix_files/cdat/miniconda2b/share +
-376K /home/share/unix_files/cdat/miniconda2b/ssl +
-</code> +
- +
- +
-===== Cloning uvcdat to add specific packages for LSCE =====+
  
 <WRAP center round tip 60%> <WRAP center round tip 60%>
-This section is about the creation of the **cdatm15** environment+This section is about the creation of the **cdatm17** environment
  
-Notes about [[https://wiki.lsce.ipsl.fr/pmip3/doku.php/other:python:starting#conda-based_versions_of_uv-cdat|actually using the cdatm15 conda-based python]]+Notes about [[https://wiki.lsce.ipsl.fr/pmip3/doku.php/other:python:starting#conda-based_versions_of_uv-cdat|actually using the cdatm17 conda-based python]]
 </WRAP> </WRAP>
  
  
-Note: using hard links, cloning a full environment //only// adds an extra 500M of disk space+Note: using hard links, cloning a full environment //only// adds an extra 582M of disk space
  
-<code>bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2 +<code>$ du -sh /home/share/unix_files/cdat/miniconda3/envs/* 
-9.0G /home/share/unix_files/cdat/miniconda2+2.8G    /home/share/unix_files/cdat/miniconda3/envs/cdat-8.0_py2 
 +$ conda create -n cdatm17_py2 --clone cdat-8.0_py2 
 +Source:      /home/share/unix_files/cdat/miniconda3/envs/cdat-8.0_py2 
 +Destination: /home/share/unix_files/cdat/miniconda3/envs/cdatm17_py2 
 +Packages: 226 
 +Files: 3
  
-bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2/envs/* 
-2.8G /home/share/unix_files/cdat/miniconda2/envs/cdatm14 
-2.8G /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.10 
-2.0G /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0 
- 
-bash-4.1$ conda create -n cdatm15 --clone uvcdat-2.10  
-Source:      /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.10 
-Destination: /home/share/unix_files/cdat/miniconda2/envs/cdatm15 
 [...] [...]
  
-bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2 +$ du -sh /home/share/unix_files/cdat/miniconda3/envs/* 
-9.5G /home/share/unix_files/cdat/miniconda2+2.8G    /home/share/unix_files/cdat/miniconda3/envs/cdat-8.0_py2 
 +582M    /home/share/unix_files/cdat/miniconda3/envs/cdatm17_py2
  
-bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2/envs/* 
-2.8G /home/share/unix_files/cdat/miniconda2/envs/cdatm14 
-2.8G /home/share/unix_files/cdat/miniconda2/envs/cdatm15 
-573M /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.10 
-2.0G /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0 
-</code> 
- 
-Disk space used after cloning, in <wrap em>in the  new directory hierarchy</wrap> (''cdat/miniconda2b'', on the obelix servers) 
- 
-<code>bash-4.2$ du -sh /home/share/unix_files/cdat/miniconda2b 
-4.8G /home/share/unix_files/cdat/miniconda2b 
- 
-bash-4.2$ du -sh /home/share/unix_files/cdat/miniconda2b/* 
-4.4M /home/share/unix_files/cdat/miniconda2b/bin 
-4.0K /home/share/unix_files/cdat/miniconda2b/conda-bld 
-280K /home/share/unix_files/cdat/miniconda2b/conda-meta 
-3.4G /home/share/unix_files/cdat/miniconda2b/envs 
-20K /home/share/unix_files/cdat/miniconda2b/etc 
-3.8M /home/share/unix_files/cdat/miniconda2b/include 
-100M /home/share/unix_files/cdat/miniconda2b/lib 
-8.0K /home/share/unix_files/cdat/miniconda2b/LICENSE.txt 
-1.4G /home/share/unix_files/cdat/miniconda2b/pkgs 
-16K /home/share/unix_files/cdat/miniconda2b/share 
-376K /home/share/unix_files/cdat/miniconda2b/ssl 
- 
-bash-4.2$ du -sh /home/share/unix_files/cdat/miniconda2b/envs/* 
-2.8G /home/share/unix_files/cdat/miniconda2b/envs/cdatm15 
-574M /home/share/unix_files/cdat/miniconda2b/envs/uvcdat-2.10 
 </code> </code>
  
Line 234: Line 157:
 We create a **//cdatm// symbolic link** in the ''envs'' directory, that has a //stable name// but can be moved to point to the latest default CDAT. In that case, most users can just activate this //cdatm// version and always get the latest stable version We create a **//cdatm// symbolic link** in the ''envs'' directory, that has a //stable name// but can be moved to point to the latest default CDAT. In that case, most users can just activate this //cdatm// version and always get the latest stable version
  
-<code>bash-4.2$ cd /home/share/unix_files/cdat/miniconda2b/envs +<code>$ cd /home/share/unix_files/cdat/miniconda3/envs 
- +$ conda env list 
-bash-4.2$ conda env list +base                  *  /home/share/unix_files/cdat/miniconda3 
-cdatm15                  /home/share/unix_files/cdat/miniconda2b/envs/cdatm15 +cdat-8.0_py2             /home/share/unix_files/cdat/miniconda3/envs/cdat-8.0_py2 
-uvcdat-2.10              /home/share/unix_files/cdat/miniconda2b/envs/uvcdat-2.10 +cdatm17_py2              /home/share/unix_files/cdat/miniconda3/envs/cdatm17_py2 
-root                  *  /home/share/unix_files/cdat/miniconda2b +$ ln -s cdatm17_py2 cdatm 
- +$ conda env list 
-bash-4.2$ ln -s cdatm15 cdatm +base                  *  /home/share/unix_files/cdat/miniconda3 
- +cdat-8.0_py2             /home/share/unix_files/cdat/miniconda3/envs/cdat-8.0_py2 
-bash-4.2$ conda env list +cdatm                    /home/share/unix_files/cdat/miniconda3/envs/cdatm 
-cdatm                    /home/share/unix_files/cdat/miniconda2b/envs/cdatm +cdatm17_py2              /home/share/unix_files/cdat/miniconda3/envs/cdatm17_py2 
-cdatm15                  /home/share/unix_files/cdat/miniconda2b/envs/cdatm15 +$ source activate cdatm
-uvcdat-2.10              /home/share/unix_files/cdat/miniconda2b/envs/uvcdat-2.10 +
-root                  *  /home/share/unix_files/cdat/miniconda2b +
- +
-bash-4.2$ source activate cdatm +
 (cdatm) bash-4.2$ conda env list (cdatm) bash-4.2$ conda env list
-cdatm                 *  /home/share/unix_files/cdat/miniconda2b/envs/cdatm +base                     /home/share/unix_files/cdat/miniconda3 
-cdatm15                  /home/share/unix_files/cdat/miniconda2b/envs/cdatm15 +cdat-8.0_py2             /home/share/unix_files/cdat/miniconda3/envs/cdat-8.0_py2 
-uvcdat-2.10              /home/share/unix_files/cdat/miniconda2b/envs/uvcdat-2.10 +cdatm                 *  /home/share/unix_files/cdat/miniconda3/envs/cdatm 
-root                     /home/share/unix_files/cdat/miniconda2b +cdatm17_py2              /home/share/unix_files/cdat/miniconda3/envs/cdatm17_py2</code>
-</code>+
  
 ===== Customizing UV-CDAT for LSCE ===== ===== Customizing UV-CDAT for LSCE =====
other/uvcdat/cdat_conda/cdat_8_0_py2.txt · Last modified: 2021/02/27 14:55 by jypeter
Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki