User Tools

Site Tools


other:uvcdat:cdat_conda:cdat_8_0_py2

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Next revision
Previous revision
Next revision Both sides next revision
other:uvcdat:cdat_conda:cdat_8_0_py2 [2018/11/07 17:08]
jypeter created
other:uvcdat:cdat_conda:cdat_8_0_py2 [2018/11/08 15:30]
jypeter
Line 15: Line 15:
  
 ===== Installation with Miniconda3 ===== ===== Installation with Miniconda3 =====
 +
 +==== Installing Miniconda3 ==== 
  
 We have installed a new version of Miniconda3, based on the old [[other:​uvcdat:​conda_notes#​installing_miniconda|installing miniconda]] instructions We have installed a new version of Miniconda3, based on the old [[other:​uvcdat:​conda_notes#​installing_miniconda|installing miniconda]] instructions
  
- * Follow the [[https://​conda.io/​docs/​install/​quick.html|Quick install]] steps to install Miniconda+  ​* Follow the [[https://​conda.io/​docs/​install/​quick.html|Quick install]] steps to install Miniconda
     * Get the [[https://​repo.continuum.io/​miniconda/​Miniconda3-latest-Linux-x86_64.sh|Python 3.7 Linux 64-bit (bash installer)]] (unless there is a [[https://​conda.io/​miniconda.html|more recent version?]])     * Get the [[https://​repo.continuum.io/​miniconda/​Miniconda3-latest-Linux-x86_64.sh|Python 3.7 Linux 64-bit (bash installer)]] (unless there is a [[https://​conda.io/​miniconda.html|more recent version?]])
       * ''​cd /​home/​scratch01/​jypeter''​ (some temporary storage space for the installer)       * ''​cd /​home/​scratch01/​jypeter''​ (some temporary storage space for the installer)
Line 44: Line 46:
 12K     ​x86_64-conda_cos6-linux-gnu</​code>​ 12K     ​x86_64-conda_cos6-linux-gnu</​code>​
         * Accept the automatic update of the //PATH// variable in ''​.bashrc''​ (note: the installer automatically creates a backup of ''​.bashrc''​)         * Accept the automatic update of the //PATH// variable in ''​.bashrc''​ (note: the installer automatically creates a backup of ''​.bashrc''​)
-          * ''#​ added by Miniconda2 4.3.21 ​installer''​\\ ''​export PATH="/​home/​share/​unix_files/​cdat/​miniconda2/​bin:​$PATH"''​ +          * ''#​ added by Miniconda3 ​installer''​\\ ''​export PATH="/​home/​share/​unix_files/​cdat/​miniconda3/​bin:​$PATH"''​ 
-    * Open a new terminal (if you use bash) or type ''​bash'' ​and check if the python from the conda distribution has been correctly installed +    * Open a new terminal (if you use bash) or open a new terminal and type ''​bash''​ 
-      * Update and clean the new installation +      * check if the python from the conda distribution has been correctly installed\\ <​code>​ > bash 
-        * ''​conda update ​%%--%%all''​ +$ which python 
-        * ''​conda clean %%--%%all''​ +/​home/​share/​unix_files/​cdat/​miniconda3/​bin/​python 
-      * <​code>​bash-4.1$ which python +$ which conda 
-/​home/​share/​unix_files/​cdat/​miniconda2/bin/python +/​home/​share/​unix_files/​cdat/​miniconda3/​bin/​conda</​code>​ 
-bash-4.1$ du -sh /​home/​share/​unix_files/​cdat/​miniconda2 +      * Update and clean the new installation\\ <​code>​$ ​conda update --all 
-114M    /​home/​share/​unix_files/​cdat/​miniconda2 +$ du -sh /​home/​share/​unix_files/​cdat/​miniconda3 
-bash-4.1$ conda list +515M    ​/home/share/​unix_files/​cdat/​miniconda3 
-# packages in environment at /​home/​share/​unix_files/​cdat/​miniconda2:+$ conda clean --all 
 +$ du -sh /​home/​share/​unix_files/​cdat/​miniconda3 
 +270M    /​home/​share/​unix_files/​cdat/​miniconda3 
 +$ conda list 
 +# packages in environment at /​home/​share/​unix_files/​cdat/​miniconda3:
 # #
-cffi                      1.9.1                    py27_0 +# Name                    Version ​                  ​Build  Channel
-conda                     ​4.3.13 ​                  ​py27_0 +
-conda-env ​                ​2.6.0 ​                        0 +
-cryptography ​             1.7.1                    py27_0+
 [...] [...]
 +python ​                   3.7.1                h0371630_3
 +readline ​                 7.0                  h7b6447c_5
 +requests ​                 2.20.0 ​                  ​py37_0
 +[...]</​code>​
 +  * Remove the miniconda3 initialization lines that were automatically added at the end of ''​.bashrc''​ and put them (and other useful commands) in a special initialization file (see ''​~jypeter/​.conda3_jyp.sh''​ below), that can be sourced only when we actually want to use conda and CDAT\\ <​code>​bash-4.2$ which python
 +/​usr/​bin/​python
 +$ cat ~jypeter/​.conda3_jyp.sh
 +# Conda initialization by JYP
 +#
 +# Use this for working with conda and CDAT centrally managed by JYP
 +#
 +# More details in:
 +#   ​https://​wiki.lsce.ipsl.fr/​pmip3/​doku.php/​other:​python:​starting#​conda-based_v ​                                   ersions_of_uv-cdat
 +#   ​https://​wiki.lsce.ipsl.fr/​pmip3/​doku.php/​other:​uvcdat:​conda_notes
 +#
 +# Jean-Yves Peterschmitt - LSCE - 11/2018
  
-<note important>​\\ We used the [[other:​uvcdat:​cdat_conda:​cdat_2_8_0#​downgrading_conda|recommended downgraded version of conda]] that was available after installing UV-CDAT 2.8</note>+export PATH="/home/​share/​unix_files/​cdat/​miniconda3/​bin:​$PATH"​
  
-DIsk space used before 2.10 installation (after running ''​conda clean --all''​)+# Use the alias below to easily determine where your python 
 +# interpreter is located 
 +alias wp="​which python"​
  
-<​code>​ > du -sh /​home/​share/​unix_files/​cdat/​miniconda2 +# Where are ALL the python interpreters in the search path 
-6.5G /​home/​share/​unix_files/​cdat/​miniconda2+alias wpa="​which -a python"​
  
- > du -sh /​home/​share/​unix_files/​cdat/​miniconda2/​* ​ +# The end 
-4.4M /home/​share/​unix_files/​cdat/​miniconda2/​bin +$ source ~jypeter/.conda3_jyp.sh 
-4.0K /​home/​share/​unix_files/​cdat/​miniconda2/​conda-bld +$ wp 
-272K /​home/​share/​unix_files/​cdat/​miniconda2/​conda-meta +/​home/​share/​unix_files/​cdat/​miniconda3/bin/python 
-6.4G /​home/​share/​unix_files/​cdat/​miniconda2/​envs +</code>
-20K /​home/​share/​unix_files/​cdat/​miniconda2/​etc +
-3.8M /​home/​share/​unix_files/​cdat/​miniconda2/include +
-99M /​home/​share/​unix_files/​cdat/​miniconda2/lib +
-4.0K /home/​share/​unix_files/​cdat/​miniconda2/​LICENSE.txt +
-76K /​home/​share/​unix_files/​cdat/​miniconda2/​pkgs +
-376K /​home/​share/​unix_files/​cdat/​miniconda2/​share +
-364K /​home/​share/​unix_files/​cdat/​miniconda2/​ssl+
  
- >​du -sh /​home/​share/​unix_files/​cdat/​miniconda2/​envs/​* +==== Installing CDAT 8.0 ====
-2.8G /​home/​share/​unix_files/​cdat/​miniconda2/​envs/​cdatm14 +
-1.7G /​home/​share/​unix_files/​cdat/​miniconda2/​envs/​keras +
-2.0G /​home/​share/​unix_files/​cdat/​miniconda2/​envs/​uvcdat-2.8.0</​code>​+
  
-Actual installation 
  
 <​code>​bash-4.1$ conda create -n uvcdat-2.10 -c conda-forge -c uvcdat uvcdat=2.10 <​code>​bash-4.1$ conda create -n uvcdat-2.10 -c conda-forge -c uvcdat uvcdat=2.10
 +conda create -n cdat-8.0 -c cdat/​label/​v80 -c conda-forge -c cdat python=2.7 cdat
 # Generate the list of installed packages # Generate the list of installed packages
 conda list -n uvcdat-2.10 > /​home/​scratch01/​jypeter/​uvcdat-2.10_list_170721.txt</​code>​ conda list -n uvcdat-2.10 > /​home/​scratch01/​jypeter/​uvcdat-2.10_list_170721.txt</​code>​
Line 96: Line 106:
 List of installed packages: {{:​other:​uvcdat:​cdat_conda:​uvcdat-2.10_list_170721.txt}} List of installed packages: {{:​other:​uvcdat:​cdat_conda:​uvcdat-2.10_list_170721.txt}}
  
-Disk space after 2.10 installation+Disk space after 8.installation
  
 <​code>​bash-4.1$ conda env list <​code>​bash-4.1$ conda env list
Line 128: Line 138:
 </​code>​ </​code>​
  
-Disk space after removing the keras environment that we don't need any more 
  
-<​code>​jypeter@asterix2 - ...cdat - 67 >du -sh /​home/​share/​unix_files/​cdat/​miniconda2 +===== Cloning ​cdat to add specific packages for LSCE =====
-9.0G /​home/​share/​unix_files/​cdat/​miniconda2 +
-jypeter@asterix2 - ...cdat - 68 >du -sh /​home/​share/​unix_files/​cdat/​miniconda2/​* +
-4.4M /​home/​share/​unix_files/​cdat/​miniconda2/​bin +
-4.0K /​home/​share/​unix_files/​cdat/​miniconda2/​conda-bld +
-272K /​home/​share/​unix_files/​cdat/​miniconda2/​conda-meta +
-7.5G /​home/​share/​unix_files/​cdat/​miniconda2/​envs +
-20K /​home/​share/​unix_files/​cdat/​miniconda2/​etc +
-3.8M /​home/​share/​unix_files/​cdat/​miniconda2/​include +
-99M /​home/​share/​unix_files/​cdat/​miniconda2/​lib +
-4.0K /​home/​share/​unix_files/​cdat/​miniconda2/​LICENSE.txt +
-1.4G /​home/​share/​unix_files/​cdat/​miniconda2/​pkgs +
-376K /​home/​share/​unix_files/​cdat/​miniconda2/​share +
-364K /​home/​share/​unix_files/​cdat/​miniconda2/​ssl +
-jypeter@asterix2 - ...cdat - 69 >du -sh /​home/​share/​unix_files/​cdat/​miniconda2/​envs/​* +
-2.8G /​home/​share/​unix_files/​cdat/​miniconda2/​envs/​cdatm14 +
-2.8G /​home/​share/​unix_files/​cdat/​miniconda2/​envs/​uvcdat-2.10 +
-2.0G /​home/​share/​unix_files/​cdat/​miniconda2/​envs/​uvcdat-2.8.0</​code>​ +
- +
-Disk space used in <wrap em>a brand new directory hierarchy</​wrap>​ (''​cdat/​miniconda2b'',​ on the obelix servers) +
- +
-<​code>​bash-4.2$ conda env list +
-uvcdat-2.10 ​             /​home/​share/​unix_files/​cdat/​miniconda2b/​envs/​uvcdat-2.10 +
-root                  *  /​home/​share/​unix_files/​cdat/​miniconda2b +
- +
-bash-4.2$ du -sh /​home/​share/​unix_files/​cdat/​miniconda2b +
-4.3G /​home/​share/​unix_files/​cdat/​miniconda2b +
- +
-bash-4.2$ du -sh /​home/​share/​unix_files/​cdat/​miniconda2b/​* +
-4.4M /​home/​share/​unix_files/​cdat/​miniconda2b/​bin +
-4.0K /​home/​share/​unix_files/​cdat/​miniconda2b/​conda-bld +
-280K /​home/​share/​unix_files/​cdat/​miniconda2b/​conda-meta +
-2.8G /​home/​share/​unix_files/​cdat/​miniconda2b/​envs +
-20K /​home/​share/​unix_files/​cdat/​miniconda2b/​etc +
-3.8M /​home/​share/​unix_files/​cdat/​miniconda2b/​include +
-100M /​home/​share/​unix_files/​cdat/​miniconda2b/​lib +
-8.0K /​home/​share/​unix_files/​cdat/​miniconda2b/​LICENSE.txt +
-1.4G /​home/​share/​unix_files/​cdat/​miniconda2b/​pkgs +
-16K /​home/​share/​unix_files/​cdat/​miniconda2b/​share +
-376K /​home/​share/​unix_files/​cdat/​miniconda2b/​ssl +
-</​code>​ +
- +
- +
-===== Cloning ​uvcdat ​to add specific packages for LSCE =====+
  
 <WRAP center round tip 60%> <WRAP center round tip 60%>
-This section is about the creation of the **cdatm15** environment+This section is about the creation of the **cdatm17** environment
  
 Notes about [[https://​wiki.lsce.ipsl.fr/​pmip3/​doku.php/​other:​python:​starting#​conda-based_versions_of_uv-cdat|actually using the cdatm15 conda-based python]] Notes about [[https://​wiki.lsce.ipsl.fr/​pmip3/​doku.php/​other:​python:​starting#​conda-based_versions_of_uv-cdat|actually using the cdatm15 conda-based python]]
Line 184: Line 150:
 Note: using hard links, cloning a full environment //only// adds an extra 500M of disk space Note: using hard links, cloning a full environment //only// adds an extra 500M of disk space
  
-<​code>​bash-4.1$ du -sh /​home/​share/​unix_files/​cdat/​miniconda2 +<​code>​bash-4.1$ du -sh /​home/​share/​unix_files/​cdat/​miniconda3 
-9.0G /​home/​share/​unix_files/​cdat/​miniconda2+9.0G /​home/​share/​unix_files/​cdat/​miniconda3
  
-bash-4.1$ du -sh /​home/​share/​unix_files/​cdat/​miniconda2/envs/* +bash-4.1$ du -sh /​home/​share/​unix_files/​cdat/​miniconda3/envs/*
-2.8G /​home/​share/​unix_files/​cdat/​miniconda2/​envs/​cdatm14 +
-2.8G /​home/​share/​unix_files/​cdat/​miniconda2/​envs/​uvcdat-2.10 +
-2.0G /​home/​share/​unix_files/​cdat/​miniconda2/​envs/​uvcdat-2.8.0+
  
-bash-4.1$ conda create -n cdatm15 ​--clone uvcdat-2.10 ​+bash-4.1$ conda create -n cdatm16 ​--clone uvcdat-2.10 ​
 Source: ​     /​home/​share/​unix_files/​cdat/​miniconda2/​envs/​uvcdat-2.10 Source: ​     /​home/​share/​unix_files/​cdat/​miniconda2/​envs/​uvcdat-2.10
 Destination:​ /​home/​share/​unix_files/​cdat/​miniconda2/​envs/​cdatm15 Destination:​ /​home/​share/​unix_files/​cdat/​miniconda2/​envs/​cdatm15
other/uvcdat/cdat_conda/cdat_8_0_py2.txt · Last modified: 2021/02/27 13:55 by jypeter