User Tools

Site Tools


other:uvcdat:conda_notes

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
other:uvcdat:conda_notes [2017/02/23 16:37]
jypeter
other:uvcdat:conda_notes [2021/12/21 10:33] (current)
jypeter [Installation history]
Line 2: Line 2:
  
 <WRAP center round tip 60%> <WRAP center round tip 60%>
-Check the [[.:cdat_conda|UV-CDAT conda-based versions installed by JYP]] page to see what is maintained by JYP+Check the [[.cdat_conda:index|UV-CDAT conda-based versions installed by JYP]] page to see what is maintained by JYP
 </​WRAP>​ </​WRAP>​
  
  
-Recent versions (2.8.and later?) of UV-CDAT are installed and maintained with the [[https://​conda.io/​docs/​index.html|conda]] **package, dependency and environment manager**. This page is about conda/​Miniconda/​Anaconda installation and related notes+Versions ​2.4.and later of UV-CDAT are installed and maintained with the [[https://​conda.io/​docs/​index.html|conda]] **package, dependency and environment manager**. This page is about **conda/​Miniconda/​Anaconda** installation and related notes
  
 ===== What should I install? ===== ===== What should I install? =====
Line 15: Line 15:
     * some parts of CDAT still need Python 2.7, so we will use the ''​2''​ version of //conda// (but it would also work with ''​3''​)     * some parts of CDAT still need Python 2.7, so we will use the ''​2''​ version of //conda// (but it would also work with ''​3''​)
   * You can do almost anything with //conda//, and you are mostly limited by [[#​disk_usage|disk space]].   * You can do almost anything with //conda//, and you are mostly limited by [[#​disk_usage|disk space]].
-    * If you know that you are going to use UV-CDAT, you might as well use the ''​conda''​ from Miniconda to avoid uselessly duplicating packages downloaded from the official Anaconda repository (used by Anaconda) with the same packages downloaded from the conda-forge repository (used by UV-CDAT).+    * If you know that you are going to use UV-CDAT, you might as well use the ''​conda''​ from //Miniconda// to avoid uselessly duplicating packages downloaded from the official Anaconda repository (used by Anaconda) with the same packages downloaded from the conda-forge repository (used by UV-CDAT).
  
 +  * When you are done with this page (not just this section), you should have a look at the [[https://​www.freecodecamp.org/​news/​why-you-need-python-environments-and-how-to-manage-them-with-conda-85f155f4353c/​|Why You Need Python Environments and How to Manage Them with Conda]] page that covers interesting topics
 ===== Miniconda ===== ===== Miniconda =====
 +
 +<WRAP center round tip 60%>
 +Note: we have shown in [[.:​cdat_conda:​cdat_2_8_0|UV-CDAT 2.8.0 installation notes]] that using Miniconda for installing UV-CDAT requires indeed less space than using Anaconda
 +</​WRAP>​
  
 Miniconda is basically just a self contained minimal Python distribution,​ with only the packages required to run //conda// Miniconda is basically just a self contained minimal Python distribution,​ with only the packages required to run //conda//
Line 24: Line 29:
  
   * Follow the [[https://​conda.io/​docs/​install/​quick.html|Quick install]] steps to install Miniconda   * Follow the [[https://​conda.io/​docs/​install/​quick.html|Quick install]] steps to install Miniconda
-    * Get the [[https://​repo.continuum.io/​miniconda/​Miniconda2-latest-Linux-x86_64.sh|Python 2.7 Linux 64-bit (bash installer)]] +    * Get the [[https://​repo.continuum.io/​miniconda/​Miniconda2-latest-Linux-x86_64.sh|Python 2.7 Linux 64-bit (bash installer)]] ​(unless there is a [[https://​conda.io/​miniconda.html|more recent version?]]) 
-      * ''​cd /​home/​scratch01/​jypeter''​ +      * ''​cd /​home/​scratch01/​jypeter'' ​(some temporary storage space for the installer) 
-      * ''​wget https://​repo.continuum.io/​miniconda/​Miniconda2-latest-Linux-x86_64.sh''​ (28 Mo, 16 Feb 2017)+      * ''​wget https://​repo.continuum.io/​miniconda/​Miniconda2-latest-Linux-x86_64.sh''​ (28 Mo, 02 Aug 2017)
     * Execute the installer     * Execute the installer
       * ''​bash Miniconda2-latest-Linux-x86_64.sh''​       * ''​bash Miniconda2-latest-Linux-x86_64.sh''​
         * Accept the license         * Accept the license
-        * Specify an explicit installation path **outside of the home directory**,​ with enough disk space, preferably on a disk that is not backed up: ''/​home/​share/​unix_files/​cdat/​miniconda2''​+        * Specify an explicit installation path **outside of the home directory**,​ with enough disk space (more than 3G if you are going to install UV-CDAT and some extra packages), preferably on a disk that is not backed up: ''/​home/​share/​unix_files/​cdat/​miniconda2''​
           * Resulting ''​miniconda2''​ directory size is 118M\\ <​code>​ > du -sh miniconda2/           * Resulting ''​miniconda2''​ directory size is 118M\\ <​code>​ > du -sh miniconda2/
 118M    miniconda2/ 118M    miniconda2/
Line 46: Line 51:
 8.0K    ssl</​code>​ 8.0K    ssl</​code>​
         * Accept the automatic update of the //PATH// variable in ''​.bashrc''​ (note: the installer automatically creates a backup of ''​.bashrc''​)         * Accept the automatic update of the //PATH// variable in ''​.bashrc''​ (note: the installer automatically creates a backup of ''​.bashrc''​)
-          * ''#​ added by Miniconda2 4.3.11 installer''​\\ ''​export PATH="/​home/​share/​unix_files/​cdat/​miniconda2/​bin:​$PATH"''​+          * ''#​ added by Miniconda2 4.3.21 installer''​\\ ''​export PATH="/​home/​share/​unix_files/​cdat/​miniconda2/​bin:​$PATH"''​
     * Open a new terminal (if you use bash) or type ''​bash''​ and check if the python from the conda distribution has been correctly installed     * Open a new terminal (if you use bash) or type ''​bash''​ and check if the python from the conda distribution has been correctly installed
       * Update and clean the new installation       * Update and clean the new installation
-        * ''​conda ​updpate ​--all''​ +        * ''​conda ​update %%--%%all''​ 
-        * ''​conda clean --all''​+        * ''​conda clean %%--%%all''​
       * <​code>​bash-4.1$ which python       * <​code>​bash-4.1$ which python
 /​home/​share/​unix_files/​cdat/​miniconda2/​bin/​python /​home/​share/​unix_files/​cdat/​miniconda2/​bin/​python
Line 76: Line 81:
 ==== Installing Anaconda ==== ==== Installing Anaconda ====
  
-  * Follow the [[https://​conda.io/​docs/​install/​full.html|Full installation]] steps to install ​Miniconda+  * Follow the [[https://​conda.io/​docs/​install/​full.html|Full installation]] steps (and what is detailed above in [[#​installing_miniconda|Installing Miniconda]]) ​to install ​Anaconda
     * Installation package ''​Anaconda2-4.3.0-Linux-x86_64.sh''​ is 462M (17 Feb 2017)     * Installation package ''​Anaconda2-4.3.0-Linux-x86_64.sh''​ is 462M (17 Feb 2017)
     * Size after initial installation : 2G\\ <​code>​jypeter@asterix3 - ...anaconda2 - 93 >du -sh /​home/​share/​unix_files/​cdat/​anaconda2     * Size after initial installation : 2G\\ <​code>​jypeter@asterix3 - ...anaconda2 - 93 >du -sh /​home/​share/​unix_files/​cdat/​anaconda2
Line 113: Line 118:
 //conda// will use hard links and share packages common to several installed environments to minimize disk usage, but there is no harm in being careful and checking things! //conda// will use hard links and share packages common to several installed environments to minimize disk usage, but there is no harm in being careful and checking things!
  
-  * You should avoid installing conda //stuff// in your home directory (''​~/​.conda''​)+  * Because of the likely large disk space usage, you should avoid installing conda //stuff// in your home directory (''​~/​.conda''​)
   * You should regularly monitor the size of the directories listed in ''​conda info''​   * You should regularly monitor the size of the directories listed in ''​conda info''​
     * ''​~/​.conda'':​ the hidden conda directory in your home directory     * ''​~/​.conda'':​ the hidden conda directory in your home directory
Line 142: Line 147:
   * Deleting an environment   * Deleting an environment
     * ''​conda remove %%--%%name crap_environment %%--%%all''​\\ ''​conda clean --all''​     * ''​conda remove %%--%%name crap_environment %%--%%all''​\\ ''​conda clean --all''​
-  ​* Activating or going back to //root// +    * The following seems to do the same: ''​conda env remove -n crap_environment''​ 
-    * ''​source activate existing_environment''​ +  ​* Activating or going back to //root/base// 
-      * ''​source ​activate''​ will activate the //root// environment +    * ''​conda activate existing_environment''​ (used to be ''​source activate existing_environment'' ​in older versions of conda) 
-    * ''​source deactivate''​+      * ''​conda activate''​ will activate the //root/base// environment 
 +    * You can use ''​which python''​ to determine where is the current Python you are using, or ''​which -a python''​ (''​-a''​ will only work in //bash//) to display all the Python executables in the current search path 
 +    * ''​conda deactivate''​ (used to be ''​source deactivate''​)
   * Installed packages   * Installed packages
     * ''​conda list''​     * ''​conda list''​
Line 151: Line 158:
   * Searching packages   * Searching packages
     * ''​conda search %%--%%full-name package_exact_name''​\\ ''​conda search package_partial_name''​     * ''​conda search %%--%%full-name package_exact_name''​\\ ''​conda search package_partial_name''​
 +    * See also the ''​%%--%%channel''​ option below
   * Installing and removing packages\\ Add ''​-n target_environment''​ to act on another environment than the active one   * Installing and removing packages\\ Add ''​-n target_environment''​ to act on another environment than the active one
     * ''​conda install package1 package2 packageN''​     * ''​conda install package1 package2 packageN''​
Line 158: Line 166:
     * ''​conda remove package1 package2 packageN''​     * ''​conda remove package1 package2 packageN''​
   * Updating conda (update the //conda// package, not the full Miniconda)   * Updating conda (update the //conda// package, not the full Miniconda)
-    * ''​conda update conda''​+    * ''​conda update ​-n base -c defaults ​conda''​ 
 +  * Downgrading conda (required for installing UV-CDAT 2.10) 
 +    * <​code>​bash-4.1$ conda install -n root "​conda<​4.3.13"​ 
 +Fetching package metadata ......... 
 +Solving package specifications:​ . 
 + 
 +Package plan for installation in environment /​home/​share/​unix_files/​cdat/​miniconda2:​ 
 + 
 +The following packages will be DOWNGRADED due to dependency conflicts:​ 
 + 
 +    conda: 4.3.13-py27_0 --> 4.3.11-py27_0 
 + 
 +Proceed ([y]/n)? y 
 + 
 +conda-4.3.11-p 100% |###############################################################​| Time: 0:​00:​00 ​  4.28 MB/s 
 + 
 +bash-4.1$ conda info 
 +Current conda install: 
 + 
 +               ​platform : linux-64 
 +          conda version : 4.3.11 
 +       conda is private : False 
 +      conda-env version : 4.3.11</​code>​ 
   * conda does not automatically clean the content of the ''​pkgs''​ directory! Use the following to //Remove index cache, lock files, tarballs, unused cache packages, and source cache//   * conda does not automatically clean the content of the ''​pkgs''​ directory! Use the following to //Remove index cache, lock files, tarballs, unused cache packages, and source cache//
     * ''​conda clean %%--%%all''​     * ''​conda clean %%--%%all''​
  
 +==== Installation history ====
 +
 +You can get the current list of installed packages with ''​conda list'',​ but it may also be useful to know the installation order:
 +
 +  * ''​conda list %%--%%revisions''​ revisions will display **exactly** what was installed and when (including the updates, and automatic dependencies installation)
 +    * You can theoretically roll back to a previous ''​RR''​ //​revision//​ number with: ''​conda install %%--%%revision RR''​
 +  * There is more information in the ''​conda-meta/​history''​ installation log files. e.g., if you need to know the list of installation commands used:\\ <​code>​ $ egrep -e '​cmd:'​ -e '​==>'​ /​home/​share/​unix_files/​cdat/​miniconda3/​envs/​cdatm18_py2/​conda-meta/​history
 +==> 2019-03-07 16:19:55 <==
 +# cmd: /​home/​share/​unix_files/​cdat/​miniconda3/​bin/​conda create -n cdatm18_py2 --clone cdat-8.1_py2
 +==> 2019-03-07 16:35:11 <==
 +# cmd: /​home/​share/​unix_files/​cdat/​miniconda3/​bin/​conda install -n cdatm18_py2 -c conda-forge pillow pandas statsmodels seaborn scikit-image seawater gsw netcdf4 pyferret basemap-data-hires xlsxwriter cmocean rpy2 gdal windspharm
 +==> 2019-03-07 16:57:50 <==
 +# cmd: /​home/​share/​unix_files/​cdat/​miniconda3/​bin/​conda install -n cdatm18_py2 -c conda-forge -c vacumm vacumm
 +==> 2019-03-12 14:35:31 <==
 +# cmd: /​home/​share/​unix_files/​cdat/​miniconda3/​bin/​conda install -n cdatm18_py2 -c conda-forge pillow pandas statsmodels seaborn scikit-image seawater gsw netcdf4 pyferret basemap-data-hires xlsxwriter cmocean rpy2 gdal windspharm
 +==> 2019-03-12 15:49:54 <==
 +# cmd: /​home/​share/​unix_files/​cdat/​miniconda3/​bin/​conda install -n cdatm18_py2 -c stefraynaud -c conda-forge spanlib
 +==> 2019-03-12 17:30:26 <==
 +# cmd: /​home/​share/​unix_files/​cdat/​miniconda3/​bin/​conda install -n cdatm18_py2 -c conda-forge cmor</​code>​
 +    * You can also, use a script and the ''​conda''​ module available **in the ''​base''​ environment**:​ [[https://​github.com/​conda/​conda/​issues/​4545#​issuecomment-469984684|example]]
 ==== Misc ==== ==== Misc ====
  
Line 169: Line 220:
   * [[https://​conda.io/​docs/​|conda Full documentation]]   * [[https://​conda.io/​docs/​|conda Full documentation]]
   * [[https://​conda.io/​docs/​faq.html|conda FAQ]]   * [[https://​conda.io/​docs/​faq.html|conda FAQ]]
-  * [[https://​conda.io/​docs/​using/envs.html#share-an-environment|Sharing (replicating) an environment]]+  * [[https://​conda.io/​docs/​user-guide/tasks/​manage-environments.html#sharing-an-environment|Sharing (replicating) an environment]]
   * [[https://​conda.io/​docs/​install/​central.html|Centralized installation]]   * [[https://​conda.io/​docs/​install/​central.html|Centralized installation]]
  
other/uvcdat/conda_notes.1487867859.txt.gz · Last modified: 2017/02/23 16:37 by jypeter