Differences

This shows you the differences between two versions of the page.

--- other:uvcdat:conda_notes [2017/02/21 11:45] – jypeter
+++ other:uvcdat:conda_notes [2024/06/17 17:22] (current) – [Installing and maintaining UV-CDAT with conda] Added out-of-date warning jypeter
@@ Line 1: / Line 1: @@
 ====== Installing and maintaining UV-CDAT with conda ======
+<WRAP center round alert 60%>
+**This is an out-of-date legacy page!**
+You should use the [[other:uvcdat:cdat_conda:miniconda3_install|JYP steps for installing Miniconda3]] page to get recent ''conda''/Miniconda/Anaconda information
+</WRAP>
 <WRAP center round tip 60%>
-Check the [[.:cdat_conda|UV-CDAT conda-based versions installed by JYP]] page to see what is maintained by JYP
+Check the [[.cdat_conda:index|UV-CDAT conda-based versions installed by JYP]] page to see what is maintained by JYP
 </WRAP>
-Recent versions (2.8.0 and later?) of UV-CDAT are installed and maintained with the [[https://conda.io/docs/index.html|conda]] **package, dependency and environment manager**. This page is about conda/Miniconda/Anaconda installation and related notes
+Versions 2.4.1 and later of UV-CDAT are installed and maintained with the [[https://conda.io/docs/index.html|conda]] **package, dependency and environment manager**. This page is about **conda/Miniconda/Anaconda** installation and related notes
 ===== What should I install? =====
-  * The difference between conda, Miniconda and Anaconda is detailed in the [[https://conda.io/docs/faq.html#general-questions|General questions]] part of the FAQ
+  * The difference between **conda**, **Miniconda** and **Anaconda** is detailed in the [[https://conda.io/docs/faq.html#general-questions|General questions]] part of the FAQ
   * Installers with a ''2'' extension will use **Python2.7**. Those with ''3'' will use **Python3.x**
     * any version of //conda// can install any version of Python (and UV-CDAT), and you can have several versions of Python (and UV-CDAT) installed at the same time
     * some parts of CDAT still need Python 2.7, so we will use the ''2'' version of //conda// (but it would also work with ''3'')
-  * You can do almost anything with conda, and you are only limited by [[#disk_usage|disk space]].
+  * You can do almost anything with //conda//, and you are mostly limited by [[#disk_usage|disk space]].
-    * If you know that you are going to use UV-CDAT, you might as well use the ''conda'' from Miniconda to avoid duplicating packages downloaded from the official Anaconda repository (used by Anaconda) and from the conda-forge repository (used by UV-CDAT)
+    * If you know that you are going to use UV-CDAT, you might as well use the ''conda'' from //Miniconda// to avoid uselessly duplicating packages downloaded from the official Anaconda repository (used by Anaconda) with the same packages downloaded from the conda-forge repository (used by UV-CDAT).
+  * When you are done with this page (not just this section), you should have a look at the [[https://www.freecodecamp.org/news/why-you-need-python-environments-and-how-to-manage-them-with-conda-85f155f4353c/|Why You Need Python Environments and How to Manage Them with Conda]] page that covers interesting topics
 ===== Miniconda =====
-==== Practicing with Miniconda ====
+<WRAP center round tip 60%>
+Note: we have shown in [[.:cdat_conda:cdat_2_8_0|UV-CDAT 2.8.0 installation notes]] that using Miniconda for installing UV-CDAT requires indeed less space than using Anaconda
+</WRAP>
+Miniconda is basically just a self contained minimal Python distribution, with only the packages required to run //conda//
+==== Installing Miniconda ====
   * Follow the [[https://conda.io/docs/install/quick.html|Quick install]] steps to install Miniconda
-    * Get the [[https://repo.continuum.io/miniconda/Miniconda2-latest-Linux-x86_64.sh|Python 2.7 Linux 64-bit (bash installer)]]
+    * Get the [[https://repo.continuum.io/miniconda/Miniconda2-latest-Linux-x86_64.sh|Python 2.7 Linux 64-bit (bash installer)]] (unless there is a [[https://conda.io/miniconda.html|more recent version?]])
-      * ''cd /home/scratch01/jypeter''
+      * ''cd /home/scratch01/jypeter'' (some temporary storage space for the installer)
-      * ''wget https://repo.continuum.io/miniconda/Miniconda2-latest-Linux-x86_64.sh'' (28 Mo, 16 Feb 2017)
+      * ''wget https://repo.continuum.io/miniconda/Miniconda2-latest-Linux-x86_64.sh'' (28 Mo, 02 Aug 2017)
     * Execute the installer
       * ''bash Miniconda2-latest-Linux-x86_64.sh''
         * Accept the license
-        * Specify an explicit installation path **outside of the home directory**, with enough disk space, preferably on a disk that is not backed up: ''/home/share/unix_files/cdat/miniconda2''
+        * Specify an explicit installation path **outside of the home directory**, with enough disk space (more than 3G if you are going to install UV-CDAT and some extra packages), preferably on a disk that is not backed up: ''/home/share/unix_files/cdat/miniconda2''
-          * Resulting ''miniconda2'' directory size is 118M\\ <code>jypeter@asterix3 - ...versions - 51 >du -sh miniconda2/
+          * Resulting ''miniconda2'' directory size is 118M\\ <code> > du -sh miniconda2/
 M    miniconda2/
-jypeter@asterix3 - ...versions - 52 >cd miniconda2/
+ > cd miniconda2/
-jypeter@asterix3 - ...miniconda2 - 54 >du -sh *
+ > du -sh *
 .4M    bin
 K    conda-meta
@@ Line 44: / Line 58: @@
 .0K    ssl</code>
         * Accept the automatic update of the //PATH// variable in ''.bashrc'' (note: the installer automatically creates a backup of ''.bashrc'')
-          * ''# added by Miniconda2 4.3.11 installer''\\ ''export PATH="/home/share/unix_files/cdat/miniconda2/bin:$PATH"''
+          * ''# added by Miniconda2 4.3.21 installer''\\ ''export PATH="/home/share/unix_files/cdat/miniconda2/bin:$PATH"''
     * Open a new terminal (if you use bash) or type ''bash'' and check if the python from the conda distribution has been correctly installed
+      * Update and clean the new installation
+        * ''conda update %%--%%all''
+        * ''conda clean %%--%%all''
       * <code>bash-4.1$ which python
-/home/share/unix_files/cdat/versions/miniconda2/bin/python
+/home/share/unix_files/cdat/miniconda2/bin/python
+bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2
+M    /home/share/unix_files/cdat/miniconda2
 bash-4.1$ conda list
-# packages in environment at /home/share/unix_files/cdat/versions/miniconda2:
+# packages in environment at /home/share/unix_files/cdat/miniconda2:
 #
 cffi                      1.9.1                    py27_0
-conda                     4.3.11                   py27_0
+conda                     4.3.13                   py27_0
 conda-env                 2.6.0                         0
 cryptography              1.7.1                    py27_0
@@ Line 69: / Line 88: @@
 ==== Installing Anaconda ====
-  * Follow the [[https://conda.io/docs/install/full.html|Full installation]] steps to install Miniconda
+  * Follow the [[https://conda.io/docs/install/full.html|Full installation]] steps (and what is detailed above in [[#installing_miniconda|Installing Miniconda]]) to install Anaconda
     * Installation package ''Anaconda2-4.3.0-Linux-x86_64.sh'' is 462M (17 Feb 2017)
     * Size after initial installation : 2G\\ <code>jypeter@asterix3 - ...anaconda2 - 93 >du -sh /home/share/unix_files/cdat/anaconda2
@@ Line 106: / Line 125: @@
 //conda// will use hard links and share packages common to several installed environments to minimize disk usage, but there is no harm in being careful and checking things!
-  * You should avoid installing conda //stuff// in your home directory (''~/.conda'')
+  * Because of the likely large disk space usage, you should avoid installing conda //stuff// in your home directory (''~/.conda'')
   * You should regularly monitor the size of the directories listed in ''conda info''
     * ''~/.conda'': the hidden conda directory in your home directory
@@ Line 121: / Line 140: @@
 .7G    /home/share/unix_files/cdat/anaconda2/lib
 </code>
+  * See the note about ''conda clean --all'' in the [[#useful_conda_commands|Useful conda commands]] section
 ==== Useful conda commands ====
@@ Line 133: / Line 153: @@
     * ''conda create %%--%%name new_environment package1 package2 packageN''\\ ''conda create -n new_environment package1 package2 packageN''
   * Deleting an environment
-    * ''conda remove %%--%%name crap_environment %%--%%all''
+    * ''conda remove %%--%%name crap_environment %%--%%all''\\ ''conda clean --all''
-  * Activating or going back to //root//
+    * The following seems to do the same: ''conda env remove -n crap_environment''
-    * ''source activate existing_environment''
+  * Activating or going back to //root/base//
-      * ''source activate'' will activate the //root// environment
+    * ''conda activate existing_environment'' (used to be ''source activate existing_environment'' in older versions of conda)
-    * ''source deactivate''
+      * ''conda activate'' will activate the //root/base// environment
+    * You can use ''which python'' to determine where is the current Python you are using, or ''which -a python'' (''-a'' will only work in //bash//) to display all the Python executables in the current search path
+    * ''conda deactivate'' (used to be ''source deactivate'')
   * Installed packages
     * ''conda list''
@@ Line 143: / Line 165: @@
   * Searching packages
     * ''conda search %%--%%full-name package_exact_name''\\ ''conda search package_partial_name''
+    * See also the ''%%--%%channel'' option below
   * Installing and removing packages\\ Add ''-n target_environment'' to act on another environment than the active one
     * ''conda install package1 package2 packageN''
@@ Line 150: / Line 173: @@
     * ''conda remove package1 package2 packageN''
   * Updating conda (update the //conda// package, not the full Miniconda)
-    * ''conda update conda''
+    * ''conda update -n base -c defaults conda''
+  * Downgrading conda (required for installing UV-CDAT 2.10)
+    * <code>bash-4.1$ conda install -n root "conda<4.3.13"
+Fetching package metadata .........
+Solving package specifications: .
+Package plan for installation in environment /home/share/unix_files/cdat/miniconda2:
+The following packages will be DOWNGRADED due to dependency conflicts:
+    conda: 4.3.13-py27_0 --> 4.3.11-py27_0
+Proceed ([y]/n)? y
+conda-4.3.11-p 100% |###############################################################| Time: 0:00:00   4.28 MB/s
+bash-4.1$ conda info
+Current conda install:
+               platform : linux-64
+          conda version : 4.3.11
+       conda is private : False
+      conda-env version : 4.3.11</code>
+  * conda does not automatically clean the content of the ''pkgs'' directory! Use the following to //Remove index cache, lock files, tarballs, unused cache packages, and source cache//
+    * ''conda clean %%--%%all''
+==== Installation history ====
+You can get the current list of installed packages with ''conda list'', but it may also be useful to know the installation order:
+  * ''conda list %%--%%revisions'' revisions will display **exactly** what was installed and when (including the updates, and automatic dependencies installation)
+    * You can theoretically roll back to a previous ''RR'' //revision// number with: ''conda install %%--%%revision RR''
+  * There is more information in the ''conda-meta/history'' installation log files. e.g., if you need to know the list of installation commands used:\\ <code> $ egrep -e 'cmd:' -e '==>' /home/share/unix_files/cdat/miniconda3/envs/cdatm18_py2/conda-meta/history
+==> 2019-03-07 16:19:55 <==
+# cmd: /home/share/unix_files/cdat/miniconda3/bin/conda create -n cdatm18_py2 --clone cdat-8.1_py2
+==> 2019-03-07 16:35:11 <==
+# cmd: /home/share/unix_files/cdat/miniconda3/bin/conda install -n cdatm18_py2 -c conda-forge pillow pandas statsmodels seaborn scikit-image seawater gsw netcdf4 pyferret basemap-data-hires xlsxwriter cmocean rpy2 gdal windspharm
+==> 2019-03-07 16:57:50 <==
+# cmd: /home/share/unix_files/cdat/miniconda3/bin/conda install -n cdatm18_py2 -c conda-forge -c vacumm vacumm
+==> 2019-03-12 14:35:31 <==
+# cmd: /home/share/unix_files/cdat/miniconda3/bin/conda install -n cdatm18_py2 -c conda-forge pillow pandas statsmodels seaborn scikit-image seawater gsw netcdf4 pyferret basemap-data-hires xlsxwriter cmocean rpy2 gdal windspharm
+==> 2019-03-12 15:49:54 <==
+# cmd: /home/share/unix_files/cdat/miniconda3/bin/conda install -n cdatm18_py2 -c stefraynaud -c conda-forge spanlib
+==> 2019-03-12 17:30:26 <==
+# cmd: /home/share/unix_files/cdat/miniconda3/bin/conda install -n cdatm18_py2 -c conda-forge cmor</code>
+    * You can also, use a script and the ''conda'' module available **in the ''base'' environment**: [[https://github.com/conda/conda/issues/4545#issuecomment-469984684|example]]
 ==== Misc ====
   * [[https://docs.continuum.io/anaconda/pkg-docs|Available 'official' packages]]
   * [[https://anaconda.org/|Other packages]]
+  * [[https://conda-forge.github.io/feedstocks|Packages in conda-forge]]
   * [[https://conda.io/docs/|conda Full documentation]]
   * [[https://conda.io/docs/faq.html|conda FAQ]]
-  * [[https://conda.io/docs/using/envs.html#share-an-environment|Sharing (replicating) an environment]]
+  * [[https://conda.io/docs/user-guide/tasks/manage-environments.html#sharing-an-environment|Sharing (replicating) an environment]]
   * [[https://conda.io/docs/install/central.html|Centralized installation]]