This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
other:uvcdat:conda_notes [2017/02/21 10:45] jypeter |
other:uvcdat:conda_notes [2017/08/02 14:14] jypeter A few updates |
||
---|---|---|---|
Line 2: | Line 2: | ||
<WRAP center round tip 60%> | <WRAP center round tip 60%> | ||
- | Check the [[.:cdat_conda|UV-CDAT conda-based versions installed by JYP]] page to see what is maintained by JYP | + | Check the [[.cdat_conda:index|UV-CDAT conda-based versions installed by JYP]] page to see what is maintained by JYP |
</WRAP> | </WRAP> | ||
- | Recent versions (2.8.0 and later?) of UV-CDAT are installed and maintained with the [[https://conda.io/docs/index.html|conda]] **package, dependency and environment manager**. This page is about conda/Miniconda/Anaconda installation and related notes | + | Versions 2.4.1 and later of UV-CDAT are installed and maintained with the [[https://conda.io/docs/index.html|conda]] **package, dependency and environment manager**. This page is about **conda/Miniconda/Anaconda** installation and related notes |
===== What should I install? ===== | ===== What should I install? ===== | ||
- | * The difference between conda, Miniconda and Anaconda is detailed in the [[https://conda.io/docs/faq.html#general-questions|General questions]] part of the FAQ | + | * The difference between **conda**, **Miniconda** and **Anaconda** is detailed in the [[https://conda.io/docs/faq.html#general-questions|General questions]] part of the FAQ |
* Installers with a ''2'' extension will use **Python2.7**. Those with ''3'' will use **Python3.x** | * Installers with a ''2'' extension will use **Python2.7**. Those with ''3'' will use **Python3.x** | ||
* any version of //conda// can install any version of Python (and UV-CDAT), and you can have several versions of Python (and UV-CDAT) installed at the same time | * any version of //conda// can install any version of Python (and UV-CDAT), and you can have several versions of Python (and UV-CDAT) installed at the same time | ||
* some parts of CDAT still need Python 2.7, so we will use the ''2'' version of //conda// (but it would also work with ''3'') | * some parts of CDAT still need Python 2.7, so we will use the ''2'' version of //conda// (but it would also work with ''3'') | ||
- | * You can do almost anything with conda, and you are only limited by [[#disk_usage|disk space]]. | + | * You can do almost anything with //conda//, and you are mostly limited by [[#disk_usage|disk space]]. |
- | * If you know that you are going to use UV-CDAT, you might as well use the ''conda'' from Miniconda to avoid duplicating packages downloaded from the official Anaconda repository (used by Anaconda) and from the conda-forge repository (used by UV-CDAT) | + | * If you know that you are going to use UV-CDAT, you might as well use the ''conda'' from //Miniconda// to avoid uselessly duplicating packages downloaded from the official Anaconda repository (used by Anaconda) with the same packages downloaded from the conda-forge repository (used by UV-CDAT). |
===== Miniconda ===== | ===== Miniconda ===== | ||
- | ==== Practicing with Miniconda ==== | + | <WRAP center round tip 60%> |
+ | Note: we have shown in [[.:cdat_conda:cdat_2_8_0|UV-CDAT 2.8.0 installation notes]] that using Miniconda for installing UV-CDAT requires indeed less space than using Anaconda | ||
+ | </WRAP> | ||
+ | |||
+ | Miniconda is basically just a self contained minimal Python distribution, with only the packages required to run //conda// | ||
+ | |||
+ | ==== Installing Miniconda ==== | ||
* Follow the [[https://conda.io/docs/install/quick.html|Quick install]] steps to install Miniconda | * Follow the [[https://conda.io/docs/install/quick.html|Quick install]] steps to install Miniconda | ||
- | * Get the [[https://repo.continuum.io/miniconda/Miniconda2-latest-Linux-x86_64.sh|Python 2.7 Linux 64-bit (bash installer)]] | + | * Get the [[https://repo.continuum.io/miniconda/Miniconda2-latest-Linux-x86_64.sh|Python 2.7 Linux 64-bit (bash installer)]] (unless there is a [[https://conda.io/miniconda.html|more recent version?]]) |
- | * ''cd /home/scratch01/jypeter'' | + | * ''cd /home/scratch01/jypeter'' (some temporary storage space for the installer) |
- | * ''wget https://repo.continuum.io/miniconda/Miniconda2-latest-Linux-x86_64.sh'' (28 Mo, 16 Feb 2017) | + | * ''wget https://repo.continuum.io/miniconda/Miniconda2-latest-Linux-x86_64.sh'' (28 Mo, 02 Aug 2017) |
* Execute the installer | * Execute the installer | ||
* ''bash Miniconda2-latest-Linux-x86_64.sh'' | * ''bash Miniconda2-latest-Linux-x86_64.sh'' | ||
* Accept the license | * Accept the license | ||
- | * Specify an explicit installation path **outside of the home directory**, with enough disk space, preferably on a disk that is not backed up: ''/home/share/unix_files/cdat/miniconda2'' | + | * Specify an explicit installation path **outside of the home directory**, with enough disk space (more than 3G if you are going to install UV-CDAT and some extra packages), preferably on a disk that is not backed up: ''/home/share/unix_files/cdat/miniconda2'' |
- | * Resulting ''miniconda2'' directory size is 118M\\ <code>jypeter@asterix3 - ...versions - 51 >du -sh miniconda2/ | + | * Resulting ''miniconda2'' directory size is 118M\\ <code> > du -sh miniconda2/ |
118M miniconda2/ | 118M miniconda2/ | ||
- | jypeter@asterix3 - ...versions - 52 >cd miniconda2/ | + | > cd miniconda2/ |
- | jypeter@asterix3 - ...miniconda2 - 54 >du -sh * | + | > du -sh * |
4.4M bin | 4.4M bin | ||
268K conda-meta | 268K conda-meta | ||
Line 44: | Line 50: | ||
8.0K ssl</code> | 8.0K ssl</code> | ||
* Accept the automatic update of the //PATH// variable in ''.bashrc'' (note: the installer automatically creates a backup of ''.bashrc'') | * Accept the automatic update of the //PATH// variable in ''.bashrc'' (note: the installer automatically creates a backup of ''.bashrc'') | ||
- | * ''# added by Miniconda2 4.3.11 installer''\\ ''export PATH="/home/share/unix_files/cdat/miniconda2/bin:$PATH"'' | + | * ''# added by Miniconda2 4.3.21 installer''\\ ''export PATH="/home/share/unix_files/cdat/miniconda2/bin:$PATH"'' |
* Open a new terminal (if you use bash) or type ''bash'' and check if the python from the conda distribution has been correctly installed | * Open a new terminal (if you use bash) or type ''bash'' and check if the python from the conda distribution has been correctly installed | ||
+ | * Update and clean the new installation | ||
+ | * ''conda update %%--%%all'' | ||
+ | * ''conda clean %%--%%all'' | ||
* <code>bash-4.1$ which python | * <code>bash-4.1$ which python | ||
- | /home/share/unix_files/cdat/versions/miniconda2/bin/python | + | /home/share/unix_files/cdat/miniconda2/bin/python |
+ | bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2 | ||
+ | 114M /home/share/unix_files/cdat/miniconda2 | ||
bash-4.1$ conda list | bash-4.1$ conda list | ||
- | # packages in environment at /home/share/unix_files/cdat/versions/miniconda2: | + | # packages in environment at /home/share/unix_files/cdat/miniconda2: |
# | # | ||
cffi 1.9.1 py27_0 | cffi 1.9.1 py27_0 | ||
- | conda 4.3.11 py27_0 | + | conda 4.3.13 py27_0 |
conda-env 2.6.0 0 | conda-env 2.6.0 0 | ||
cryptography 1.7.1 py27_0 | cryptography 1.7.1 py27_0 | ||
Line 69: | Line 80: | ||
==== Installing Anaconda ==== | ==== Installing Anaconda ==== | ||
- | * Follow the [[https://conda.io/docs/install/full.html|Full installation]] steps to install Miniconda | + | * Follow the [[https://conda.io/docs/install/full.html|Full installation]] steps (and what is detailed above in [[#installing_miniconda|Installing Miniconda]]) to install Anaconda |
* Installation package ''Anaconda2-4.3.0-Linux-x86_64.sh'' is 462M (17 Feb 2017) | * Installation package ''Anaconda2-4.3.0-Linux-x86_64.sh'' is 462M (17 Feb 2017) | ||
* Size after initial installation : 2G\\ <code>jypeter@asterix3 - ...anaconda2 - 93 >du -sh /home/share/unix_files/cdat/anaconda2 | * Size after initial installation : 2G\\ <code>jypeter@asterix3 - ...anaconda2 - 93 >du -sh /home/share/unix_files/cdat/anaconda2 | ||
Line 106: | Line 117: | ||
//conda// will use hard links and share packages common to several installed environments to minimize disk usage, but there is no harm in being careful and checking things! | //conda// will use hard links and share packages common to several installed environments to minimize disk usage, but there is no harm in being careful and checking things! | ||
- | * You should avoid installing conda //stuff// in your home directory (''~/.conda'') | + | * Because of the likely large disk space usage, you should avoid installing conda //stuff// in your home directory (''~/.conda'') |
* You should regularly monitor the size of the directories listed in ''conda info'' | * You should regularly monitor the size of the directories listed in ''conda info'' | ||
* ''~/.conda'': the hidden conda directory in your home directory | * ''~/.conda'': the hidden conda directory in your home directory | ||
Line 121: | Line 132: | ||
1.7G /home/share/unix_files/cdat/anaconda2/lib | 1.7G /home/share/unix_files/cdat/anaconda2/lib | ||
</code> | </code> | ||
+ | * See the note about ''conda clean --all'' in the [[#useful_conda_commands|Useful conda commands]] section | ||
==== Useful conda commands ==== | ==== Useful conda commands ==== | ||
Line 133: | Line 145: | ||
* ''conda create %%--%%name new_environment package1 package2 packageN''\\ ''conda create -n new_environment package1 package2 packageN'' | * ''conda create %%--%%name new_environment package1 package2 packageN''\\ ''conda create -n new_environment package1 package2 packageN'' | ||
* Deleting an environment | * Deleting an environment | ||
- | * ''conda remove %%--%%name crap_environment %%--%%all'' | + | * ''conda remove %%--%%name crap_environment %%--%%all''\\ ''conda clean --all'' |
* Activating or going back to //root// | * Activating or going back to //root// | ||
* ''source activate existing_environment'' | * ''source activate existing_environment'' | ||
* ''source activate'' will activate the //root// environment | * ''source activate'' will activate the //root// environment | ||
+ | * You can use ''which python'' to determine where is the current Python you are using, or ''which -a python'' to display all the Python executables in the currrent search path | ||
* ''source deactivate'' | * ''source deactivate'' | ||
* Installed packages | * Installed packages | ||
Line 143: | Line 156: | ||
* Searching packages | * Searching packages | ||
* ''conda search %%--%%full-name package_exact_name''\\ ''conda search package_partial_name'' | * ''conda search %%--%%full-name package_exact_name''\\ ''conda search package_partial_name'' | ||
+ | * See also the ''%%--%%channel'' option below | ||
* Installing and removing packages\\ Add ''-n target_environment'' to act on another environment than the active one | * Installing and removing packages\\ Add ''-n target_environment'' to act on another environment than the active one | ||
* ''conda install package1 package2 packageN'' | * ''conda install package1 package2 packageN'' | ||
Line 151: | Line 165: | ||
* Updating conda (update the //conda// package, not the full Miniconda) | * Updating conda (update the //conda// package, not the full Miniconda) | ||
* ''conda update conda'' | * ''conda update conda'' | ||
+ | * Downgrading conda (required for installing UV-CDAT 2.10) | ||
+ | * <code>bash-4.1$ conda install -n root "conda<4.3.13" | ||
+ | Fetching package metadata ......... | ||
+ | Solving package specifications: . | ||
+ | |||
+ | Package plan for installation in environment /home/share/unix_files/cdat/miniconda2: | ||
+ | |||
+ | The following packages will be DOWNGRADED due to dependency conflicts: | ||
+ | |||
+ | conda: 4.3.13-py27_0 --> 4.3.11-py27_0 | ||
+ | |||
+ | Proceed ([y]/n)? y | ||
+ | |||
+ | conda-4.3.11-p 100% |###############################################################| Time: 0:00:00 4.28 MB/s | ||
+ | |||
+ | bash-4.1$ conda info | ||
+ | Current conda install: | ||
+ | |||
+ | platform : linux-64 | ||
+ | conda version : 4.3.11 | ||
+ | conda is private : False | ||
+ | conda-env version : 4.3.11</code> | ||
+ | |||
+ | * conda does not automatically clean the content of the ''pkgs'' directory! Use the following to //Remove index cache, lock files, tarballs, unused cache packages, and source cache// | ||
+ | * ''conda clean %%--%%all'' | ||
==== Misc ==== | ==== Misc ==== | ||
Line 156: | Line 195: | ||
* [[https://docs.continuum.io/anaconda/pkg-docs|Available 'official' packages]] | * [[https://docs.continuum.io/anaconda/pkg-docs|Available 'official' packages]] | ||
* [[https://anaconda.org/|Other packages]] | * [[https://anaconda.org/|Other packages]] | ||
+ | * [[https://conda-forge.github.io/feedstocks|Packages in conda-forge]] | ||
* [[https://conda.io/docs/|conda Full documentation]] | * [[https://conda.io/docs/|conda Full documentation]] | ||
* [[https://conda.io/docs/faq.html|conda FAQ]] | * [[https://conda.io/docs/faq.html|conda FAQ]] |