other:uvcdat:conda_notes
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision Last revision Both sides next revision | ||
|
other:uvcdat:conda_notes [2017/02/22 08:59] jypeter [Misc] Added link to conda-forge |
other:uvcdat:conda_notes [2019/11/05 13:45] jypeter [What should I install?] Added a link to "Why You Need Python Environments and How to Manage Them with Conda " |
||
|---|---|---|---|
| Line 2: | Line 2: | ||
| <WRAP center round tip 60%> | <WRAP center round tip 60%> | ||
| - | Check the [[.:cdat_conda|UV-CDAT conda-based versions installed by JYP]] page to see what is maintained by JYP | + | Check the [[.cdat_conda:index|UV-CDAT conda-based versions installed by JYP]] page to see what is maintained by JYP |
| </WRAP> | </WRAP> | ||
| - | Recent versions (2.8.0 and later?) of UV-CDAT are installed and maintained with the [[https://conda.io/docs/index.html|conda]] **package, dependency and environment manager**. This page is about conda/Miniconda/Anaconda installation and related notes | + | Versions 2.4.1 and later of UV-CDAT are installed and maintained with the [[https://conda.io/docs/index.html|conda]] **package, dependency and environment manager**. This page is about **conda/Miniconda/Anaconda** installation and related notes |
| ===== What should I install? ===== | ===== What should I install? ===== | ||
| Line 15: | Line 15: | ||
| * some parts of CDAT still need Python 2.7, so we will use the ''2'' version of //conda// (but it would also work with ''3'') | * some parts of CDAT still need Python 2.7, so we will use the ''2'' version of //conda// (but it would also work with ''3'') | ||
| * You can do almost anything with //conda//, and you are mostly limited by [[#disk_usage|disk space]]. | * You can do almost anything with //conda//, and you are mostly limited by [[#disk_usage|disk space]]. | ||
| - | * If you know that you are going to use UV-CDAT, you might as well use the ''conda'' from Miniconda to avoid uselessly duplicating packages downloaded from the official Anaconda repository (used by Anaconda) with the same packages downloaded from the conda-forge repository (used by UV-CDAT). | + | * If you know that you are going to use UV-CDAT, you might as well use the ''conda'' from //Miniconda// to avoid uselessly duplicating packages downloaded from the official Anaconda repository (used by Anaconda) with the same packages downloaded from the conda-forge repository (used by UV-CDAT). |
| + | * When you are done with this page (not just this section), you should have a look at the [[https://www.freecodecamp.org/news/why-you-need-python-environments-and-how-to-manage-them-with-conda-85f155f4353c/|Why You Need Python Environments and How to Manage Them with Conda]] page that covers interesting topics | ||
| ===== Miniconda ===== | ===== Miniconda ===== | ||
| + | |||
| + | <WRAP center round tip 60%> | ||
| + | Note: we have shown in [[.:cdat_conda:cdat_2_8_0|UV-CDAT 2.8.0 installation notes]] that using Miniconda for installing UV-CDAT requires indeed less space than using Anaconda | ||
| + | </WRAP> | ||
| Miniconda is basically just a self contained minimal Python distribution, with only the packages required to run //conda// | Miniconda is basically just a self contained minimal Python distribution, with only the packages required to run //conda// | ||
| Line 24: | Line 29: | ||
| * Follow the [[https://conda.io/docs/install/quick.html|Quick install]] steps to install Miniconda | * Follow the [[https://conda.io/docs/install/quick.html|Quick install]] steps to install Miniconda | ||
| - | * Get the [[https://repo.continuum.io/miniconda/Miniconda2-latest-Linux-x86_64.sh|Python 2.7 Linux 64-bit (bash installer)]] | + | * Get the [[https://repo.continuum.io/miniconda/Miniconda2-latest-Linux-x86_64.sh|Python 2.7 Linux 64-bit (bash installer)]] (unless there is a [[https://conda.io/miniconda.html|more recent version?]]) |
| - | * ''cd /home/scratch01/jypeter'' | + | * ''cd /home/scratch01/jypeter'' (some temporary storage space for the installer) |
| - | * ''wget https://repo.continuum.io/miniconda/Miniconda2-latest-Linux-x86_64.sh'' (28 Mo, 16 Feb 2017) | + | * ''wget https://repo.continuum.io/miniconda/Miniconda2-latest-Linux-x86_64.sh'' (28 Mo, 02 Aug 2017) |
| * Execute the installer | * Execute the installer | ||
| * ''bash Miniconda2-latest-Linux-x86_64.sh'' | * ''bash Miniconda2-latest-Linux-x86_64.sh'' | ||
| * Accept the license | * Accept the license | ||
| - | * Specify an explicit installation path **outside of the home directory**, with enough disk space, preferably on a disk that is not backed up: ''/home/share/unix_files/cdat/miniconda2'' | + | * Specify an explicit installation path **outside of the home directory**, with enough disk space (more than 3G if you are going to install UV-CDAT and some extra packages), preferably on a disk that is not backed up: ''/home/share/unix_files/cdat/miniconda2'' |
| * Resulting ''miniconda2'' directory size is 118M\\ <code> > du -sh miniconda2/ | * Resulting ''miniconda2'' directory size is 118M\\ <code> > du -sh miniconda2/ | ||
| 118M miniconda2/ | 118M miniconda2/ | ||
| Line 46: | Line 51: | ||
| 8.0K ssl</code> | 8.0K ssl</code> | ||
| * Accept the automatic update of the //PATH// variable in ''.bashrc'' (note: the installer automatically creates a backup of ''.bashrc'') | * Accept the automatic update of the //PATH// variable in ''.bashrc'' (note: the installer automatically creates a backup of ''.bashrc'') | ||
| - | * ''# added by Miniconda2 4.3.11 installer''\\ ''export PATH="/home/share/unix_files/cdat/miniconda2/bin:$PATH"'' | + | * ''# added by Miniconda2 4.3.21 installer''\\ ''export PATH="/home/share/unix_files/cdat/miniconda2/bin:$PATH"'' |
| * Open a new terminal (if you use bash) or type ''bash'' and check if the python from the conda distribution has been correctly installed | * Open a new terminal (if you use bash) or type ''bash'' and check if the python from the conda distribution has been correctly installed | ||
| * Update and clean the new installation | * Update and clean the new installation | ||
| - | * ''conda updpate --all'' | + | * ''conda update %%--%%all'' |
| - | * ''conda clean --all'' | + | * ''conda clean %%--%%all'' |
| * <code>bash-4.1$ which python | * <code>bash-4.1$ which python | ||
| /home/share/unix_files/cdat/miniconda2/bin/python | /home/share/unix_files/cdat/miniconda2/bin/python | ||
| Line 76: | Line 81: | ||
| ==== Installing Anaconda ==== | ==== Installing Anaconda ==== | ||
| - | * Follow the [[https://conda.io/docs/install/full.html|Full installation]] steps to install Miniconda | + | * Follow the [[https://conda.io/docs/install/full.html|Full installation]] steps (and what is detailed above in [[#installing_miniconda|Installing Miniconda]]) to install Anaconda |
| * Installation package ''Anaconda2-4.3.0-Linux-x86_64.sh'' is 462M (17 Feb 2017) | * Installation package ''Anaconda2-4.3.0-Linux-x86_64.sh'' is 462M (17 Feb 2017) | ||
| * Size after initial installation : 2G\\ <code>jypeter@asterix3 - ...anaconda2 - 93 >du -sh /home/share/unix_files/cdat/anaconda2 | * Size after initial installation : 2G\\ <code>jypeter@asterix3 - ...anaconda2 - 93 >du -sh /home/share/unix_files/cdat/anaconda2 | ||
| Line 113: | Line 118: | ||
| //conda// will use hard links and share packages common to several installed environments to minimize disk usage, but there is no harm in being careful and checking things! | //conda// will use hard links and share packages common to several installed environments to minimize disk usage, but there is no harm in being careful and checking things! | ||
| - | * You should avoid installing conda //stuff// in your home directory (''~/.conda'') | + | * Because of the likely large disk space usage, you should avoid installing conda //stuff// in your home directory (''~/.conda'') |
| * You should regularly monitor the size of the directories listed in ''conda info'' | * You should regularly monitor the size of the directories listed in ''conda info'' | ||
| * ''~/.conda'': the hidden conda directory in your home directory | * ''~/.conda'': the hidden conda directory in your home directory | ||
| Line 142: | Line 147: | ||
| * Deleting an environment | * Deleting an environment | ||
| * ''conda remove %%--%%name crap_environment %%--%%all''\\ ''conda clean --all'' | * ''conda remove %%--%%name crap_environment %%--%%all''\\ ''conda clean --all'' | ||
| - | * Activating or going back to //root// | + | * The following seems to do the same: ''conda env remove -n crap_environment'' |
| - | * ''source activate existing_environment'' | + | * Activating or going back to //root/base// |
| - | * ''source activate'' will activate the //root// environment | + | * ''conda activate existing_environment'' (used to be ''source activate existing_environment'' in older versions of conda) |
| - | * ''source deactivate'' | + | * ''conda activate'' will activate the //root/base// environment |
| + | * You can use ''which python'' to determine where is the current Python you are using, or ''which -a python'' (''-a'' will only work in //bash//) to display all the Python executables in the current search path | ||
| + | * ''conda deactivate'' (used to be ''source deactivate'') | ||
| * Installed packages | * Installed packages | ||
| * ''conda list'' | * ''conda list'' | ||
| Line 151: | Line 158: | ||
| * Searching packages | * Searching packages | ||
| * ''conda search %%--%%full-name package_exact_name''\\ ''conda search package_partial_name'' | * ''conda search %%--%%full-name package_exact_name''\\ ''conda search package_partial_name'' | ||
| + | * See also the ''%%--%%channel'' option below | ||
| * Installing and removing packages\\ Add ''-n target_environment'' to act on another environment than the active one | * Installing and removing packages\\ Add ''-n target_environment'' to act on another environment than the active one | ||
| * ''conda install package1 package2 packageN'' | * ''conda install package1 package2 packageN'' | ||
| Line 158: | Line 166: | ||
| * ''conda remove package1 package2 packageN'' | * ''conda remove package1 package2 packageN'' | ||
| * Updating conda (update the //conda// package, not the full Miniconda) | * Updating conda (update the //conda// package, not the full Miniconda) | ||
| - | * ''conda update conda'' | + | * ''conda update -n base -c defaults conda'' |
| + | * Downgrading conda (required for installing UV-CDAT 2.10) | ||
| + | * <code>bash-4.1$ conda install -n root "conda<4.3.13" | ||
| + | Fetching package metadata ......... | ||
| + | Solving package specifications: . | ||
| + | |||
| + | Package plan for installation in environment /home/share/unix_files/cdat/miniconda2: | ||
| + | |||
| + | The following packages will be DOWNGRADED due to dependency conflicts: | ||
| + | |||
| + | conda: 4.3.13-py27_0 --> 4.3.11-py27_0 | ||
| + | |||
| + | Proceed ([y]/n)? y | ||
| + | |||
| + | conda-4.3.11-p 100% |###############################################################| Time: 0:00:00 4.28 MB/s | ||
| + | |||
| + | bash-4.1$ conda info | ||
| + | Current conda install: | ||
| + | |||
| + | platform : linux-64 | ||
| + | conda version : 4.3.11 | ||
| + | conda is private : False | ||
| + | conda-env version : 4.3.11</code> | ||
| * conda does not automatically clean the content of the ''pkgs'' directory! Use the following to //Remove index cache, lock files, tarballs, unused cache packages, and source cache// | * conda does not automatically clean the content of the ''pkgs'' directory! Use the following to //Remove index cache, lock files, tarballs, unused cache packages, and source cache// | ||
| - | * ''conda clean --all'' | + | * ''conda clean %%--%%all'' |
| + | |||
| + | ==== Installation history ==== | ||
| + | |||
| + | You can get the current list of installed packages with ''conda list'', but it may also be useful to know the installation order: | ||
| + | * ''conda list %%--%%revisions'' revisions will display **exactly** what was installed and when (including the updates, and automatic dependencies installation) | ||
| + | * You can theoretically roll back to a previous //revision// with: ''conda install %%--%%revision RR'' | ||
| + | * There is more information in the ''conda-meta/history'' installation log files. e.g., if you need to know the list of installation commands used:\\ <code> $ egrep -e 'cmd:' -e '==>' /home/share/unix_files/cdat/miniconda3/envs/cdatm18_py2/conda-meta/history | ||
| + | ==> 2019-03-07 16:19:55 <== | ||
| + | # cmd: /home/share/unix_files/cdat/miniconda3/bin/conda create -n cdatm18_py2 --clone cdat-8.1_py2 | ||
| + | ==> 2019-03-07 16:35:11 <== | ||
| + | # cmd: /home/share/unix_files/cdat/miniconda3/bin/conda install -n cdatm18_py2 -c conda-forge pillow pandas statsmodels seaborn scikit-image seawater gsw netcdf4 pyferret basemap-data-hires xlsxwriter cmocean rpy2 gdal windspharm | ||
| + | ==> 2019-03-07 16:57:50 <== | ||
| + | # cmd: /home/share/unix_files/cdat/miniconda3/bin/conda install -n cdatm18_py2 -c conda-forge -c vacumm vacumm | ||
| + | ==> 2019-03-12 14:35:31 <== | ||
| + | # cmd: /home/share/unix_files/cdat/miniconda3/bin/conda install -n cdatm18_py2 -c conda-forge pillow pandas statsmodels seaborn scikit-image seawater gsw netcdf4 pyferret basemap-data-hires xlsxwriter cmocean rpy2 gdal windspharm | ||
| + | ==> 2019-03-12 15:49:54 <== | ||
| + | # cmd: /home/share/unix_files/cdat/miniconda3/bin/conda install -n cdatm18_py2 -c stefraynaud -c conda-forge spanlib | ||
| + | ==> 2019-03-12 17:30:26 <== | ||
| + | # cmd: /home/share/unix_files/cdat/miniconda3/bin/conda install -n cdatm18_py2 -c conda-forge cmor</code> | ||
| + | * You can also, use a script and the ''conda'' module available **in the ''base'' environment**: [[https://github.com/conda/conda/issues/4545#issuecomment-469984684|example]] | ||
| ==== Misc ==== | ==== Misc ==== | ||
| Line 169: | Line 220: | ||
| * [[https://conda.io/docs/|conda Full documentation]] | * [[https://conda.io/docs/|conda Full documentation]] | ||
| * [[https://conda.io/docs/faq.html|conda FAQ]] | * [[https://conda.io/docs/faq.html|conda FAQ]] | ||
| - | * [[https://conda.io/docs/using/envs.html#share-an-environment|Sharing (replicating) an environment]] | + | * [[https://conda.io/docs/user-guide/tasks/manage-environments.html#sharing-an-environment|Sharing (replicating) an environment]] |
| * [[https://conda.io/docs/install/central.html|Centralized installation]] | * [[https://conda.io/docs/install/central.html|Centralized installation]] | ||
other/uvcdat/conda_notes.txt · Last modified: 2021/12/21 10:33 by jypeter
