JYP steps for installing Miniconda3

What? Why?

Miniconda3 is a minimal python environment/distribution that can be used for creating more complete work environments. It will basically make the conda command available on your computer, and you can then use conda to install and update more python packages and their dependencies.

A python environment is basically where you install python. A python distribution is the collection of packages you have chosen to install together in the environment.

You don't need to be (and you should not be) root when you install Miniconda3. You just need enough disk space where you have write access

If you are installing Miniconda3 in a Linux environment on a Windows 10 computer using Windows Subsystem for Linux (WSL), pay special attention to the instructions on the WSL lines

The way you do things will depend on how you are going to use python. There may be some slight differences if you are installing the environment just for you, or if you need a more stable environment because you are installing for multiple users.

You could also start with the Anaconda installer that will install a much more complete python environment, ready for use, but we choose not to do that because Anaconda requires more disk space at the beginning, and all its packages come from the default channel (or repository) provided by the conda repository. This is not very useful because we will be mostly using (the same) packages provided by the conda-forge channel.

Note: some extra details are available on the much older page Installing and maintaining UV-CDAT with conda. You can check later the Useful conda commands, but the official conda documentation and the Cheat sheet are probably more up-to-date.

Installing miniconda3 on a Linux-like computer

By Linux-like, we mean: A native Linux computer A windows 10 computer with WSL+Ubuntu installed A mac where you can use Linux in a terminal

Get the Python 3.8 Linux 64-bit (bash installer) (unless there is a more recent version?)

Find some temporary space on Linux (or Windows, if you are using WSL)

e.g. Linux at LSCE:
cd /home/scratch01/<your_login>

e.g. WSL, assuming that there is a C:\Scratch\<your_login> directory:
cd /mnt/c/Scratch/<your_login>

Use wget to download the installer:
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh (90 Mb, 26 Feb 2021)

Execute the installer

bash Miniconda3-latest-Linux-x86_64.sh

Accept the license

Note: at the end of the installation, answer no to the following question, so that the installer does not change your existing shell configuration files!
Do you wish the installer to initialize Miniconda3 by running conda init? [yes|no] => NO

Specify an explicit installation path outside of your home directory, with enough disk space (more than 3 Gb if you are going to install CDAT and some extra packages), preferably on a disk that is not backed up:

Installations by JYP:

Linux at LSCE: /home/share/unix_files/cdat/miniconda3<possibly_some_version_here>/

Linux on ciclad: /data/jypmce/cdat/miniconda3<possibly_some_version_here>/

WSL: installing outside of your /home/your_login/ directory does not work. You need to accept the installation in the default location: /home/<your_login>/miniconda3

The resulting miniconda3 directory size is 342M

> du -sh miniconda3
342M miniconda3
> cd miniconda3
> du -sh *
20M bin
0 compiler_compat
4.0K condabin
684K conda-meta
0 envs
16K etc
5.5M include
4.0K info
198M lib
12K LICENSE.txt
114M pkgs
604K share
4.0K shell
0 ssl
0 x86_64-conda_cos6-linux-gnu

Initialize the newly installed conda environment (this will initialize the environment only in the current terminal):

bash shell: source <installation_path>/miniconda3/etc/profile.d/conda.sh

tcsh shell: source <installation_path>/miniconda3/etc/profile.d/conda.csh

Check if you can use the conda command, and use it to initialize the base environment

$ which conda
<installation_path>/miniconda3/condabin/conda
$ which python
/usr/bin/python
$ conda activate
(base) $ which python
<installation_path>/miniconda3/bin/python

Update the new installation

$ conda update --all
[...]

During the update, the miniconda3 directory size goes from 432 Mb to 581 Mb. This directory will keep on growing, which is the reason why you should put it on a (preferably non backed up) disk where you have enough space

Make sure we have the latest conda package (just in case we did not get it with the update)
conda update -n base conda

Remove the installer later, when you have tester your installation:
rm Miniconda3-latest-Linux-x86_64.sh

Initializing conda in new terminals

When you open a terminal, your shell needs to know where to find the conda command used to initialize an environment, or switch between environments

General case

You were asked the following question when installing miniconda3: Do you wish the installer to initialize Miniconda3 by running conda init? [yes|no]

If you answered yes, the installer probably added some very complicated lines to your shell configuration files, but you probably have conda directly available when you open a new terminal

if you answered no (as suggested), use a text editor to add an extra line to the appropriate configuration file

bash user: add this line to ~/.bashrc
source <installation_path>/miniconda3/etc/profile.d/conda.sh

tcsh user: add this line to ~/.cshrc
source <installation_path>/miniconda3/etc/profile.d/conda.csh

We choose not to add a conda activate env_name line to the shell configuration files, in order to avoid side effects. When we open a new terminal, we get the default python available on the system. When we need a specific python environment, we just open a new window and then explicitly type: conda activate env_name

Multi-user installation

In the case of python environments maintained by a single user, but used by several users, we could do the same as in the General case, but it can be useful to have the users source an intermediate initialization file, that will then source the initialization file used in the general case. This makes it easier to maintain and change the environments, without asking users to make changes.

ask bash users to add to ~/.bashrc something like
source ~main_installer_login/.conda3_jyp.sh
with a .conda3_jyp.sh file looking like conda3_jyp.sh.txt

ask tcsh users to add to ~/.cshrc something like
source ~main_installer_login/.conda3_jyp.csh
with a .conda3_jyp.csh file looking like conda3_jyp.csh.txt

Fine-tuning conda

conda will probably work fine with the default settings, but we intend to create complex environments and it is better to use some specific settings

conda config documentation

Basic information: conda info

Full configuration: conda config --show

Changing the .condarc file

Most of the packages we will install will be provided by the conda-forge channel (-c conda-forge option). The installation steps will also add dependencies that will theoretically come from conda-forge, but could also come from the defaults channel.

The following will make sure that we only get packages from conda-forge (see Managing channels and Using multiple channels => How to fix it)

$ conda config --add channels conda-forge
$ conda config --set channel_priority strict
$ cat .condarc
channels:
- conda-forge
- defaults
channel_priority: strict

Using mamba instead of conda

Even when using a proper .condarc in order to get packages coming only from conda-forge, conda install may fail to solve the dependencies after meditating during an enormous time, when we specify complex environments. Using CDAT as a base environment, and adding a lot of extra packages is a complex environment.

The only solution seems to replace conda with mamba, a reimplementation of the conda package manager in C++ (Making conda fast again)

[ PMIP3 Wiki Home ] - [ Help! ] - [ Wiki syntax ]