• Miniconda3 is a minimal/bootstrap Python distribution that can be used for creating more complex Python distributions.
  It will basically make a recent conda command available on your computer, and you can then use this conda executable to create independent Python distributions, where you can install and update Python packages and their dependencies.

• A conda environment is basically where you install the Python that you will use (independently of Miniconda3 itself).
  A python distribution is the collection of packages you have chosen to install together in a given environment.
  • Example: initializing an environment named cdatm_py3

     >which conda
    conda:   aliased to source /path_to_miniconda/etc/profile.d/conda.csh
    
     > conda activate cdatm_py3
    
    (cdatm_py3) > which python
    /path_to_miniconda/envs/cdatm_py3/bin/python

  • The special environment where Miniconda itself is installed is called base. Note that the python binary of the base environment is not located in the same directory hierarchy of the other environments (i.e. there is no envs/ subdirectory

      > conda activate base
    
    (base) > which python
    /path_to_miniconda/bin/python

• You don't need to be (and you should not be) root when you install Miniconda3. You just need enough disk space on a disk where you have write access
  • WARNING: by default, miniconda will install itself and subsequent Python environments in a hidden sub-directory of your home (~/.conda on Linux). Do not use the default installation location, if you want to avoid disk space related problems (disk full, quota exceeded, …)
  • Choose carefully where you will install Miniconda3, because the size of the installation directory will start at a few Gb and will keep on growing

• If you are installing Miniconda3 in a Linux environment on a Windows 10 computer using Windows Subsystem for Linux (WSL), pay special attention to the specific instructions on the WSL lines

• You could also start with the full Anaconda installer that will install a much more complete python environment, ready for use.
  We choose not to use the full Anaconda because Anaconda requires more disk space at the beginning, and all its packages come from the default channel (or repository) provided by the conda repository. This is not very useful (and could even cause complex dependency problems) because we will be mostly using (the same) packages provided by the conda-forge channel.

Note: some extra details are available on the much older page Installing and maintaining UV-CDAT with conda. You can check later the Useful conda commands, but the official conda documentation and the Cheat sheet are probably more up-to-date.

• We will work with a download link that always point to the latest version of the installer
  • For the Linux computers we commonly use, we need the Miniconda3 Linux 64-bit link: https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
  • The installer is based on Python 3.12 as of March 2024

• Find some temporary space on Linux (or Windows, if you are using WSL)
  • e.g. Linux at LSCE:
    $ cd /home/scratch01/$USER
  • e.g. Linux at spiritx:

    $ mkdir /homedata/$USER/Scratch
    $ cd /homedata/$USER/Scratch

  • e.g. WSL, assuming that there is a C:\Scratch\<your_login> directory:
    cd /mnt/c/Scratch/<your_login>

• Use wget to download the latest installer (138 Mb as of 22 Mar 2024):
  

  $ wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
  $ ls -lh Miniconda3-latest-Linux-x86_64.sh
  [...] 138M Feb 27 20:40 Miniconda3-latest-Linux-x86_64.sh

We assume below that we are in the directory where we have downloaded the installer

• Determine the directory where you will install miniconda3 and all the Python environments based on this version of miniconda3, but do not create the directory (the installation script will do it)
  • Remember that you will need a few Gb of available space
  • At LSCE, do not install in a subdirectory of your home directory!
  • We add some date or version information at the end of the directory, in order to differentiate it from other (older of future) installations
  • e.g. Linux at LSCE: /home/share/unix_files/cdat/miniconda3_2024-03
  • e.g. Linux at spiritx: /homedata/$USER/miniconda3_2024-03
  • e.g. WSL

• Execute the installer with $ bash Miniconda3-latest-Linux-x86_64.sh
  • Review the license (type <SPACE> several times…) and accept it
  • When asked for the installation directory, use the directory name chosen above, and not the default directory
  • Answer no to the question Do you wish to update your shell profile to automatically initialize conda?. Otherwise the installation will make changes to your shell configuration files
    • If you forget to answer no, you can apparently:
      • remove the changes to your shell configuration files by typing later: conda init –reverse $SHELL
      • or at least disable the automatic activation of the environment (that might have side effect) by typing: conda config –set auto_activate_base false
    • The resulting miniconda3 directory size is 647 Mb (as of March 2024)
      

      $ du -sh miniconda3_2024-03
      647M    miniconda3_2024-03
      
      $ cd miniconda3_2024-03
      
      $ du -sh *
      47M     bin
      16K     cmake
      8.0K    compiler_compat
      32M     _conda
      8.0K    condabin
      968K    conda-meta
      4.0K    envs
      28K     etc
      18M     include
      317M    lib
      92K     LICENSE.txt
      1.1M    man
      231M    pkgs
      396K    sbin
      1.4M    share
      12K     shell
      8.0K    ssl
      8.0K    x86_64-conda_cos7-linux-gnu
      8.0K    x86_64-conda-linux-gnu

• Initialize the newly installed conda environment (this will initialize the environment only in the current terminal):
  • bash shell: source <installation_path>/miniconda3_2024-03/etc/profile.d/conda.sh
  • tcsh shell: source <installation_path>/miniconda3_2024-03/etc/profile.d/conda.csh

• Check if you can use the conda command, and use it to initialize the base environment

  • $ which conda
    <installation_path>/miniconda3/condabin/conda
    $ which python
    /usr/bin/python
    $ conda activate
    (base) $ which python
    <installation_path>/miniconda3/bin/python

• Update the new installation

  • $ conda update --all
    [...]

  • During the update, the miniconda3 directory size goes from 432 Mb to 581 Mb. This directory will keep on growing, which is the reason why you should put it on a (preferably non backed up) disk where you have enough space

• Make sure we have the latest conda package (just in case we did not get it with the update)
  conda update -n base conda

• Remove the installer later, when you have tester your installation:
  rm Miniconda3-latest-Linux-x86_64.sh

When you open a terminal, your shell needs to know where to find the conda command used to initialize an environment, or switch between environments

You were asked the following question when installing miniconda3: Do you wish the installer to initialize Miniconda3 by running conda init? [yes|no]

• If you answered yes, the installer probably added some very complicated lines to your shell configuration files, but you probably have conda directly available when you open a new terminal
• if you answered no (as suggested), use a text editor to add an extra line to the appropriate configuration file
  • bash user: add this line to ~/.bashrc
source <installation_path>/miniconda3/etc/profile.d/conda.sh
  • tcsh user: add this line to ~/.cshrc
source <installation_path>/miniconda3/etc/profile.d/conda.csh

We choose not to add a conda activate env_name line to the shell configuration files, in order to avoid side effects. When we open a new terminal, we get the default python available on the system. When we need a specific python environment, we just open a new window and then explicitly type: conda activate env_name

In the case of python environments maintained by a single user, but used by several users, we could do the same as in the General case, but it can be useful to have the users source an intermediate initialization file, that will then source the initialization file used in the general case. This makes it easier to maintain and change the environments, without asking users to make changes.

• ask bash users to add to ~/.bashrc something like
  source ~main_installer_login/.conda3_jyp.sh
  
  with a .conda3_jyp.sh file looking like conda3_jyp.sh.txt

• ask tcsh users to add to ~/.cshrc something like
  source ~main_installer_login/.conda3_jyp.csh
  
  with a .conda3_jyp.csh file looking like conda3_jyp.csh.txt

conda will probably work fine with the default settings for creating new environments with just one package (e.g. just PyFerret).

We intend to create complex environments (combining lots of packages, with complex dependencies) and it is highly recommended to use the specific settings described in this section

• conda config documentation
• Basic information: conda info
• Full configuration: conda config –show

Most of the packages we will install will be provided by the conda-forge channel (-c conda-forge option). The installation steps will also add dependencies that will theoretically come from conda-forge, but could also come from the defaults channel.

The following will make sure that we only get packages from conda-forge unless the requested packages really don't exist on conda-forge (see Managing channels and Using multiple channels => How to fix it)

$ conda config --add channels conda-forge
$ conda config --set channel_priority strict
$ cat ~/.condarc
channels:
  - conda-forge
  - defaults
channel_priority: strict

Even when using a proper .condarc in order to get packages coming only from conda-forge, conda install may fail to solve the dependencies after meditating during an enormous time (e.g. Solving environment: failed with initial frozen solve. Retrying with flexible solve.), when we try to deal with complex environments (i.e. we already have lots of installed packages and we want to add even more packages).

Using CDAT as a base environment, and adding a lot of extra packages is a complex environment.

The only solution seems to replace conda with mamba, a reimplementation of the conda package manager in C++ (Making conda fast again):

• Install mamba in the base (i.e. miniconda) environment
  conda install mamba -n base -c conda-forge
• Use mamba instead of conda when installing packages.
  Theoretically, you don't need to specify install -c conda forge any more, because conda-forge is now the default source of new packages
  mamba install lots_of_packages

[ PMIP3 Wiki Home ] - [ Help! ] - [ Wiki syntax ]