This is an old revision of the document!
conda
command available on your computer, and you can then use this conda
executable to select an existing distribution, or create new and independent Python distributions.activate
this environment in order to use itcdatm_py3
(assuming it exists)$ source /path_to_miniconda/etc/profile.d/conda.sh $ which conda /path_to_miniconda/condabin/conda $ conda env list [...] cdatm_py3 /path_to_miniconda/envs/cdatm_py3 some_other_env /path_to_miniconda/envs/some_other_env $ conda activate cdatm_py3 (cdatm_py3) $ which python /path_to_miniconda/envs/cdatm_py3/bin/python
base
. Note that the python
binary of the special base
environment is not located in the same directory hierarchy as the python
of the other environments (i.e. there is no envs/
subdirectory$ conda activate base (base) $ which python /path_to_miniconda/bin/python
~/miniconda3
on Linux). Do not use the default installation location, if you want to avoid disk space related problems (disk full, quota exceeded, …)~/.conda
on Linux) for the installationdefault
channel (or repository) provided by the conda repository.Note: some extra details are available on the much older page Installing and maintaining UV-CDAT with conda. You can check later the Useful conda commands, but the official conda documentation and the Cheat sheet are probably more up-to-date.
By Linux-like, we mean:
Pure Windows Note: you can also follow the steps listed in the Windows section of "Quick command line install" to install a pure Windows version of miniconda3, and then follow the steps listed in the next sections of this page
PS C: cd C:\Scratch\jypeter PS C:\Scratch\jypeter> curl https://repo.anaconda.com/miniconda/Miniconda3-latest-Windows-x86_64.exe -o miniconda.exe PS C:\Scratch\jypeter> .\miniconda.exe PS C:\Scratch\jypeter> del .\miniconda.exe
$ cd /home/scratch01/$USER
$ mkdir /homedata/$USER/Scratch $ cd /homedata/$USER/Scratch
wget
to download the latest installer (138 Mb as of 22 Mar 2024):$ wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh $ ls -lh Miniconda3-latest-Linux-x86_64.sh [...] 138M Feb 27 20:40 Miniconda3-latest-Linux-x86_64.sh
We assume below that we are in the directory where we have downloaded the installer
/path_to_miniconda3/
/home/share/unix_files/cdat/miniconda3_2024-03
/homedata/$USER/miniconda3_2024-03
$ bash Miniconda3-latest-Linux-x86_64.sh
<SPACE>
several times…) and accept it/path_to_miniconda3/
$HOME/miniconda3
no
to the question Do you wish to update your shell profile to automatically initialize conda?
.no
, you can apparently:conda init –reverse $SHELL
conda config –set auto_activate_base false
miniconda3
directory size is 647 Mb (as of March 2024)$ du -sh miniconda3_2024-03 647M miniconda3_2024-03 $ du -sh miniconda3_2024-03/* 47M miniconda3_2024-03/bin 16K miniconda3_2024-03/cmake 8.0K miniconda3_2024-03/compiler_compat 32M miniconda3_2024-03/_conda 8.0K miniconda3_2024-03/condabin 976K miniconda3_2024-03/conda-meta 4.0K miniconda3_2024-03/envs 28K miniconda3_2024-03/etc 18M miniconda3_2024-03/include 317M miniconda3_2024-03/lib 92K miniconda3_2024-03/LICENSE.txt 1.1M miniconda3_2024-03/man 232M miniconda3_2024-03/pkgs 396K miniconda3_2024-03/sbin 1.5M miniconda3_2024-03/share 12K miniconda3_2024-03/shell 8.0K miniconda3_2024-03/ssl 8.0K miniconda3_2024-03/x86_64-conda_cos7-linux-gnu 8.0K miniconda3_2024-03/x86_64-conda-linux-gnu
source /path_to_miniconda3/etc/profile.d/conda.sh
source /homedata/jypmce/miniconda3_2024-03/etc/profile.d/conda.sh
source /path_to_miniconda3/etc/profile.d/conda.csh
source /home/share/unix_files/cdat/miniconda3_2024-03/etc/profile.d/conda.csh
conda
command, and use it to initialize the base environment$ which conda $ which python /usr/bin/python $ source /homedata/jypmce/miniconda3_2024-03/etc/profile.d/conda.sh $ which conda /homedata/jypmce/miniconda3_2024-03/condabin/conda $ which python /usr/bin/python $ conda activate base (base) $ which conda /homedata/jypmce/miniconda3_2024-03/bin/conda (base) $ which python /homedata/jypmce/miniconda3_2024-03/bin/python (base) $ conda deactivate $ which conda /homedata/jypmce/miniconda3_2024-03/condabin/conda $ which python /usr/bin/python
$ rm Miniconda3-latest-Linux-x86_64.sh
conda
will probably work fine with the default settings if you create simple new environments with just one package and its dependencies.conda
also works much better and faster since the end of 2023 (conda
versions starting at 23.10
), now that it is using the mamba solver instead of the default solver.conda
, try to install and use mamba, as a drop-in replacement of condaconda
used libmamba
, we had to use mamba
(instead of conda
) for dealing with our complex Python environments:conda-forge
, with complex dependencies, and we may run into dependency problems when combining packages coming from both conda and conda-forge.
The following will make sure that we only get packages from conda-forge (same thing as using the -c conda-forge
option) unless the requested packages really don't exist on conda-forge. More details in Managing channels
$ cat ~/.condarc cat: /home/jypmce/.condarc: No such file or directory $ conda config --get channels $ conda config --prepend channels conda-forge $ conda config --set channel_priority strict $ conda config --get channels --add channels 'defaults' # lowest priority --add channels 'conda-forge' # highest priority $ cat ~/.condarc channels: - conda-forge - defaults channel_priority: strict
conda info
conda config –show
We have to completely update Miniconda3 at least once after installing Miniconda3 and making conda-forge the highest priority channel
conda
$ conda list -n base | grep conda # packages in environment at /path_to_miniconda: conda 24.1.2 py312h06a4308_0 conda-libmamba-solver 23.12.0 pyhd3eb1b0_1 [... and other packages with 'conda' in their name]
$ conda update -n base --all Channels: - conda-forge - defaults Platform: linux-64 Collecting package metadata (repodata.json): done Solving environment: done ## Package Plan ## environment location: /homedata/jypmce/miniconda3_2024-03 The following packages will be downloaded: package | build ---------------------------|----------------- conda-24.3.0 | py312h7900ff3_0 1.1 MB conda-forge conda-libmamba-solver-24.1.0| pyhd8ed1ab_0 40 KB conda-forge python-3.12.2 |hab00c5b_0_cpython 30.8 MB conda-forge [...] The following NEW packages will be INSTALLED: [...] The following packages will be UPDATED: [...] conda pkgs/main::conda-24.1.2-py312h06a4308~ --> conda-forge::conda-24.3.0-py312h7900ff3_0 [...] The following packages will be SUPERSEDED by a higher-priority channel: [...] Proceed ([y]/n)? y [...] Preparing transaction: done Verifying transaction: done Executing transaction: done
conda
. Note that (almost) all the packages should now specify that they are provided by conda-forge
$ conda list -n base conda 24.3.0 py312h7900ff3_0 conda-forge conda-libmamba-solver 24.1.0 pyhd8ed1ab_0 conda-forge python 3.12.2 hab00c5b_0_cpython conda-forge
conda-forge
$ conda list -n base | grep -v conda-forge # Name Version Build Channel libedit 3.1.20230828 h5eee18b_0 libffi 3.4.4 h6a678d5_0 xz 5.4.5 h5eee18b_0
conda-forge
.anaconda
channel were more recent than the same packages provided by the conda-forge
channel when the packages were updatedconda
command.conda
$ conda update -n base --all [...] # All requested packages already installed. (base) $ conda update -n base conda [...] # All requested packages already installed.
miniconda3
directory size grew from 763 Mb to 1.6 Gb, and we have not installed any custom Python environment yet! This is the reason why you should install Miniconda3 on a (preferably non backed up) disk where you have enough space$ du -sh /homedata/jypmce/miniconda3_2024-03 736M /homedata/jypmce/miniconda3_2024-03 [... update] $ du -sh /homedata/jypmce/miniconda3_2024-03 1.6G /homedata/jypmce/miniconda3_2024-03
clean
once you have installed new Python environments (unless you are desperate for disk space and know how to do a complete re-installation if something breaks…)(base) $ conda clean --all Will remove 154 (180.9 MB) tarball(s). Proceed ([y]/n)? y Will remove 1 index cache(s). Proceed ([y]/n)? y Will remove 74 (378.0 MB) package(s). Proceed ([y]/n)? y There are no tempfile(s) to remove. There are no logfile(s) to remove. (base) $ du -sh /homedata/jypmce/miniconda3_2024-03 457M /homedata/jypmce/miniconda3_2024-03
When you open a terminal, your shell needs to know where to find the conda
command used to initialize an environment, or switch between existing environments. This can be configured in the shell configuration files
You were asked the following question when installing Miniconda3: Do you wish to update your shell profile to automatically initialize conda? [yes|no]
conda
and the newly installer python
are probably directly available when you open a new terminal.~/.bashrc
source /path_to_miniconda3/etc/profile.d/conda.sh alias pynit='conda activate base'
source /homedata/jypmce/miniconda3_2024-03/etc/profile.d/conda.sh alias pynit='conda activate base'
~/.cshrc
source /path_to_miniconda3/etc/profile.d/conda.csh'' alias pynit 'conda activate base'
source /home/share/unix_files/cdat/miniconda3_2024-03/etc/profile.d/conda.csh alias pynit 'conda activate base'
my_power_env
, you can update the pynit
alias to initialize this environment instead of the base
environment (e.g. conda activate my_power_env
pynit
!
We choose not to add a conda activate env_name
line to the shell configuration files, in order to avoid future side effects.
When we open a new terminal, we get the default Python available on the system. If we need a specific Python environment, we just open a new window and then explicitly type conda activate env_name
or the pynit
alias defined above
In the case of python environments maintained by a single user, but used by several users, we could do the same as in the General case, but it can be useful to have the users source an intermediate initialization file, that will then source the initialization file used in the general case. This makes it easier to maintain and change the environments, without asking users to make changes.
~/.bashrc
something likesource ~main_installer_login/.conda3_jyp.sh
.conda3_jyp.sh
file looking like conda3_jyp.sh.txt~/.cshrc
something likesource ~main_installer_login/.conda3_jyp.csh
.conda3_jyp.csh
file looking like conda3_jyp.csh.txt
Even when using a proper .condarc
in order to get packages coming only from conda-forge, conda install
may fail to solve the dependencies after meditating during an enormous time (e.g. Solving environment: failed with initial frozen solve. Retrying with flexible solve.), when we try to deal with complex environments (i.e. we already have lots of installed packages and we want to add even more packages).
Using CDAT as a base environment, and adding a lot of extra packages is a complex environment.
The only solution seems to replace conda
with mamba, a reimplementation of the conda package manager in C++ (Making conda fast again):
mamba
in the base (i.e. miniconda) environmentconda install mamba -n base -c conda-forge
mamba
instead of conda
when installing packages.install -c conda forge
any more, because conda-forge
is now the default source of new packagesmamba install lots_of_packages
[ PMIP3 Wiki Home ] - [ Help! ] - [ Wiki syntax ]