| Both sides previous revisionPrevious revisionNext revision | Previous revisionNext revisionBoth sides next revision |
| other:uvcdat:cdat_conda:miniconda3_install [2024/04/10 17:13] – [Fine-tuning conda to use (only) conda-forge] Improved jypeter | other:uvcdat:cdat_conda:miniconda3_install [2024/04/17 11:43] – [Useful commands] Improved jypeter |
|---|
| * Choose carefully where you will install //Miniconda3//, because the size of the installation directory will start at a few Gb and will keep on growing | * Choose carefully where you will install //Miniconda3//, because the size of the installation directory will start at a few Gb and will keep on growing |
| |
| * You //could// also start with the full //Anaconda// (instead of //Miniconda//) installer that will install a much more complete python environment, ready for use.\\ \\ **We choose not to use the full Anaconda installer** because: | * You //could// also start with the full //Anaconda// (instead of //Miniconda//) installer that will install a much more complete python environment, ready for use.\\ **We choose not to use the full //Anaconda// installer** because: |
| * it requires more disk space than //Miniconda3// at the beginning, | * it requires more disk space than //Miniconda3// during the initial installation, |
| * and all its packages come from the ''default'' //channel// (or //repository//) provided by the [[https://docs.conda.io/projects/conda/en/latest/glossary.html#conda-repository|conda repository]].\\ This is not very useful for us (and could even cause complex dependency problems later) because we will be mostly using (the same) packages provided by the [[https://conda-forge.org/feedstock-outputs/|conda-forge]] //channel//, **in order to avoid complex package dependency problems**. | * all the //Anaconda// installer packages come from the ''default'' //channel// (or //repository//) provided by the [[https://anaconda.org/|conda repository]].\\ This is not very useful for us (and could even cause complex dependency problems later) because we will be mostly using (the same) packages provided by the [[https://conda-forge.org/feedstock-outputs/|conda-forge]] //channel//, **in order to avoid complex package dependency problems**. |
| | |
| Note: some extra details are available on the much older page //Installing and maintaining UV-CDAT with conda//. You can check later the [[other:uvcdat:conda_notes#useful_conda_commands|Useful conda commands]], but the [[https://docs.conda.io/projects/conda/en/latest/index.html|official conda documentation]] and the [[https://docs.conda.io/projects/conda/en/latest/user-guide/cheatsheet.html|Cheat sheet]] are probably more up-to-date. | |
| |
| ===== Installing miniconda3 on a Linux-like computer ===== | ===== Installing miniconda3 on a Linux-like computer ===== |
| * e.g. Linux at **spiritx**: ''/homedata/$USER/miniconda3_2024-03'' | * e.g. Linux at **spiritx**: ''/homedata/$USER/miniconda3_2024-03'' |
| * Linux (including WSL) on a **personal PC/Mac desktop/laptop**: find a directory with enough space (including space for adding more packages and their future updates) | * Linux (including WSL) on a **personal PC/Mac desktop/laptop**: find a directory with enough space (including space for adding more packages and their future updates) |
| | * You may want to use a directory that you are not backing up, if you want to avoid having to save thousands of files... |
| |
| * Execute the installer with ''$ bash Miniconda3-latest-Linux-x86_64.sh'' | * Execute the installer with ''$ bash Miniconda3-latest-Linux-x86_64.sh'' |
| * e.g. (LSCE):<code>source /home/share/unix_files/cdat/miniconda3_2024-03/etc/profile.d/conda.csh | * e.g. (LSCE):<code>source /home/share/unix_files/cdat/miniconda3_2024-03/etc/profile.d/conda.csh |
| alias pynit 'conda activate base'</code> | alias pynit 'conda activate base'</code> |
| * When you have a custom Python environment installed (e.g. ''my_power_env'', you can update the ''pynit'' alias to initialize this environment instead of the ''base'' environment (e.g. ''conda activate my_power_env'' | * When you have a custom Python environment installed (e.g. ''my_power_env'', you can update the ''pynit'' alias to initialize this environment instead of the ''base'' environment (e.g. ''conda activate my_power_env'') |
| * You can also define other aliases, or use another name than ''pynit''! | * You can also define other aliases, or use another name than ''pynit''! |
| |
| We choose **not** to add a ''conda activate env_name'' line to the shell configuration files, in order to avoid future side effects. | We choose **not** to directly add a ''conda activate env_name'' line to the shell configuration files. This would permanently initialize ''env_name'' and may cause future (and obscure) side effects. |
| |
| When we open a new terminal, we get the default Python available on the system. If we need a specific Python environment, we just open a new window and then explicitly type ''conda activate env_name'' or the ''pynit'' alias defined above | When we open a new terminal, we get the default Python available on the system. If we need a specific Python environment, we just open a new window and then explicitly type ''conda activate env_name'' or the ''pynit'' alias defined above |
| * ask //tcsh// users to add to ''~/.cshrc'' something like\\ ''source ~main_installer_login/.conda3_jyp.csh''\\ \\ with a ''.conda3_jyp.csh'' file looking like {{ :other:uvcdat:cdat_conda:conda3_jyp.csh.txt |}} | * ask //tcsh// users to add to ''~/.cshrc'' something like\\ ''source ~main_installer_login/.conda3_jyp.csh''\\ \\ with a ''.conda3_jyp.csh'' file looking like {{ :other:uvcdat:cdat_conda:conda3_jyp.csh.txt |}} |
| |
| ==== Using mamba instead of conda ==== | ===== Keeping Miniconda3 up-to-date ===== |
| | |
| | * Just type ''conda update -n base %%--%%all'' as shown in the [[#updating_mini_conda|Updating (Mini)conda]] section |
| | |
| | * The ''base'' (miniconda3) and the other environments are all independent and can be updated independently |
| | |
| | * It is up to you to decide if you want to update environments or not |
| | ===== conda resources ===== |
| | |
| | |
| | |
| | |
| | ==== Web sites ==== |
| | |
| | * [[https://docs.conda.io/projects/conda/en/stable/|Official conda website]] |
| | * [[https://docs.conda.io/projects/conda/en/stable/user-guide/cheatsheet.html|Cheat sheet]] (2 pages, for people in a hurry) |
| | * [[https://docs.conda.io/projects/conda/en/stable/commands/|Commands' reference]] |
| | * Package repositories |
| | * [[https://conda-forge.org/packages/|conda-forge]] <= recommended source of packages |
| | * [[https://anaconda.org/|default]] |
| | |
| | |
| | ==== Useful commands ==== |
| | |
| | * Get help |
| | * ''conda -h'' |
| | * ''conda //command// %%--%%help'': help for a specific //command// |
| | * Available environments |
| | * ''conda env list'' |
| | * Activate an existing environment (available in ''conda env list'') or go back to the default Python |
| | * ''conda activate existing_environment'' |
| | * ''conda activate'' (without specifying an environment name) will **activate the ''base'' Miniconda3 environment** |
| | * Go back to the default Python available on the system |
| | * ''conda deactivate'' |
| | * You can type ''which python'' to determine where is the current ''python'' executable you are using is\\ You should **always know which Python you are using!** |
| | * <code>(base) $ which python |
| | /homedata/jypmce/miniconda3_2024-03/bin/python |
| | |
| | (base) $ conda deactivate |
| | |
| | $ which python |
| | /usr/bin/python</code> |
| | * Note: //before conda ''4.6''//, you had to type ''**source** activate existing_environment'' and ''**source** deactivate'' |
| | * Get the list of installed packages, their version, and where they come from (e.g. ''conda-forge'', ''pip'') |
| | * ''conda list [-n existing_environment]'' |
| | * Create an environment and install a few packages and their dependencies |
| | * ''conda create -n new_environment_name package1 package2 packageN'' |
| | * Delete all packages from an environment **and the environment** itself |
| | * ''conda remove -n crap_environment %%--%%all'' |
| | * Look for a package |
| | * ''conda search package_exact_name''\\ ''conda search '*package_partial_name*''' |
| | * Note: ''search'' with wildcards seems to fail in a tcsh shell, as of April 2024 ([[https://github.com/conda/conda/issues/9771|CSH install/search fail with '*' variable in an argument]]) |
| | * Installing and removing packages\\ Add ''-n target_environment'' to act on another environment than the active one |
| | * ''conda install package1 package2 packageN'' |
| | * ''conda'' should already be [[#fine-tuning_conda_to_use_only_conda-forge|configured to use conda-forge by default]] if you have read this page carefully |
| | * Add ''-c channel_url_or_alias'' to install from a specific //channel// |
| | * ''conda remove package1 package2 packageN'' |
| | * Updating conda (update the //conda// package, not the full Miniconda) |
| | * ''conda update -n base -c defaults conda'' |
| | * Downgrading conda (required for installing UV-CDAT 2.10) |
| | * <code>bash-4.1$ conda install -n root "conda<4.3.13" |
| | Fetching package metadata ......... |
| | Solving package specifications: . |
| | |
| | Package plan for installation in environment /home/share/unix_files/cdat/miniconda2: |
| | |
| | The following packages will be DOWNGRADED due to dependency conflicts: |
| | |
| | conda: 4.3.13-py27_0 --> 4.3.11-py27_0 |
| | |
| | Proceed ([y]/n)? y |
| | |
| | conda-4.3.11-p 100% |###############################################################| Time: 0:00:00 4.28 MB/s |
| | |
| | bash-4.1$ conda info |
| | Current conda install: |
| |
| Even when using a proper ''.condarc'' in order to get packages coming only from //conda-forge//, ''conda install'' may fail to solve the dependencies after meditating during an enormous time (e.g. [[https://github.com/conda/conda/issues/9367#issuecomment-793968239|Solving environment: failed with initial frozen solve. Retrying with flexible solve.]]), when we try to deal with **complex environments** (i.e. we already have lots of installed packages and we want to add even more packages). | platform : linux-64 |
| | conda version : 4.3.11 |
| | conda is private : False |
| | conda-env version : 4.3.11</code> |
| |
| Using CDAT as a base environment, and adding a lot of extra packages is a **complex environment**. | * conda does not automatically clean the content of the ''pkgs'' directory! Use the following to //Remove index cache, lock files, tarballs, unused cache packages, and source cache// |
| | * ''conda clean %%--%%all'' |
| |
| The only solution seems to replace ''conda'' with [[https://github.com/mamba-org/mamba|mamba]], a reimplementation of the conda package manager in C++ ([[https://wolfv.medium.com/making-conda-fast-again-4da4debfb3b7|Making conda fast again]]): | ==== Installation history ==== |
| * Install ''mamba'' in the //base// (i.e. //miniconda//) environment\\ ''conda install mamba -n base -c conda-forge'' | |
| * Use ''mamba'' instead of ''conda'' when installing packages.\\ Theoretically, you don't need to specify ''install -c conda forge'' any more, because ''conda-forge'' is now the default source of new packages\\ ''mamba install lots_of_packages'' | |
| |
| |
