User Tools

Site Tools


other:uvcdat:cdat_conda:miniconda3_install

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
other:uvcdat:cdat_conda:miniconda3_install [2021/03/09 10:37]
jypeter [Fine-tuning conda]
other:uvcdat:cdat_conda:miniconda3_install [2024/03/28 13:09] (current)
jypeter [Initializing conda in new terminals]
Line 1: Line 1:
 ====== JYP steps for installing Miniconda3 ====== ====== JYP steps for installing Miniconda3 ======
  
-===== What? Why? =====+===== Why should you use Miniconda3 ​? =====
  
-  * //​Miniconda3//​ is a minimal ​python environment/​distribution that can be used for creating more complete work environments. It will basically make the ''​conda''​ command available on your computer, and you can then use ''​conda''​ to install and update more python packages ​and their dependencies.+  ​* **//​Miniconda3//​ is a minimal/bootstrap Python ​distribution** that can be used for creating more complex Python distributions.\\ It will basically make a recent ​''​conda''​ command available on your computer, and you can then use this ''​conda'' ​executable ​to select an existing distribution,​ or create new and independent Python distributions.
  
-  * A python ​//​environment//​ is basically where you install ​python. A python //​distribution//​ is the collection of packages you have chosen to install together in the environment.+  * A **conda ​//​environment//​** is basically ​(the short name of the directory) ​where you install ​the Python that you will use (independently of //​Miniconda3//​ itself)You have to ''​activate''​ this environment in order to use it\\ **python //​distribution//​** is the collection of packages ​(and their dependencies) ​you have chosen to install together in a given environment. 
 +    * Example: initializing an environment named ''​cdatm_py3''​ (assuming it exists)<​code>​$ source /​path_to_miniconda/​etc/​profile.d/​conda.sh
  
-  * You don't need to be (and **you should not be**) //root// when you install Miniconda3. You just need enough disk space where you have write access+$ which conda 
 +/path_to_miniconda/condabin/conda
  
-  * If you are installing Miniconda3 in a Linux environment on a [[other:​win10wsl|Windows 10 computer using Windows Subsystem for Linux (WSL)]], pay special attention to the instructions on the **WSL** lines+$ conda env list 
 +[...] 
 +cdatm_py3 ​               /​path_to_miniconda/​envs/​cdatm_py3 
 +some_other_env ​          /​path_to_miniconda/​envs/​some_other_env
  
-  * The way you do things will depend on how you are going to use python. There may be some slight differences if you are installing the environment just for you, or if you need a more stable environment because you are installing for multiple users.+$ conda activate cdatm_py3
  
-  ​* You could also start with the //​Anaconda//​ installer that will install a much more complete python environment,​ ready for use, but we choose not to do that because ​Anaconda requires more disk space at the beginning, and all its packages come from the ''​default''​ //channel// (or //​repository//​) provided by the [[https://​docs.conda.io/​projects/​conda/​en/​latest/​glossary.html#​conda-repository|conda repository]]. This is not very useful because we will be mostly using (the same) packages provided by the [[https://​conda-forge.org/​feedstock-outputs/​|conda-forge]] channel.+(cdatm_py3) $ which python 
 +/​path_to_miniconda/​envs/​cdatm_py3/​bin/​python 
 +</​code>​ 
 +    * The special environment where //​Miniconda3//​ itself is installed is called ''​base''​. Note that the ''​python''​ binary of the ''​base''​ environment is not located in the same directory hierarchy of the other environments (i.e. there is no ''​envs/''​ subdirectory<​code>​$ conda activate base 
 + 
 +(base) $ which python 
 +/​path_to_miniconda/​bin/​python 
 +</​code>​ 
 + 
 +  * You **don'​t need to be (and you should not be) //root//** when you install //​Miniconda3//​. You just **need //enough// disk space** on a disk where you have write access 
 +    * **WARNING**:​ by default, //​Miniconda3//​ will install itself and subsequent Python environments in a sub-directory of your home directory (''​~/​miniconda3''​ on Linux). **Do not use the default installation location**, if you want to avoid disk space related problems (//disk full//, //quota exceeded//, ...) 
 +      * **WARNING**:​ older versions of //​Miniconda3//​ used a **hidden sub-directory** of your home directory (''​~/​.conda''​ on Linux) for the installation 
 +    * Choose carefully where you will install //​Miniconda3//,​ because the size of the installation directory will start at a few Gb and will keep on growing 
 + 
 +  ​* You could also start with the full //​Anaconda//​ installer that will install a much more complete python environment,​ ready for use.\\ **We choose not to use the full Anaconda ​installer** because it requires more disk space than //​Miniconda3// ​at the beginning, and all its packages come from the ''​default''​ //channel// (or //​repository//​) provided by the [[https://​docs.conda.io/​projects/​conda/​en/​latest/​glossary.html#​conda-repository|conda repository]]. This is not very useful ​(and could even cause complex dependency problems) ​because we will be mostly using (the same) packages provided by the [[https://​conda-forge.org/​feedstock-outputs/​|conda-forge]] ​//channel//, **in order to avoid complex package dependency problems**.
  
 Note: some extra details are available on the much older page //​Installing and maintaining UV-CDAT with conda//. You can check later the [[other:​uvcdat:​conda_notes#​useful_conda_commands|Useful conda commands]], but the [[https://​docs.conda.io/​projects/​conda/​en/​latest/​index.html|official conda documentation]] and the [[https://​docs.conda.io/​projects/​conda/​en/​latest/​user-guide/​cheatsheet.html|Cheat sheet]] are probably more up-to-date. Note: some extra details are available on the much older page //​Installing and maintaining UV-CDAT with conda//. You can check later the [[other:​uvcdat:​conda_notes#​useful_conda_commands|Useful conda commands]], but the [[https://​docs.conda.io/​projects/​conda/​en/​latest/​index.html|official conda documentation]] and the [[https://​docs.conda.io/​projects/​conda/​en/​latest/​user-guide/​cheatsheet.html|Cheat sheet]] are probably more up-to-date.
Line 21: Line 40:
 <WRAP center round tip 60%> <WRAP center round tip 60%>
 By //​Linux-like//,​ we mean: By //​Linux-like//,​ we mean:
-  * A native Linux computer+  * A native Linux computer ​(or server)
   * A windows 10 computer with [[other:​win10wsl|WSL+Ubuntu installed]]   * A windows 10 computer with [[other:​win10wsl|WSL+Ubuntu installed]]
-  * A mac where you can use Linux in a terminal+  * A Mac where you can use Linux in a terminal
 </​WRAP>​ </​WRAP>​
  
  
-  * Get the [[https://​repo.anaconda.com/​miniconda/​Miniconda3-latest-Linux-x86_64.sh|Python 3.8 Linux 64-bit (bash installer)]] (unless there is a [[https://​conda.io/​miniconda.html|more recent version?​]]) +==== Downloading ​the Miniconda3 installer ​====
-    * Find some temporary space on Linux (or Windows, if you are using [[other:​win10wsl|WSL]]) +
-      * e.g. Linux at LSCE:\\ ''​cd /​home/​scratch01/<​your_login>''​ +
-      * e.g. WSL, assuming that there is a ''​C:​\Scratch\<​your_login>''​ directory:​\\ ''​cd /​mnt/​c/​Scratch/<​your_login>''​ +
-    * Use ''​wget''​ to download the installer:​\\ ''​wget https://​repo.anaconda.com/​miniconda/​Miniconda3-latest-Linux-x86_64.sh''​ (90 Mb, 26 Feb 2021)+
  
-  * Execute ​the installer +  * We use a download link that always point to the [[https://docs.anaconda.com/free/miniconda/#​latest-miniconda-installer-links|latest version of the Linux installer]] 
-    * ''​bash Miniconda3-latest-Linux-x86_64.sh''​ +    For the Linux computers we commonly usewe need the //Miniconda3 Linux 64-bit// linkhttps://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh 
-      * Accept the license +      Note: Mac users should use the latest [[https://docs.anaconda.com/free/miniconda/#​latest-miniconda-installer-links|macOS installer]] appropriate for their computer 
-      * Noteat the end of the installation,​ **answer ​//no// to the following question**, so that the installer ​does **not** change your existing shell configuration files!\\ ''​Do you wish the installer ​to initialize Miniconda3 by running conda init? [yes|no]''​ => **NO** +    * The installer ​is based on **Python 3.12 as of March 2024**
-      * Specify an explicit installation path **outside of your //home// directory**, with enough disk space (more than 3 Gb if you are going to install CDAT and some extra packages), preferably on a disk that is not backed up: +
-        * Installations by //JYP//: +
-          * Linux at LSCE''​/home/share/unix_files/cdat/​miniconda3<​possibly_some_version_here>/''​ +
-          * Linux on ciclad: ''/​data/​jypmce/​cdat/​miniconda3<​possibly_some_version_here>/''​ +
-        WSLinstalling outside of your ''​/home/your_login/''​ directory <wrap hi>does not work</​wrap>​You need to accept the installation in the default location: ''​/home/<​your_login>​/miniconda3''​ +
-      * The resulting ''​miniconda3''​ directory size is 342M\\ <​code>​ > du -sh miniconda3 +
-342M    miniconda3+
  
- > cd miniconda3+  * Find some temporary space on Linux (or Windows, if you are using [[other:​win10wsl|WSL]]) 
 +    * e.g. Linux at **LSCE**:\\ ''​$ cd /​home/​scratch01/​$USER''​ 
 +    * e.g. Linux at **spiritx**:<​code>$ mkdir /​homedata/​$USER/​Scratch 
 +cd /​homedata/​$USER/​Scratch</​code>​ 
 +    * e.g. Linux on a **personal PC/Mac desktop/​laptop**:​ use a scratch/​temporary directory with //enough// space
  
- du -sh * +  * Use ''​wget''​ to download the latest installer (138 Mb as of 22 Mar 2024):\\ <code>$ wget https://​repo.anaconda.com/​miniconda/​Miniconda3-latest-Linux-x86_64.sh 
-20M     bin +$ ls -lh Miniconda3-latest-Linux-x86_64.sh 
-0       ​compiler_compat +[...] 138M Feb 27 20:40 Miniconda3-latest-Linux-x86_64.sh</​code>​
-4.0K    condabin +
-684K    conda-meta +
-0       ​envs +
-16K     etc +
-5.5M    include +
-4.0K    info +
-198M    lib +
-12K     ​LICENSE.txt +
-114M    pkgs +
-604K    share +
-4.0K    shell +
-0       ssl +
-0       ​x86_64-conda_cos6-linux-gnu</​code>​+
  
-  ​* Initialize the newly installed conda environment (this will initialize the environment only in the current terminal):​ +==== Using the Miniconda3 installer ==== 
-    * bash shell: ''​source ​<​installation_path>​/miniconda3/​etc/​profile.d/​conda.sh''​ + 
-    * tcsh shell: ''​source ​<​installation_path>​/miniconda3/​etc/​profile.d/​conda.csh''​+We assume below that we are in the [[#​downloading_the_installer|directory where we have downloaded the installer]] 
 + 
 +  * **Determine the directory where you will install miniconda3 and all the Python environments** based on this version of miniconda3, but **do not create the directory** (the installation script will do it)\\ **We will later refer to this location as ''/​path_to_miniconda3/''​** 
 +    * Remember that you will need a few Gb of available space 
 +    * At LSCE, do not install in a subdirectory of your home directory! 
 +      * On other servers and computers, you should also **avoid installing in your //home// directory**,​ if you want to be able to easily make backups or your //home//, without backing up thousands of files related to your Python installation... 
 +    * We add some date or version information at the end of the directory, in order to differentiate it from other (older of future) installations 
 +    * e.g. Linux at **LSCE**: ''/​home/​share/​unix_files/​cdat/​miniconda3_2024-03''​ 
 +    * e.g. Linux at **spiritx**:​ ''/​homedata/​$USER/​miniconda3_2024-03''​ 
 +    * Linux (including WSL) on a **personal PC/Mac desktop/​laptop**:​ find a directory with enough space (including space for adding more packages and their future updates) 
 + 
 +  * Execute the installer with ''​$ bash Miniconda3-latest-Linux-x86_64.sh''​ 
 +    * Review the license (type ''<​SPACE>''​ several times...) and accept it 
 +    * When asked for the installation directory, specify the directory location chosen in the previous step, and **not the default directory** 
 +      * e.g. specify\\ ''/​path_to_miniconda3/''​\\ instead of the default\\ ''​$HOME/​miniconda3''​ 
 +    * **Answer ''​no''​ to the question ''​Do you wish to update your shell profile to automatically initialize conda?''​**.\\ Otherwise the installation will make changes to your shell configuration files 
 +      * If you forget to answer ''​no'',​ you can apparently:​ 
 +        * remove the changes to your shell configuration files by typing later: ''​conda init --reverse $SHELL''​ 
 +        * or at least disable the automatic activation of the environment (that might have side effect) by typing: ''​conda config --set auto_activate_base false''​ 
 +      * The resulting ''​miniconda3''​ directory size is 647 Mb (as of March 2024)\\ <​code>​$ du -sh miniconda3_2024-03 
 +647M    miniconda3_2024-03 
 + 
 +$ du -sh miniconda3_2024-03/​* 
 +47M     ​miniconda3_2024-03/​bin 
 +16K     ​miniconda3_2024-03/​cmake 
 +8.0K    miniconda3_2024-03/​compiler_compat 
 +32M     ​miniconda3_2024-03/​_conda 
 +8.0K    miniconda3_2024-03/​condabin 
 +976K    miniconda3_2024-03/​conda-meta 
 +4.0K    miniconda3_2024-03/​envs 
 +28K     ​miniconda3_2024-03/​etc 
 +18M     ​miniconda3_2024-03/​include 
 +317M    miniconda3_2024-03/​lib 
 +92K     ​miniconda3_2024-03/​LICENSE.txt 
 +1.1M    miniconda3_2024-03/​man 
 +232M    miniconda3_2024-03/​pkgs 
 +396K    miniconda3_2024-03/​sbin 
 +1.5M    miniconda3_2024-03/​share 
 +12K     ​miniconda3_2024-03/​shell 
 +8.0K    miniconda3_2024-03/​ssl 
 +8.0K    miniconda3_2024-03/​x86_64-conda_cos7-linux-gnu 
 +8.0K    miniconda3_2024-03/​x86_64-conda-linux-gnu 
 +</​code>​ 
 + 
 +  ​* Initialize the newly installed conda environment (this will initialize the environment ​**only in the current terminal**): 
 +    ​* **bash shell**: ''​source /path_to_miniconda3/​etc/​profile.d/​conda.sh''​ 
 +      * e.g. (spiritx): ''​source /​homedata/​jypmce/​miniconda3_2024-03/​etc/​profile.d/​conda.sh''​ 
 +    ​* **tcsh shell**: ''​source /path_to_miniconda3/​etc/​profile.d/​conda.csh''​ 
 +      * e.g. (LSCE): ''​source /​home/​share/​unix_files/​cdat/​miniconda3_2024-03/​etc/​profile.d/​conda.csh''​
  
   * Check if you can use the ''​conda''​ command, and use it to initialize the //base// environment   * Check if you can use the ''​conda''​ command, and use it to initialize the //base// environment
-    * <​code>​$ which conda +    * e.g. on **spiritx1**: ​<​code>​$ which conda
-<​installation_path>/​miniconda3/​condabin/​conda+
 $ which python $ which python
 /​usr/​bin/​python /​usr/​bin/​python
-$ conda activate+ 
 +$ source /​homedata/​jypmce/​miniconda3_2024-03/​etc/​profile.d/​conda.sh 
 + 
 +$ which conda 
 +/​homedata/​jypmce/​miniconda3_2024-03/​condabin/​conda 
 +$ which python 
 +/​usr/​bin/​python 
 + 
 +$ conda activate ​base 
 + 
 +(base) $ which conda 
 +/​homedata/​jypmce/​miniconda3_2024-03/​bin/​conda
 (base) $ which python (base) $ which python
-<​installation_path>​/miniconda3/bin/python</​code>​ +/homedata/jypmce/miniconda3_2024-03/bin/python
-  * Update the new installation +
-    * <​code>​$ conda update ​--all +
-[...]</code> +
-    * During the update, the ''​miniconda3''​ directory size goes from 432 Mb to 581 Mb. This directory will keep on growing, which is the reason why you should put it on a (preferably non backed up) disk where you have //enough// space+
  
-  * Make sure we have the latest ''​conda''​ package ​(just in case we did not get it with the //update//)\\ ''​conda update -n base conda''​+(baseconda deactivate
  
-  * Remove the installer later, when you have tester your installation:​\\ ''​rm Miniconda3-latest-Linux-x86_64.sh''​+$ which conda 
 +/​homedata/​jypmce/​miniconda3_2024-03/​condabin/​conda 
 +$ which python 
 +/​usr/​bin/​python 
 +</​code>​
  
-===== Initializing ​conda in new terminals =====+  * Remove the //​Miniconda3//​ installer: ''​$ rm Miniconda3-latest-Linux-x86_64.sh''​ 
 +==== Fine-tuning ​conda to use (only) conda-forge ​====
  
-When you open a terminal, your shell needs to know where to find the ''​conda'' ​command used to initialize an environment,​ or switch between ​environments+  * ''​conda'' ​will probably work fine with the default settings **if you create simple new environments** with just one package and its dependencies.
  
-==== General case ====+  * Luckily, ''​conda''​ also works much better and faster since the end of 2023 (''​conda''​ versions starting at ''​23.10''​),​ now that it is [[https://​conda.org/​blog/​2023-07-05-conda-libmamba-solver-rollout/​|using the mamba solver]] instead of the default solver. 
 +    * If you have followed all the steps of this section in order to **use only //​conda-forge//​ packages**, and there are still some problems with a recent version of ''​conda'',​ try to install and use [[https://​mamba.readthedocs.io/​|mamba]],​ as a drop-in replacement of conda 
 +    * Historical note: before ''​conda''​ used ''​libmamba'',​ we had to use ''​mamba''​ (instead of ''​conda''​) for dealing with our complex Python environments:​ 
 +      * [[https://​wolfv.medium.com/​making-conda-fast-again-4da4debfb3b7|Making conda fast again]] 
 +      * [[https://​github.com/​conda/​conda/​issues/​9367#​issuecomment-793968239|Solving environment:​ failed with initial frozen solve. Retrying with flexible solve.]]
  
-You were asked the following question ​when installing miniconda3: ​//Do you wish the installer to initialize ​Miniconda3 ​by running conda init? [yes|no]//+  * We are power users, and **we intend to create complex Python environments** combining lots of packages available from ''​conda-forge'',​ with complex dependencies,​ and we may run into dependency problems ​when combining packages coming from both conda and conda-forge. 
 +    * => **The solution is to configure conda to always use conda-forge**,​ and completely update ​//​Miniconda3// ​itself with packages coming only from conda-forge
  
-  * If you answered //​yes//, ​the installer probably added some very complicated lines to your shell configuration files, but you probably have ''​conda''​ directly available when you open a new terminal +=== Changing ​the .condarc ​file ===
-  * if you answered //no// (as suggested), use a text editor to add an extra line to the appropriate configuration ​file +
-    * //bash// user: add this line to ''​~/​.bashrc''​\\ ''​source <​installation_path>/​miniconda3/​etc/​profile.d/​conda.sh''​ +
-    * //tcsh// user: add this line to ''​~/​.cshrc''​\\ ''​source <​installation_path>/​miniconda3/​etc/​profile.d/​conda.csh''​+
  
-We choose ​**not** to add a ''​conda ​activate env_name'' ​line to the shell configuration files, in order to avoid side effects. When we open a new terminal, we get the default python available ​on the systemWhen we need a specific python environment,​ we just open a new window and then explicitly type''​conda activate env_name''​+The following will **make sure that we only get packages from //​conda-forge//​**  (same thing as using the ''​-c conda-forge'' ​option) unless ​the requested packages really don't exist on //​conda-forge//​More details in [[https://docs.conda.io/​projects/​conda/​en/​latest/​user-guide/​tasks/​manage-channels.html|Managing channels]]
  
-==== Multi-user installation ====+<​code>​$ cat ~/​.condarc 
 +cat: /​home/​jypmce/​.condarc:​ No such file or directory
  
-In the case of **python environments maintained by a single user, but used by several users**, we could do the same as in the //General case//, but it can be useful to have the users //source// an intermediate initialization file, that will then //source// the initialization file used in the general case. This makes it easier to maintain and change the environments,​ without asking users to make changes.+$ conda config --get channels
  
-  * ask //bash// users to add to ''​~/​.bashrc''​ something like\\ ''​source ​ ~main_installer_login/​.conda3_jyp.sh''​\\ \\ with a ''​.conda3_jyp.sh''​ file looking like {{ :​other:​uvcdat:​cdat_conda:​conda3_jyp.sh.txt |}}+$ conda config --prepend channels conda-forge
  
-  * ask //tcsh// users to add to ''​~/​.cshrc''​ something like\\ ''​source ​ ~main_installer_login/​.conda3_jyp.csh''​\\ \\ with a ''​.conda3_jyp.csh''​ file looking like {{ :​other:​uvcdat:​cdat_conda:​conda3_jyp.csh.txt |}}+$ conda config --set channel_priority strict
  
-===== Fine-tuning ​conda =====+$ conda config ​--get channels 
 +--add channels '​defaults' ​  # lowest priority 
 +--add channels 'conda-forge' ​  # highest priority
  
-conda will probably work fine with the default settings for creating new environments with just one package (e.g. [[https://​github.com/​NOAA-PMEL/​PyFerret/#​anaconda-installation---linux-os-x-and-windows-10bash|just PyFerret]]).+$ cat ~/.condarc 
 +channels: 
 +  ​conda-forge 
 +  ​defaults 
 +channel_priority:​ strict 
 +</​code>​
  
-We intend to **create complex environments** (combining lots of packages, with complex dependencies) and it is highly recommended to use the specific settings described in this section+=== Getting conda configuration information ===
  
-Getting conda configuration information:​ 
   * [[https://​docs.conda.io/​projects/​conda/​en/​latest/​commands/​config.html|conda config]] documentation   * [[https://​docs.conda.io/​projects/​conda/​en/​latest/​commands/​config.html|conda config]] documentation
   * Basic information:​ ''​conda info''​   * Basic information:​ ''​conda info''​
   * Full configuration:​ ''​conda config --show''​   * Full configuration:​ ''​conda config --show''​
  
-==== Changing the .condarc file ====+==== Updating (Mini)conda ​====
  
-Most of the packages we will install will be provided by the //conda-forge// channel (''​-c conda-forge''​ option). The installation steps will also add dependencies that will theoretically come from //conda-forge//, but could also come from the //​defaults// ​channel.+We have to **completely update ​//Miniconda3// at least once after installing //​Miniconda3//​** and [[#fine-tuning_conda_to_use_only_conda-forge|making ​conda-forge the highest priority ​channel]]
  
-The following will **make sure that we only get packages from //conda-forge//** (see [[https://docs.conda.io/​projects/​conda/​en/​latest/​user-guide/​tasks/​manage-channels.html|Managing channels]] ​and [[https://conda-forge.org/docs/​user/​tipsandtricks.html#​how-to-fix-it|Using multiple channels =How to fix it]])+  ​Check the current version of ''​conda''​ 
 +    ​<​code>​$ ​conda list -n base | grep conda 
 +# packages in environment at /path_to_miniconda: 
 +conda                     24.1.2          py312h06a4308_0 
 +conda-libmamba-solver ​    ​23.12.0 ​           pyhd3eb1b0_1 
 +[... and other packages with 'conda' in their name] 
 +</code>
  
-<​code>​$ conda config ​--add channels ​conda-forge +  * Completely update the //​Miniconda3//​ installation 
-$ conda config ​--set channel_priority strict +    * <​code>​$ conda update ​-n base --all 
-cat .condarc +Channels: 
-channels+ - conda-forge 
-  - conda-forge + - defaults 
-  - defaults +Platform: linux-64 
-channel_prioritystrict</​code>​+Collecting package metadata (repodata.json):​ done 
 +Solving environment:​ done 
 + 
 +## Package Plan ## 
 + 
 +  environment location: /​homedata/​jypmce/​miniconda3_2024-03 
 + 
 + 
 +The following packages will be downloaded:​ 
 + 
 +    package ​                   |            build 
 +    ---------------------------|----------------- 
 +    conda-24.3.0 ​              ​| ​ py312h7900ff3_0 ​        1.1 MB  conda-forge 
 +    conda-libmamba-solver-24.1.0| ​    ​pyhd8ed1ab_0 ​         40 KB  conda-forge 
 +    python-3.12.2 ​             |hab00c5b_0_cpython ​       30.8 MB  conda-forge 
 +[...] 
 +The following NEW packages will be INSTALLED:​ 
 +[...] 
 +The following packages will be UPDATED: 
 +[...] 
 +  conda              pkgs/​main::​conda-24.1.2-py312h06a4308~ --> conda-forge::​conda-24.3.0-py312h7900ff3_0 
 +[...] 
 +The following packages will be SUPERSEDED by a higher-priority channel: 
 +[...] 
 +Proceed ([y]/n)? y 
 +[...] 
 +Preparing transaction:​ done 
 +Verifying transaction:​ done 
 +Executing transaction:​ done 
 +</​code>​ 
 +  * Check the updated version of ''​conda''​. Note that **(almost) all** the packages should now specify that they are provided by ''​conda-forge''​ 
 +    * <​code>​$ conda list -n base 
 +conda                     ​24.3.0 ​         py312h7900ff3_0 ​   conda-forge 
 +conda-libmamba-solver ​    ​24.1.0 ​            ​pyhd8ed1ab_0 ​   conda-forge 
 +python ​                   3.12.2 ​         hab00c5b_0_cpython ​   conda-forge 
 +</​code>​ 
 +    * You can use the following to determine if some packages are not provided by ''​conda-forge''​ 
 +      * <​code>​conda list -n base | grep -v conda-forge 
 +# Name                    Version ​                  ​Build ​ Channel 
 +libedit ​                  3.1.20230828 ​        ​h5eee18b_0 
 +libffi ​                   3.4.4                h6a678d5_0 
 +xz                        5.4.5                h5eee18b_0 
 +</​code>​ 
 +      * It should not be a major problem if a few packages are not provided by ''​conda-forge''​.\\ It only means that some packages provided by the ''​anaconda''​ //channel// were more recent than the same packages provided by the ''​conda-forge''​ //channel// when the packages were updated 
 +        * ''​libedit''​ example[[https://​anaconda.org/​conda-forge/​libedit|version 3.1.20191231 @ conda-forge]] and [[https://​anaconda.org/​anaconda/​libedit|version 3.1.20230828 @ anaconda]] 
 + 
 +  ​* You can later update again the full //​Miniconda3//,​ or just the ''​conda''​ command.\\ This should not make any changes to the Python environment(s) have have installed with ''​conda''​ 
 +    * <​code>​$ conda update ​-n base --all 
 +[...] 
 +# All requested packages already installed. 
 + 
 +(base) $ conda update ​-n base conda 
 +[...] 
 +# All requested packages already installed. 
 +</​code>​ 
 + 
 +  ​* Note: during this installation,​ the ''​miniconda3''​ directory size grew from 763 Mb to 1.6 Gb, and we have not installed any custom Python environment yet! This is the reason why you should install //​Miniconda3//​ on a (preferably non backed up) disk where you have //enough// space 
 +    * <​code>​$ du -sh /​homedata/​jypmce/​miniconda3_2024-03 
 +736M    /​homedata/​jypmce/​miniconda3_2024-03 
 +[... update] 
 +$ du -sh /​homedata/​jypmce/​miniconda3_2024-03 
 +1.6G    /​homedata/​jypmce/​miniconda3_2024-03 
 +</​code>​ 
 + 
 +  * You can [[https://​docs.conda.io/​projects/​conda/​en/​latest/​commands/​clean.html|clean]] the initial //​Miniconda3//​ installation to free up some disk space. The documentation specifies //WARNING: This will break environments with packages installed using symlinks back to the package cache.// and **you should probably avoid using ''​clean''​ once you have installed new Python environments** (unless you are desperate for disk space and know how to do a complete re-installation if something breaks...) 
 +    * <​code>​(base) $ conda clean --all 
 +Will remove 154 (180.9 MB) tarball(s). 
 +Proceed ([y]/n)? y 
 + 
 +Will remove 1 index cache(s). 
 +Proceed ([y]/n)? y 
 + 
 +Will remove 74 (378.0 MB) package(s). 
 +Proceed ([y]/n)? y 
 + 
 +There are no tempfile(s) to remove. 
 +There are no logfile(s) to remove. 
 + 
 +(base) $ du -sh /​homedata/​jypmce/​miniconda3_2024-03 
 +457M    /​homedata/​jypmce/​miniconda3_2024-03 
 +</​code>​ 
 +===== Initializing conda in terminals ===== 
 + 
 +When you open a terminal, your shell needs to know where to find the ''​conda''​ command used to initialize an environment,​ or switch between environments. This can be configured in the shell configuration files 
 + 
 +==== General case ==== 
 + 
 +You were asked the following question when installing miniconda3: //Do you wish the installer to initialize Miniconda3 by running conda init? [yes|no]//​ 
 + 
 +  * If you answered //yes//, the installer probably added some very complicated lines to your shell configuration files, but you probably have ''​conda''​ directly available when you open a new terminal 
 +  * if you answered //no// (as suggested), use a text editor to add an extra line to the appropriate configuration file 
 +    * //bash// user: add this line to ''​~/​.bashrc''​\\ ''​source <​installation_path>/​miniconda3/​etc/​profile.d/​conda.sh''​ 
 +    * //tcsh// user: add this line to ''​~/​.cshrc''​\\ ''​source <​installation_path>/​miniconda3/​etc/​profile.d/​conda.csh''​ 
 + 
 +We choose **not** to add a ''​conda activate env_name''​ line to the shell configuration files, in order to avoid side effects. When we open a new terminal, we get the default python available on the system. When we need a specific python environment,​ we just open a new window and then explicitly type: ''​conda activate env_name''​ 
 + 
 +==== Multi-user installation ==== 
 + 
 +In the case of **python environments maintained by a single user, but used by several users**, we could do the same as in the //General case//, but it can be useful to have the users //source// an intermediate initialization file, that will then //source// the initialization file used in the general case. This makes it easier to maintain and change the environments,​ without asking users to make changes. 
 + 
 +  * ask //bash// users to add to ''​~/​.bashrc''​ something like\\ ''​source ​ ~main_installer_login/​.conda3_jyp.sh''​\\ \\ with a ''​.conda3_jyp.sh''​ file looking like {{ :​other:​uvcdat:​cdat_conda:​conda3_jyp.sh.txt |}} 
 + 
 +  * ask //tcsh// users to add to ''​~/​.cshrc''​ something like\\ ''​source ​ ~main_installer_login/​.conda3_jyp.csh''​\\ \\ with a ''​.conda3_jyp.csh''​ file looking like {{ :​other:​uvcdat:​cdat_conda:​conda3_jyp.csh.txt |}}
  
 ==== Using mamba instead of conda ==== ==== Using mamba instead of conda ====
  
-Even when using a proper ''​.condarc''​ in order to get packages coming only from //​conda-forge//,​ ''​conda install''​ may fail to solve the dependencies after meditating during an enormous time, when we specify ​**complex environments**. Using CDAT as a base environment,​ and adding a lot of extra packages is a complex environment.+Even when using a proper ''​.condarc''​ in order to get packages coming only from //​conda-forge//,​ ''​conda install''​ may fail to solve the dependencies after meditating during an enormous time (e.g. [[https://​github.com/​conda/​conda/​issues/​9367#​issuecomment-793968239|Solving environment:​ failed with initial frozen solve. Retrying with flexible solve.]]), when we try to deal with **complex environments** ​(i.e. we already have lots of installed packages and we want to add even more packages). 
 + 
 +Using CDAT as a base environment,​ and adding a lot of extra packages is a **complex environment**.
  
-The only solution seems to replace ''​conda''​ with [[https://​github.com/​mamba-org/​mamba|mamba]],​ a reimplementation of the conda package manager in C++ ([[https://​wolfv.medium.com/​making-conda-fast-again-4da4debfb3b7|Making conda fast again]])+The only solution seems to replace ''​conda''​ with [[https://​github.com/​mamba-org/​mamba|mamba]],​ a reimplementation of the conda package manager in C++ ([[https://​wolfv.medium.com/​making-conda-fast-again-4da4debfb3b7|Making conda fast again]])
 +  * Install ''​mamba''​ in the //base// (i.e. //​miniconda//​) environment\\ ''​conda install mamba -n base -c conda-forge''​ 
 +  * Use ''​mamba''​ instead of ''​conda''​ when installing packages.\\ Theoretically,​ you don't need to specify ''​install -c conda forge''​ any more, because ''​conda-forge''​ is now the default source of new packages\\ ''​mamba install lots_of_packages''​
  
  
other/uvcdat/cdat_conda/miniconda3_install.1615286247.txt.gz · Last modified: 2021/03/09 10:37 by jypeter · Currently locked by: jypeter