Differences

This shows you the differences between two versions of the page.

--- other:uvcdat:cdat_conda:miniconda3_install [2024/07/09 13:50] – [Web sites] Added link to "Conda package specification" jypeter
+++ other:uvcdat:cdat_conda:miniconda3_install [2025/07/29 16:36] (current) – [Installation history] Updated the examples jypeter
@@ Line 1: / Line 1: @@
 ====== JYP steps for installing Miniconda3 ======
-===== Before installing Python on a shared computer =====
+===== Before installing Python on a shared Linux server =====
 ==== Desperately seeking a specific package ====
-Check if the Python package you are [[https://www.imdb.com/title/tt0089017/|desperately seeking]] is not already installed, before installing your own Python **on shared interactive Linux servers, or on batch computing clusters** !!
+Check if the Python package you are [[https://www.imdb.com/title/tt0089017/|desperately seeking]] is not already installed, before installing your own Python environment **on shared interactive Linux servers, or on batch computing clusters** !!
   * Ask other users who know the local environment
-  * Read the servers/clusters documentation: [[https://wiki.lsce.ipsl.fr/pmip3/doku.php/other:python:starting|Python at LSCE by JYP]], [[https://intranet.lsce.ipsl.fr/informatique/index.php|LSCE]], [[https://documentations.ipsl.fr/spirit/|ESPRI Computing and Data Center]], ...
+  * Read the servers/clusters Python-related documentation: [[https://wiki.lsce.ipsl.fr/pmip3/doku.php/other:python:starting|Python at LSCE by JYP]], [[https://intranet.lsce.ipsl.fr/informatique/index.php|LSCE]], [[https://documentations.ipsl.fr/spirit/spirit_clusters/python_modules.html|IPSL Mesocenter]], ...
   * Use ''[[other:newppl:starting#using_module_to_access_optional_programs|module avail [mod_name]]]'' to find out the Python versions available, and then ''module load'' and ''conda list'' to get more information on the available Python packages
-    * e.g. on ''spiritx'' ([[https://documentations.ipsl.fr/spirit/common/module_command.html|modules on spirit]])): <code>$ module avail python
+    * e.g. on ''spiritx'' ([[https://documentations.ipsl.fr/spirit/common/module_command.html|modules on spirit]])): <code>$ module avail -t python
-python/meso-3.8  python/meso-3.9  python/meso-3.10  python/meso-3.11
+python/meso-3.8
+python/meso-3.9
+python/meso-3.10
+python/meso-3.11
-$ module avail anaconda
+jypmce@spiritx1:~$ module avail -l 2>&1 | grep meso
-anaconda-meso/2022.10    anaconda3-py/2020.11
+anaconda-meso/2022.10                                       2023/04/21 18:02:35
-anaconda-meso/2023.09-0  anaconda3-py/2021.11
+anaconda-meso/2023.09-0                                     2024/01/25 15:07:50
+pangeo-meso/2023.04.15                                      2024/10/18 15:11:30
+pangeo-meso/2024.01.22                                      2024/10/29 15:16:33
+pangeo-meso/2025.01.24                                      2025/02/28 15:45:21
+python/meso-3.8                                             2024/10/18 15:12:15
+python/meso-3.9                                             2024/10/18 15:12:01
+python/meso-3.10                                            2024/10/18 15:11:30
+python/meso-3.11                                            2024/10/29 15:16:33
-$ module avail pangeo
+jypmce@spiritx1:~$ module load pangeo-meso/2025.01.24
-pangeo-meso/2023.04.15  pangeo-meso/2024.01.22
-$ module load pangeo-meso/2024.01.22
+(pangeo-meso-2025.01.24) jypmce@spiritx1:~$ which python
-(pangeo-meso-2024.01.22) $ which python
+/net/nfs/tools/u20/Python/miniforge_24.11.3-0/envs/pangeo-meso-2025.01.24/bin/python
-/net/nfs/tools/u20/Python/miniconda3_py311_23.11.0-2/envs/pangeo-meso-2024.01.22/bin/python
-$ conda list | grep cartopy
+(pangeo-meso-2025.01.24) jypmce@spiritx1:~$ conda list | grep cartopy
-cartopy                   0.22.0          py311h320fe9a_1    conda-forge
+cartopy                   0.24.0          py312hf9745cd_0    conda-forge
 </code>
   * Lots of Python packages do very similar things. Maybe you can use another similar package that is already installed...
-  * If the packages you need are not available, it may be faster and easier to ask the local system administrators if they can install the packages, rather than doing everything yourself
+  * If the packages you need are not available, it may be faster and easier to ask JYP or the local system administrators if they can install the packages in an existing environment, rather than doing everything yourself
 ==== Using an unlisted Python distribution on a jupyter server ====
-If there is a //jupyter notebook server// connected to your system, and a Python environment providing all the packages you need, but that is not directly available on the notebook server, follow the [[other:python:starting#using_a_non-standard_kernel|using a non-standard kernel]] instructions, rather than starting your own local server
+In the case when:
+  * there is a //jupyter notebook server// connected to your Linux cluster (e.g. https://jupyter.lsce.ipsl.fr/)
+  * and an existing Python environment on this cluster providing all the packages you need
+but this Python environment is not directly available on the notebook server (in the listed //kernels//), you should follow the [[other:python:starting#using_a_non-standard_kernel|using a non-standard kernel]] instructions, rather than starting your own local notebook server (on this multi-user computer)
 ===== Why should you use Miniconda3 ? =====
+FIXME: try using [[https://conda-forge.org/download/|Miniforge3]] instead of Miniconda3 ?
   * **//Miniconda3// is a minimal/bootstrap Python distribution** that can be used for creating more complex Python distributions.\\ It will basically make a recent ''conda'' command/executable available on your computer, and you can then use this ''conda'' executable to select an existing distribution, or create new and independent Python distributions.
@@ Line 124: / Line 137: @@
       * ''Anaconda Prompt''
       * ''Anaconda Powershell Prompt''
-        * **We will now use this //Anaconda PowerShell// shortcut each time we need to use Miniconda3 or Python**
+        * **We will now use this //Anaconda Powershell// shortcut each time we need to use Miniconda3 or Python**
+      * It is **highly recommended to [[other:win10apps#windows_terminal|install Windows Terminal and make it the default terminal application]]**, so that the ''conda'' prompts will open in a modern (and highly configurable) terminal
   * Remove the installer<code>PS C:\Scratch\your_login> del .\miniconda.exe</code>
@@ Line 559: / Line 573: @@
     * FIXME: update/rewrite the [[other:uvcdat:cdat_conda:cdat_2024_03]] page
+===== Creating Windows menu shortcuts to start environments =====
+FIXME => Move this section somewhere else?
+  * Use ''conda install'' and what is described in [[https://github.com/conda/menuinst/issues/246|Easily creating shortcuts with the powershell_shortcut_miniconda and console_shortcut_miniconda packages, and can this be replicated with menuinst ?]]
+  * Can theoretically be done with [[https://conda.github.io/menuinst/|menuinst]]
+    * [[https://github.com/conda/menuinst|menuinst @ GH]]
+  * FIXME Find a way to add Windows Terminal profiles
+    * See ''menuinst''+[[https://github.com/conda/menuinst/issues/196|Add Windows Terminal profile option]]
 ===== conda resources =====
@@ Line 564: / Line 590: @@
 ==== Web sites ====
-  * [[https://docs.conda.io/projects/conda/en/stable/|Official conda website]]
+  * [[https://docs.conda.io/projects/conda/en/stable/|Official Documentation site]]
     * [[https://docs.conda.io/projects/conda/en/stable/user-guide/cheatsheet.html|Cheat sheet]] (2 pages, for people in a hurry)
     * [[https://docs.conda.io/projects/conda/en/stable/commands/|Commands' reference]]
-    * [[https://conda.org/blog/|conda Blog]]: ''conda'' new features, etc...
     * [[https://docs.conda.io/projects/conda-build/en/stable/resources/package-spec.html#conda-package-specification|Conda package specification]]
       * This helps understand (and [[https://github.com/conda/conda/issues/14026|use]]) the content of the ''/path_to_miniconda/pkgs'' directory
-  * Package repositories
+  * [[https://conda.org/blog/|conda community site]]:
+    * [[https://conda.org/blog/|conda Blog]]: ''conda'' new features, etc...
+  * [[https://github.com/conda/conda|conda @ GitHub]]
+  * [[https://conda.org/blog/2024-08-14-conda-ecosystem-explained#conda-package-repository-and-channels|Package repositories]]
     * [[https://conda-forge.org/packages/|conda-forge]] <= recommended source of packages
     * [[https://anaconda.org/|default]]
@@ Line 591: / Line 622: @@
   * **Available environments**
     * ''conda env list''
+      * Same as ''conda info --envs''
     * The currently active environment has a "*****" character on its line<code>(base) $ conda env list
 # conda environments:
@@ Line 648: / Line 680: @@
   * **Delete an environment**
     * ''conda remove -n crap_environment %%--%%all''
+      * This will remove all packages of ''crap_environment'' and the ''crap_environment'' environment itself
   * **Find out if a package is available** on the //conda-forge// channel (or other channels)
@@ Line 664: / Line 697: @@
     * **Specific versions can be specified** by using the [[#the_conda_matchspec_format|conda MatchSpec format]]
     * ''conda remove package1 package2 packageN''
+      * Same as ''conda //uninstall// package1 package2 packageN''
     * Matching ''pip'' commands (**only** if the package is not available in conda!)
       * ''pip install package''
@@ Line 716: / Line 750: @@
 </code>
-  * Checking the health of an environment
+  * **Checking the //health// of an environment** with ''[[https://docs.conda.io/projects/conda/en/stable/commands/doctor.html|conda doctor]]''
     * Some details in [[https://conda.org/blog/2023-06-01-conda-doctor/|Detecting conda environment inconsistencies easily]]
+      * ''conda doctor'' does not do much yet, and not necessarily in a useful way, but this will hopefully improve over time!
+        * [[https://github.com/conda/conda/issues/14017|'conda doctor' output should be improved]]
+        * [[https://github.com/conda/conda/issues/13574#issuecomment-2214291304|What exactly are altered files?]]
     * ''conda doctor -n base''
     * Example:<code> >conda doctor -n base
@@ Line 753: / Line 790: @@
     * ''base'' history file: ''/path_to_miniconda/conda-meta/history''
     * ''existing_environment'' history file: ''/path_to_miniconda/envs/existing_environment/conda-meta/history''
-    * e.g. installation commands for the ''base'' environment:\\ <code> $ egrep -e 'cmd:' -e '==>' /homedata/jypmce/miniconda3_2024-03/conda-meta/history
+    * e.g. installation commands for the ''base'' environment:\\ <code>$egrep -e 'cmd:' -e '==>' /home/share/unix_files/cdat/miniconda3_2024-03/conda-meta/history
 ==> 2024-02-23 18:39:23 <==
 # cmd: constructor /tmp/tmp.4um5tGKcqW/miniconda3/ --output-dir /tmp/tmp.4um5tGKcqW
-==> 2024-03-27 16:10:01 <==
+==> 2024-03-27 16:13:08 <==
-# cmd: /homedata/jypmce/miniconda3_2024-03/bin/conda update -n base --all
+# cmd: /home/share/unix_files/cdat/miniconda3_2024-03/bin/conda update -n base --all
-==> 2024-03-28 11:38:08 <==
-[...]
+[ ... ]
-==> 2024-04-24 17:47:32 <==
-# cmd: /homedata/jypmce/miniconda3_2024-03/bin/conda update -n base --all</code>
+==> 2025-07-24 17:10:51 <==
+# cmd: /home/share/unix_files/cdat/miniconda3_2024-03/bin/conda update -n base --all
+==> 2025-07-28 17:05:44 <==
+# cmd: /home/share/unix_files/cdat/miniconda3_2024-03/bin/conda update -n base --all</code>
+    * e.g. installation commands for a more complex //work// environment (once the ''base'' environment is available):\\ <code>$ egrep -e 'cmd:' -e '==>' /home/share/unix_files/cdat/miniconda3_2024-03/envs/bauer_env/conda-meta/history
+==> 2024-06-18 11:16:12 <==
+# cmd: /home/share/unix_files/cdat/miniconda3_2024-03/bin/conda create -n bauer_env xcdat
+==> 2024-06-18 11:25:14 <==
+# cmd: /home/share/unix_files/cdat/miniconda3_2024-03/bin/conda install cartopy cmocean
+[ ... lots of extra packages and updates ... ]
+# cmd: /home/share/unix_files/cdat/miniconda3_2024-03/bin/conda update -n bauer_env --all numpy<2.3
+==> 2025-07-29 16:18:44 <==
+# cmd: /home/share/unix_files/cdat/miniconda3_2024-03/bin/conda install -n bauer_env rpy2</code>
     * You can also, use a script and the ''conda'' module available **in the ''base'' environment**: [[https://github.com/conda/conda/issues/4545#issuecomment-469984684|example]]
@@ Line 826: / Line 876: @@
   * [[https://github.com/conda/conda/issues/14026|Add a conda command to find the package owning a file]]
     * The link also shows how to do it with standard Linux tools (find, grep), based on the [[https://docs.conda.io/projects/conda-build/en/stable/resources/package-spec.html#conda-package-specification|Conda package specification]] page
+    * [[https://github.com/kelvinou01/conda-which|conda-which]] seems to work (see [[https://github.com/conda/conda/issues/14026#issuecomment-2268821608|test / example]]) !
 /* standard page footer */