| Both sides previous revisionPrevious revisionNext revision | Previous revision |
| other:uvcdat:cdat_conda:miniconda3_install [2025/07/24 15:54] – [Desperately seeking a specific package] jypeter | other:uvcdat:cdat_conda:miniconda3_install [2025/09/17 14:24] (current) – [Useful conda commands] More export details jypeter |
|---|
| ==== Using an unlisted Python distribution on a jupyter server ==== | ==== Using an unlisted Python distribution on a jupyter server ==== |
| |
| If there is a //jupyter notebook server// connected to your system, and a Python environment providing all the packages you need, but that is not directly available on the notebook server, follow the [[other:python:starting#using_a_non-standard_kernel|using a non-standard kernel]] instructions, rather than starting your own local server | In the case when: |
| | * there is a //jupyter notebook server// connected to your Linux cluster (e.g. https://jupyter.lsce.ipsl.fr/) |
| | * and an existing Python environment on this cluster providing all the packages you need |
| | but this Python environment is not directly available on the notebook server (in the listed //kernels//), you should follow the [[other:python:starting#using_a_non-standard_kernel|using a non-standard kernel]] instructions, rather than starting your own local notebook server (on this multi-user computer) |
| ===== Why should you use Miniconda3 ? ===== | ===== Why should you use Miniconda3 ? ===== |
| | |
| | FIXME: try using [[https://conda-forge.org/download/|Miniforge3]] instead of Miniconda3 ? |
| |
| * **//Miniconda3// is a minimal/bootstrap Python distribution** that can be used for creating more complex Python distributions.\\ It will basically make a recent ''conda'' command/executable available on your computer, and you can then use this ''conda'' executable to select an existing distribution, or create new and independent Python distributions. | * **//Miniconda3// is a minimal/bootstrap Python distribution** that can be used for creating more complex Python distributions.\\ It will basically make a recent ''conda'' command/executable available on your computer, and you can then use this ''conda'' executable to select an existing distribution, or create new and independent Python distributions. |
| * [[https://github.com/conda/conda|conda @ GitHub]] | * [[https://github.com/conda/conda|conda @ GitHub]] |
| |
| * Package repositories | * [[https://conda.org/blog/2024-08-14-conda-ecosystem-explained#conda-package-repository-and-channels|Package repositories]] |
| * [[https://conda-forge.org/packages/|conda-forge]] <= recommended source of packages | * [[https://conda-forge.org/packages/|conda-forge]] <= recommended source of packages |
| * [[https://anaconda.org/|default]] | * [[https://anaconda.org/|default]] |
| * **Available environments** | * **Available environments** |
| * ''conda env list'' | * ''conda env list'' |
| | * Same as ''conda info %%--%%envs'' |
| * The currently active environment has a "*****" character on its line<code>(base) $ conda env list | * The currently active environment has a "*****" character on its line<code>(base) $ conda env list |
| # conda environments: | # conda environments: |
| * **Delete an environment** | * **Delete an environment** |
| * ''conda remove -n crap_environment %%--%%all'' | * ''conda remove -n crap_environment %%--%%all'' |
| | * This will remove all packages of ''crap_environment'' and the ''crap_environment'' environment itself |
| |
| * **Find out if a package is available** on the //conda-forge// channel (or other channels) | * **Find out if a package is available** on the //conda-forge// channel (or other channels) |
| * **Specific versions can be specified** by using the [[#the_conda_matchspec_format|conda MatchSpec format]] | * **Specific versions can be specified** by using the [[#the_conda_matchspec_format|conda MatchSpec format]] |
| * ''conda remove package1 package2 packageN'' | * ''conda remove package1 package2 packageN'' |
| | * Same as ''conda //uninstall// package1 package2 packageN'' |
| * Matching ''pip'' commands (**only** if the package is not available in conda!) | * Matching ''pip'' commands (**only** if the package is not available in conda!) |
| * ''pip install package'' | * ''pip install package'' |
| * ''conda update -n existing_environment %%--%%all'' | * ''conda update -n existing_environment %%--%%all'' |
| * [[#the_first_time_we_update_miniconda3|'base' environment update example]] | * [[#the_first_time_we_update_miniconda3|'base' environment update example]] |
| | |
| | * **Export the description of an environment** (as a checkpoint, or to replicate the environment somewhere else) |
| | * There are lots of [[https://docs.conda.io/projects/conda/en/stable/commands/export.html#|export options]] available, but we recommend the following combination of options. |
| | * The options used below will create a text file ([[https://en.wikipedia.org/wiki/YAML|YAML]] format) listing the names of the Python packages that were installed with ''conda install -n existing_environment_name package_name''. This file does not specify the packages' version and their dependencies and can be used later to easily reproduce an environment (using the latest available version of the requested packages) |
| | * ''conda export -n existing_environment_name %%--%%format yml %%--%%from-history -f some_path/existing_environment_yyyy-mm-dd.yml'' |
| | * Windows example:\\ ''conda export -n base %%--%%format yml %%--%%from-history -f C:\Users\your_login\Desktop\base_yyyy-mm-dd.yml'' |
| | * The environment can later be replicated (with the latest version of the requested packages) with:\\ ''conda env create -n existing_environment_clone -f some_path/existing_environment_yyyy-mm-dd.yml'' |
| | * <wrap hi>''pip'' WARNING!</wrap> |
| | * ''conda export'' will only export information about packages installed with ''conda install''. You should also check if you have installed packages with ''pip'' in your environment, and write it down... |
| | * Linux/Mac example: |
| | * <code>(base) > conda list -n test_env | grep pypi |
| | sbck 1.4.0 pypi_0 pypi |
| | xsbck 1.0.0 pypi_0 pypi |
| | (base) > conda list -n test_env | grep pypi > some_path/test_env_pypi_yyyy-mm-dd.txt</code> |
| | * Windows example: |
| | * <code>(base) PS C:\> conda list -n test_env | findstr pypi |
| | brotli 1.1.0 pypi_0 pypi |
| | (base) PS C:\> conda list -n test_env | findstr pypi > C:\Users\your_login\Desktop\test_env_pypi_yyyy-mm-dd.txt</code> |
| | |
| |
| * **Determine the relationship between installed packages** | * **Determine the relationship between installed packages** |
| * ''base'' history file: ''/path_to_miniconda/conda-meta/history'' | * ''base'' history file: ''/path_to_miniconda/conda-meta/history'' |
| * ''existing_environment'' history file: ''/path_to_miniconda/envs/existing_environment/conda-meta/history'' | * ''existing_environment'' history file: ''/path_to_miniconda/envs/existing_environment/conda-meta/history'' |
| * e.g. installation commands for the ''base'' environment:\\ <code> $ egrep -e 'cmd:' -e '==>' /homedata/jypmce/miniconda3_2024-03/conda-meta/history | * e.g. installation commands for the ''base'' environment:\\ <code>$egrep -e 'cmd:' -e '==>' /home/share/unix_files/cdat/miniconda3_2024-03/conda-meta/history |
| ==> 2024-02-23 18:39:23 <== | ==> 2024-02-23 18:39:23 <== |
| # cmd: constructor /tmp/tmp.4um5tGKcqW/miniconda3/ --output-dir /tmp/tmp.4um5tGKcqW | # cmd: constructor /tmp/tmp.4um5tGKcqW/miniconda3/ --output-dir /tmp/tmp.4um5tGKcqW |
| ==> 2024-03-27 16:10:01 <== | ==> 2024-03-27 16:13:08 <== |
| # cmd: /homedata/jypmce/miniconda3_2024-03/bin/conda update -n base --all | # cmd: /home/share/unix_files/cdat/miniconda3_2024-03/bin/conda update -n base --all |
| ==> 2024-03-28 11:38:08 <== | |
| [...] | [ ... ] |
| ==> 2024-04-24 17:47:32 <== | |
| # cmd: /homedata/jypmce/miniconda3_2024-03/bin/conda update -n base --all</code> | ==> 2025-07-24 17:10:51 <== |
| | # cmd: /home/share/unix_files/cdat/miniconda3_2024-03/bin/conda update -n base --all |
| | ==> 2025-07-28 17:05:44 <== |
| | # cmd: /home/share/unix_files/cdat/miniconda3_2024-03/bin/conda update -n base --all</code> |
| | * e.g. installation commands for a more complex //work// environment (once the ''base'' environment is available):\\ <code>$ egrep -e 'cmd:' -e '==>' /home/share/unix_files/cdat/miniconda3_2024-03/envs/bauer_env/conda-meta/history |
| | ==> 2024-06-18 11:16:12 <== |
| | # cmd: /home/share/unix_files/cdat/miniconda3_2024-03/bin/conda create -n bauer_env xcdat |
| | ==> 2024-06-18 11:25:14 <== |
| | # cmd: /home/share/unix_files/cdat/miniconda3_2024-03/bin/conda install cartopy cmocean |
| | |
| | [ ... lots of extra packages and updates ... ] |
| | |
| | # cmd: /home/share/unix_files/cdat/miniconda3_2024-03/bin/conda update -n bauer_env --all numpy<2.3 |
| | ==> 2025-07-29 16:18:44 <== |
| | # cmd: /home/share/unix_files/cdat/miniconda3_2024-03/bin/conda install -n bauer_env rpy2</code> |
| * You can also, use a script and the ''conda'' module available **in the ''base'' environment**: [[https://github.com/conda/conda/issues/4545#issuecomment-469984684|example]] | * You can also, use a script and the ''conda'' module available **in the ''base'' environment**: [[https://github.com/conda/conda/issues/4545#issuecomment-469984684|example]] |
| |
| * The link also shows how to do it with standard Linux tools (find, grep), based on the [[https://docs.conda.io/projects/conda-build/en/stable/resources/package-spec.html#conda-package-specification|Conda package specification]] page | * The link also shows how to do it with standard Linux tools (find, grep), based on the [[https://docs.conda.io/projects/conda-build/en/stable/resources/package-spec.html#conda-package-specification|Conda package specification]] page |
| * [[https://github.com/kelvinou01/conda-which|conda-which]] seems to work (see [[https://github.com/conda/conda/issues/14026#issuecomment-2268821608|test / example]]) ! | * [[https://github.com/kelvinou01/conda-which|conda-which]] seems to work (see [[https://github.com/conda/conda/issues/14026#issuecomment-2268821608|test / example]]) ! |
| | * Note: ''conda-which'' **has to be (obviously) installed in the ''base'' environment** |
| |
| /* standard page footer */ | /* standard page footer */ |
