other:uvcdat:cdat_conda:miniconda3_install
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision | ||
|
other:uvcdat:cdat_conda:miniconda3_install [2024/04/19 16:00] jypeter [Initializing conda in terminals] |
other:uvcdat:cdat_conda:miniconda3_install [2024/05/30 15:43] (current) jypeter [Why should you use Miniconda3 ?] |
||
|---|---|---|---|
| Line 1: | Line 1: | ||
| ====== JYP steps for installing Miniconda3 ====== | ====== JYP steps for installing Miniconda3 ====== | ||
| + | ===== Before installing Python on a shared computer ===== | ||
| + | |||
| + | ==== Desperately seeking a specific package ==== | ||
| + | |||
| + | Check if the Python package you are [[https://www.imdb.com/title/tt0089017/|desperately seeking]] is not already installed, before installing your own Python on shared interactive Linux servers, or on batch computing clusters !! | ||
| + | |||
| + | * Ask other users who know the local environment | ||
| + | |||
| + | * Read the servers/clusters documentation: [[https://wiki.lsce.ipsl.fr/pmip3/doku.php/other:python:starting|Python at LSCE by JYP]], [[https://intranet.lsce.ipsl.fr/informatique/index.php|LSCE]], [[https://espri.ipsl.fr/|ESPRI Computing and Data Center]], ... | ||
| + | |||
| + | * Use ''[[other:newppl:starting#using_module_to_access_optional_programs|module avail [mod_name]]]'' to find out the Python versions available, and then ''module load'' and ''conda list'' to get more information | ||
| + | * e.g. on ''spiritx'' ([[https://documentations.ipsl.fr/spirit/common/module_command.html|modules on spirit]])): <code>$ module avail python | ||
| + | python/meso-3.8 python/meso-3.9 python/meso-3.10 python/meso-3.11 | ||
| + | $ module avail anaconda | ||
| + | anaconda-meso/2022.10 anaconda3-py/2020.11 | ||
| + | anaconda-meso/2023.09-0 anaconda3-py/2021.11 | ||
| + | </code> | ||
| + | |||
| + | * Lots of Python packages do very similar things. Maybe you can use another installed package... | ||
| + | |||
| + | * If the packages you need are not available, it may be faster and easier to ask the local system administrators if they can install the packages, rather than doing everything yourself | ||
| + | |||
| + | ==== Using an unlisted Python distribution on a jupyter server ==== | ||
| + | |||
| + | If there is a //jupyter notebook server// connected to your system, and a Python environment providing all the packages you need, but that is not directly available on the notebook server, follow the [[other:python:starting#using_a_non-standard_kernel|using a non-standard kernel]] instructions, rather than starting your own local server | ||
| ===== Why should you use Miniconda3 ? ===== | ===== Why should you use Miniconda3 ? ===== | ||
| - | * **//Miniconda3// is a minimal/bootstrap Python distribution** that can be used for creating more complex Python distributions.\\ It will basically make a recent ''conda'' command available on your computer, and you can then use this ''conda'' executable to select an existing distribution, or create new and independent Python distributions. | + | * **//Miniconda3// is a minimal/bootstrap Python distribution** that can be used for creating more complex Python distributions.\\ It will basically make a recent ''conda'' command/executable available on your computer, and you can then use this ''conda'' executable to select an existing distribution, or create new and independent Python distributions. |
| - | * A **conda //environment//** is basically (the short name of the directory) where you install the Python that you will use (independently of //Miniconda3// itself). You have to ''activate'' this environment in order to use it\\ A **python //distribution//** is the collection of packages (and their dependencies) you have chosen to install together in a given environment. | + | * A **conda //environment//** is basically (the short name of the directory) where you install the Python that you will use (independently of //Miniconda3// itself). You have to ''activate'' this environment in order to use it\\ A **Python //distribution//** is the collection of packages (and their dependencies) you have chosen to install together in a given environment. |
| - | * Example: initializing an environment named ''cdatm_py3'' (assuming it exists)<code>$ source /path_to_miniconda/etc/profile.d/conda.sh | + | * Example: initializing an environment named ''cdatm_py3'' (assuming it exists) in a //bash// shell <code>$ source /path_to_miniconda/etc/profile.d/conda.sh |
| $ which conda | $ which conda | ||
| Line 32: | Line 57: | ||
| * Choose carefully where you will install //Miniconda3//, because the size of the installation directory will start at a few Gb and will keep on growing | * Choose carefully where you will install //Miniconda3//, because the size of the installation directory will start at a few Gb and will keep on growing | ||
| - | * You //could// also start with the full //Anaconda// (instead of //Miniconda//) installer that will install a much more complete python environment, ready for use.\\ **We choose not to use the full //Anaconda// installer** because: | + | * You //could// also start with the full //Anaconda// (instead of //Miniconda//) installer, that will install a much more complete python environment, ready for use.\\ **We choose not to use the full //Anaconda// installer** because: |
| * it requires more disk space than //Miniconda3// during the initial installation, | * it requires more disk space than //Miniconda3// during the initial installation, | ||
| * all the //Anaconda// installer packages come from the ''default'' //channel// (or //repository//) provided by the [[https://anaconda.org/|conda repository]].\\ This is not very useful for us (and could even cause complex dependency problems later) because we will be mostly using (the same) packages provided by the [[https://conda-forge.org/feedstock-outputs/|conda-forge]] //channel//, **in order to avoid complex package dependency problems** | * all the //Anaconda// installer packages come from the ''default'' //channel// (or //repository//) provided by the [[https://anaconda.org/|conda repository]].\\ This is not very useful for us (and could even cause complex dependency problems later) because we will be mostly using (the same) packages provided by the [[https://conda-forge.org/feedstock-outputs/|conda-forge]] //channel//, **in order to avoid complex package dependency problems** | ||
| Line 41: | Line 66: | ||
| ==== Miniconda3 on a Windows computer ==== | ==== Miniconda3 on a Windows computer ==== | ||
| - | Some of the steps below are adapted from the [[https://docs.anaconda.com/free/miniconda/#quick-command-line-install|Windows section of "Quick command line install"]] to install a //pure Windows// version of miniconda3 | + | Some of the steps below are adapted from the [[https://docs.anaconda.com/free/miniconda/#quick-command-line-install|Windows section of "Quick command line install"]] to install a //pure Windows// version of Miniconda3 |
| === Downloading the Miniconda3 installer on Windows === | === Downloading the Miniconda3 installer on Windows === | ||
| Line 54: | Line 79: | ||
| * Open a //terminal//, where you can type the required installation commands | * Open a //terminal//, where you can type the required installation commands | ||
| * Windows //Powershell terminal//: ''Start'' => ''W'' => ''Windows PowerShell'' => ''Windows PowerShell'' | * Windows //Powershell terminal//: ''Start'' => ''W'' => ''Windows PowerShell'' => ''Windows PowerShell'' | ||
| - | * [[other:win10apps#windows_terminal|Windows Terminal]] | + | * [[other:win10apps#windows_terminal|PowerShell in a Windows Terminal]] |
| * Use ''curl'' to download the latest installer (78 Mb as of April 2024): <code>PS C: cd C:\Scratch\your_login | * Use ''curl'' to download the latest installer (78 Mb as of April 2024): <code>PS C: cd C:\Scratch\your_login | ||
| Line 64: | Line 89: | ||
| === BASIC Miniconda3 installation on Windows === | === BASIC Miniconda3 installation on Windows === | ||
| - | * **Determine the folder where you will install Miniconda3 and all your future Python environments** based on this version of Miniconda3\\ **We will later refer to this location as ''/path_to_miniconda3/''** in the Linux sections | + | * **Determine the folder where you will install Miniconda3 and ALL your future Python environments** based on this version of Miniconda3\\ **We will later refer to this location as ''/path_to_miniconda3/''** in the Linux sections |
| * Remember that **you will need a few Gb of available space** | * Remember that **you will need a few Gb of available space** | ||
| * You should **avoid installing Miniconda3 in your Windows //home// folder** (''C:\Users\your_login'' or [[other:win10config#local_home_folder|<some_disk_different_from_c>:\Users\<your_login>]]), if you want to be able to easily make backups or your //home//, without backing up thousands of files related to your Python environments... | * You should **avoid installing Miniconda3 in your Windows //home// folder** (''C:\Users\your_login'' or [[other:win10config#local_home_folder|<some_disk_different_from_c>:\Users\<your_login>]]), if you want to be able to easily make backups or your //home//, without backing up thousands of files related to your Python environments... | ||
| * We add some date or version information at the end of the folder name, in order to differentiate it from other (older of future) installations | * We add some date or version information at the end of the folder name, in order to differentiate it from other (older of future) installations | ||
| - | * Example installation folder: ''C:\Utils\miniconda3_2024-03'' | + | * Example installation folder: ''C:\Utils\miniconda3_2024-03'' (if we start installing in March 2024) |
| * Launch the Miniconda3 installer with<code>PS C:\Scratch\your_login> .\miniconda.exe</code> | * Launch the Miniconda3 installer with<code>PS C:\Scratch\your_login> .\miniconda.exe</code> | ||
| Line 91: | Line 116: | ||
| * Remove the installer<code>PS C:\Scratch\your_login> del .\miniconda.exe</code> | * Remove the installer<code>PS C:\Scratch\your_login> del .\miniconda.exe</code> | ||
| - | * Open an //Anaconda PowerShell Prompt//, and type a few commands to make sure that ''conda'' is working<code>base) PS C:\Users\your_login> conda env list | + | * Open an //Anaconda PowerShell Prompt//, and type a few commands to make sure that ''conda'' is working<code>(base) PS C:\Users\your_login> conda env list |
| base * C:\Utils\miniconda3_2024-03 | base * C:\Utils\miniconda3_2024-03 | ||
| Line 131: | Line 156: | ||
| === Downloading the Miniconda3 installer === | === Downloading the Miniconda3 installer === | ||
| + | |||
| + | * Important: <wrap hi>do not install your own Python environment(s) on shared computers before you have made sure that the packages you need are not already installed, or cannot be easily installed in an existing environment (by the person who maintains this environment)!</wrap> | ||
| + | * Check the output of ''module avail'', read your system documentation, ask other users... | ||
| * We use a download link that always point to the [[https://docs.anaconda.com/free/miniconda/#latest-miniconda-installer-links|latest version of the Linux installer]] | * We use a download link that always point to the [[https://docs.anaconda.com/free/miniconda/#latest-miniconda-installer-links|latest version of the Linux installer]] | ||
| Line 136: | Line 164: | ||
| * Note: Mac users should use the latest [[https://docs.anaconda.com/free/miniconda/#latest-miniconda-installer-links|macOS installer]] appropriate for their computer | * Note: Mac users should use the latest [[https://docs.anaconda.com/free/miniconda/#latest-miniconda-installer-links|macOS installer]] appropriate for their computer | ||
| * The installer is based on **Python 3.12.2 and Conda 24.3.0 as of April 2024** | * The installer is based on **Python 3.12.2 and Conda 24.3.0 as of April 2024** | ||
| + | |||
| * Find some temporary space on Linux (or Windows, if you are using [[other:win10wsl|WSL]]) | * Find some temporary space on Linux (or Windows, if you are using [[other:win10wsl|WSL]]) | ||
| - | * e.g. Linux at **LSCE**:\\ ''$ cd /home/scratch01/$USER'' | + | * e.g. Linux on the **LSCE** servers:\\ ''$ cd /home/scratch01/$USER'' |
| - | * e.g. Linux at **spiritx**:<code>$ mkdir /homedata/$USER/Scratch | + | * e.g. Linux on the IPSL **spiritx** cluster:<code>$ mkdir /homedata/$USER/Scratch |
| $ cd /homedata/$USER/Scratch</code> | $ cd /homedata/$USER/Scratch</code> | ||
| - | * e.g. Linux on a **personal PC/Mac desktop/laptop**: use a scratch/temporary directory with //enough// space | + | * e.g. Linux on a **personal PC/Mac desktop/laptop**: use a local scratch/temporary directory with //enough// space |
| * Use ''wget'' to download the latest installer (138 Mb as of 22 Mar 2024):\\ <code>$ wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh | * Use ''wget'' to download the latest installer (138 Mb as of 22 Mar 2024):\\ <code>$ wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh | ||
| Line 278: | Line 307: | ||
| * Check the current version of ''conda'' | * Check the current version of ''conda'' | ||
| - | * <code>$ conda list -n base | grep conda | + | * <code>$ conda list -n base | grep '^conda' |
| # packages in environment at /path_to_miniconda: | # packages in environment at /path_to_miniconda: | ||
| conda 24.1.2 py312h06a4308_0 | conda 24.1.2 py312h06a4308_0 | ||
| Line 503: | Line 532: | ||
| ==== Installation history ==== | ==== Installation history ==== | ||
| + | You can get the current list of installed packages with ''conda list -n existing_environment'', but it may also be useful to know in which order the packages were installed: | ||
| + | * ''conda list -n existing_environment %%--%%revisions'': display **exactly** what was installed and when (including the updates, and automatic dependencies installation) | ||
| + | * You can theoretically roll back to a previous ''RR'' //revision// number with: ''conda install -n existing_environment %%--%%revision RR'' | ||
| + | * There is more information in the ''/path_to_miniconda/conda-meta/history'' installation log files. | ||
| + | * ''base'' history file: ''/path_to_miniconda/conda-meta/history'' | ||
| + | * ''existing_environment'' history file: ''/path_to_miniconda/envs/existing_environment/conda-meta/history'' | ||
| + | * e.g. installation commands for the ''base'' environment:\\ <code> $ egrep -e 'cmd:' -e '==>' /homedata/jypmce/miniconda3_2024-03/conda-meta/history | ||
| + | |||
| + | ==> 2024-02-23 18:39:23 <== | ||
| + | # cmd: constructor /tmp/tmp.4um5tGKcqW/miniconda3/ --output-dir /tmp/tmp.4um5tGKcqW | ||
| + | ==> 2024-03-27 16:10:01 <== | ||
| + | # cmd: /homedata/jypmce/miniconda3_2024-03/bin/conda update -n base --all | ||
| + | ==> 2024-03-28 11:38:08 <== | ||
| + | [...] | ||
| + | ==> 2024-04-24 17:47:32 <== | ||
| + | # cmd: /homedata/jypmce/miniconda3_2024-03/bin/conda update -n base --all</code> | ||
| + | * You can also, use a script and the ''conda'' module available **in the ''base'' environment**: [[https://github.com/conda/conda/issues/4545#issuecomment-469984684|example]] | ||
| ===== Uninstalling Miniconda3 ===== | ===== Uninstalling Miniconda3 ===== | ||
| Line 513: | Line 559: | ||
| * ''Start'' => ''Settings'' => ''Apps'' => ''Apps & features'' | * ''Start'' => ''Settings'' => ''Apps'' => ''Apps & features'' | ||
| * Select ''Miniconda3 py3xxxxx'' | * Select ''Miniconda3 py3xxxxx'' | ||
| - | * Click on ''Uninstall'' and follow the instructions displayed by the uninstaller window\\ This will remove: | + | * Click on ''Uninstall'' and follow the instructions displayed by the uninstaller window |
| - | * The (very big) directory that you specified when you installed //Miniconda3// | + | * This will remove: |
| - | * The ''conda'' shortcuts from the ''Start Menu'' | + | * The (very big) directory that you specified when you installed //Miniconda3// |
| + | * The ''conda'' shortcuts from the ''Start Menu'' | ||
| + | * This will **not** remove your local configuration file ''C:\Users\your_login\.condarc'' | ||
| ==== Linux-like computer ==== | ==== Linux-like computer ==== | ||
| + | |||
| + | FIXME | ||
other/uvcdat/cdat_conda/miniconda3_install.1713542416.txt.gz · Last modified: 2024/04/19 16:00 by jypeter
