This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
other:uvcdat:cdat_conda:miniconda3_install [2024/04/16 13:45] jypeter [Useful commands] Copied an old page, and started updating it |
other:uvcdat:cdat_conda:miniconda3_install [2024/04/19 16:00] jypeter [Initializing conda in terminals] |
||
---|---|---|---|
Line 21: | Line 21: | ||
/path_to_miniconda/envs/cdatm_py3/bin/python | /path_to_miniconda/envs/cdatm_py3/bin/python | ||
</code> | </code> | ||
- | * The //special// environment where //Miniconda3// itself is installed is called ''base''. Note that the ''python'' binary of the //special// ''base'' environment is not located in the same directory hierarchy as the ''python'' of the other environments (i.e. there is no ''envs/'' subdirectory<code>$ conda activate base | + | * The //special// environment where //Miniconda3// itself is installed is called ''base'' (it also used to be called ''root''). Note that the ''python'' binary of the //special// ''base'' environment is not located in the same directory hierarchy as the ''python'' of the other environments (i.e. there is no ''envs/'' subdirectory<code>$ conda activate base |
(base) $ which python | (base) $ which python | ||
Line 34: | Line 34: | ||
* You //could// also start with the full //Anaconda// (instead of //Miniconda//) installer that will install a much more complete python environment, ready for use.\\ **We choose not to use the full //Anaconda// installer** because: | * You //could// also start with the full //Anaconda// (instead of //Miniconda//) installer that will install a much more complete python environment, ready for use.\\ **We choose not to use the full //Anaconda// installer** because: | ||
* it requires more disk space than //Miniconda3// during the initial installation, | * it requires more disk space than //Miniconda3// during the initial installation, | ||
- | * all the //Anaconda// installer packages come from the ''default'' //channel// (or //repository//) provided by the [[https://anaconda.org/|conda repository]].\\ This is not very useful for us (and could even cause complex dependency problems later) because we will be mostly using (the same) packages provided by the [[https://conda-forge.org/feedstock-outputs/|conda-forge]] //channel//, **in order to avoid complex package dependency problems**. | + | * all the //Anaconda// installer packages come from the ''default'' //channel// (or //repository//) provided by the [[https://anaconda.org/|conda repository]].\\ This is not very useful for us (and could even cause complex dependency problems later) because we will be mostly using (the same) packages provided by the [[https://conda-forge.org/feedstock-outputs/|conda-forge]] //channel//, **in order to avoid complex package dependency problems** |
- | ===== Installing miniconda3 on a Linux-like computer ===== | ||
- | <WRAP center round tip 60%> | + | ===== Downloading Miniconda3, and BASIC installation ===== |
- | By **//Linux-like//**, we mean: | + | |
- | * A native Linux computer (or server) | + | |
- | * A windows 10 computer with [[other:win10wsl|WSL+Ubuntu installed]] | + | |
- | * A Mac where you can use Linux in a terminal | + | |
- | </WRAP> | + | |
- | <WRAP center round info 60%> | + | ==== Miniconda3 on a Windows computer ==== |
- | **//Pure Windows//** Note: you can also follow the steps listed in the [[https://docs.anaconda.com/free/miniconda/#quick-command-line-install|Windows section of "Quick command line install"]] to install a //pure Windows// version of miniconda3, and then follow the steps listed in the next sections of this page.\\ We assume below that the installer is downloaded in the ''C:\Scratch\your_login'' directory, and we install Miniconda3 in a ''C:\Utils\miniconda3_2024-03'' directory. <code>PS C: cd C:\Scratch\your_login | + | |
+ | Some of the steps below are adapted from the [[https://docs.anaconda.com/free/miniconda/#quick-command-line-install|Windows section of "Quick command line install"]] to install a //pure Windows// version of miniconda3 | ||
+ | |||
+ | === Downloading the Miniconda3 installer on Windows === | ||
+ | |||
+ | * We use a download link that always point to the [[https://docs.anaconda.com/free/miniconda/#latest-miniconda-installer-links|latest version of the Windows installer]] | ||
+ | * https://repo.anaconda.com/miniconda/Miniconda3-latest-Windows-x86_64.exe | ||
+ | * The installer is based on **Python 3.12.2 and Conda 24.3.0 as of April 2024** | ||
+ | |||
+ | * Find some temporary space on Windows | ||
+ | * e.g. ''C:\Scratch\your_login'' or whatever you use as a [[other:win10config#local_scratch_folder|local Scratch folder]] | ||
+ | |||
+ | * Open a //terminal//, where you can type the required installation commands | ||
+ | * Windows //Powershell terminal//: ''Start'' => ''W'' => ''Windows PowerShell'' => ''Windows PowerShell'' | ||
+ | * [[other:win10apps#windows_terminal|Windows Terminal]] | ||
+ | |||
+ | * Use ''curl'' to download the latest installer (78 Mb as of April 2024): <code>PS C: cd C:\Scratch\your_login | ||
PS C:\Scratch\your_login> curl https://repo.anaconda.com/miniconda/Miniconda3-latest-Windows-x86_64.exe -o miniconda.exe | PS C:\Scratch\your_login> curl https://repo.anaconda.com/miniconda/Miniconda3-latest-Windows-x86_64.exe -o miniconda.exe | ||
- | PS C:\Scratch\your_login> .\miniconda.exe | + | PS C:\Scratch\your_login> dir miniconda.exe |
- | PS C:\Scratch\your_login> del .\miniconda.exe | + | [...] |
- | </code> | + | -a---- 19/04/2024 11:18 81274240 miniconda.exe</code> |
+ | |||
+ | === BASIC Miniconda3 installation on Windows === | ||
+ | |||
+ | * **Determine the folder where you will install Miniconda3 and all your future Python environments** based on this version of Miniconda3\\ **We will later refer to this location as ''/path_to_miniconda3/''** in the Linux sections | ||
+ | * Remember that **you will need a few Gb of available space** | ||
+ | * You should **avoid installing Miniconda3 in your Windows //home// folder** (''C:\Users\your_login'' or [[other:win10config#local_home_folder|<some_disk_different_from_c>:\Users\<your_login>]]), if you want to be able to easily make backups or your //home//, without backing up thousands of files related to your Python environments... | ||
+ | * We add some date or version information at the end of the folder name, in order to differentiate it from other (older of future) installations | ||
+ | * Example installation folder: ''C:\Utils\miniconda3_2024-03'' | ||
+ | |||
+ | * Launch the Miniconda3 installer with<code>PS C:\Scratch\your_login> .\miniconda.exe</code> | ||
+ | |||
+ | * In the installer window: | ||
+ | * (Review and) accept the License Agreement | ||
+ | * Choose to install for //Just Me// (the recommended default choice) | ||
+ | * Specify the //Destination Folder// selected above, e.g. ''C:\Utils\miniconda3_2024-03'' | ||
+ | * Use the pre-selected //Advanced Installation Options//, i.e | ||
+ | * //Create start menu shortcutes// | ||
+ | * //Register Miniconda3 as my default Python 3.12// <= Useful? | ||
+ | |||
+ | * When the installer has finished running, you should have: | ||
+ | * A new ''C:\Utils\miniconda3_2024-03'' folder | ||
+ | * This folder size will be ~650 Mb and will keep on growing (up to several Gb) as you add more Python environments and packages | ||
+ | * => **Be sure to have enough space on this disk!** | ||
+ | * Two new shortcuts in the Windows ''Start'' menu | ||
+ | * ''Anaconda Prompt'' | ||
+ | * ''Anaconda Powershell Prompt'' | ||
+ | * **We will now use this //Anaconda PowerShell// shortcut each time we need to use Miniconda3 or Python** | ||
+ | |||
+ | * Remove the installer<code>PS C:\Scratch\your_login> del .\miniconda.exe</code> | ||
+ | |||
+ | * Open an //Anaconda PowerShell Prompt//, and type a few commands to make sure that ''conda'' is working<code>base) PS C:\Users\your_login> conda env list | ||
+ | base * C:\Utils\miniconda3_2024-03 | ||
+ | |||
+ | (base) PS C:\Users\your_login> conda list | ||
+ | # packages in environment at C:\Utils\miniconda3_2024-03: | ||
+ | # Name Version Build Channel | ||
+ | [...] | ||
+ | conda 24.3.0 py312haa95532_0 | ||
+ | [...] | ||
+ | python 3.12.2 h1d929f7_0 | ||
+ | [...] | ||
+ | |||
+ | (base) PS C:\Users\your_login> conda deactivate | ||
+ | |||
+ | PS C:\Users\your_login> conda env list | ||
+ | base C:\Utils\miniconda3_2024-03</code> | ||
+ | |||
+ | * Next: be sure to <wrap hi>follow the steps in the [[other:uvcdat:cdat_conda:miniconda3_install#fine-tuning_conda_to_use_only_conda-forge|Fine-tuning conda to use (only) conda-forge]], and all the other Miniconda3 instructions</wrap> | ||
+ | * ''conda'' commands will work on all computers (Windows and Linux-like) | ||
+ | * The ''.condarc'' is located in your Windows //home// folder\\ => ''C:\Users\your_login\.condarc'' | ||
+ | |||
+ | ==== Miniconda3 on a Linux-like computer ==== | ||
+ | |||
+ | |||
+ | <WRAP center round tip 60%> | ||
+ | By **//Linux-like//**, we mean: | ||
+ | * A **Linux** desktop/laptop | ||
+ | * The installation will be used by **one** person | ||
+ | * A **Linux** server | ||
+ | * The installation will be used by **one or more** persons | ||
+ | * A **windows computer with [[other:win10wsl|WSL+Ubuntu installed]]** | ||
+ | * You should use the [[#miniconda3_on_a_windows_computer|Windows installer]], if all the packages you need are available for Windows | ||
+ | * The installation will be used by **one** person | ||
+ | * A **Mac** where you can use Linux in a terminal | ||
+ | * The installation will be used by **one** person | ||
</WRAP> | </WRAP> | ||
- | ==== Downloading the Miniconda3 installer ==== | + | === Downloading the Miniconda3 installer === |
* We use a download link that always point to the [[https://docs.anaconda.com/free/miniconda/#latest-miniconda-installer-links|latest version of the Linux installer]] | * We use a download link that always point to the [[https://docs.anaconda.com/free/miniconda/#latest-miniconda-installer-links|latest version of the Linux installer]] | ||
* For the Linux computers we commonly use, we need the //Miniconda3 Linux 64-bit// link: https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh | * For the Linux computers we commonly use, we need the //Miniconda3 Linux 64-bit// link: https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh | ||
* Note: Mac users should use the latest [[https://docs.anaconda.com/free/miniconda/#latest-miniconda-installer-links|macOS installer]] appropriate for their computer | * Note: Mac users should use the latest [[https://docs.anaconda.com/free/miniconda/#latest-miniconda-installer-links|macOS installer]] appropriate for their computer | ||
- | * The installer is based on **Python 3.12 as of March 2024** | + | * The installer is based on **Python 3.12.2 and Conda 24.3.0 as of April 2024** |
* Find some temporary space on Linux (or Windows, if you are using [[other:win10wsl|WSL]]) | * Find some temporary space on Linux (or Windows, if you are using [[other:win10wsl|WSL]]) | ||
Line 72: | Line 147: | ||
[...] 138M Feb 27 20:40 Miniconda3-latest-Linux-x86_64.sh</code> | [...] 138M Feb 27 20:40 Miniconda3-latest-Linux-x86_64.sh</code> | ||
- | ==== Using the Miniconda3 installer ==== | + | === Using the Miniconda3 installer === |
We assume below that we are in the [[#downloading_the_installer|directory where we have downloaded the installer]] | We assume below that we are in the [[#downloading_the_installer|directory where we have downloaded the installer]] | ||
Line 154: | Line 229: | ||
* Remove the //Miniconda3// installer: ''$ rm Miniconda3-latest-Linux-x86_64.sh'' | * Remove the //Miniconda3// installer: ''$ rm Miniconda3-latest-Linux-x86_64.sh'' | ||
- | ==== Fine-tuning conda to use (only) conda-forge ==== | + | |
+ | ===== Fine-tuning conda to use (only) conda-forge ===== | ||
* ''conda'' will probably work fine with the default settings **if you create simple new environments** with just one package and its dependencies. | * ''conda'' will probably work fine with the default settings **if you create simple new environments** with just one package and its dependencies. | ||
Line 297: | Line 373: | ||
</code> | </code> | ||
===== Initializing conda in terminals ===== | ===== Initializing conda in terminals ===== | ||
+ | <WRAP center round tip 60%> | ||
+ | |||
+ | Reminder: if you have installed Miniconda3 on a Windows computer, you only have to open a terminal with | ||
+ | |||
+ | * ''Start'' menu => ''Anaconda Powershell Prompt'' | ||
+ | * ''conda activate the_environment_you_want'' | ||
+ | </WRAP> | ||
+ | |||
When you open a terminal, your shell needs to know where to find the ''conda'' command used to initialize an environment, or switch between existing environments. This can be **configured in the //shell// configuration files** | When you open a terminal, your shell needs to know where to find the ''conda'' command used to initialize an environment, or switch between existing environments. This can be **configured in the //shell// configuration files** | ||
Line 339: | Line 423: | ||
===== conda resources ===== | ===== conda resources ===== | ||
- | Note: some extra details are available on the much older page //Installing and maintaining UV-CDAT with conda//. You can check later the [[other:uvcdat:conda_notes#useful_conda_commands|Useful conda commands]], but the [[https://docs.conda.io/projects/conda/en/latest/index.html|official conda documentation]] and the [[https://docs.conda.io/projects/conda/en/latest/user-guide/cheatsheet.html|Cheat sheet]] are probably more up-to-date. | + | |
Line 345: | Line 429: | ||
* [[https://docs.conda.io/projects/conda/en/stable/|Official conda website]] | * [[https://docs.conda.io/projects/conda/en/stable/|Official conda website]] | ||
- | * [[https://docs.conda.io/projects/conda/en/stable/user-guide/cheatsheet.html|Cheat sheet]] (2 pages for people in a hurry) | + | * [[https://docs.conda.io/projects/conda/en/stable/user-guide/cheatsheet.html|Cheat sheet]] (2 pages, for people in a hurry) |
* [[https://docs.conda.io/projects/conda/en/stable/commands/|Commands' reference]] | * [[https://docs.conda.io/projects/conda/en/stable/commands/|Commands' reference]] | ||
+ | * Package repositories | ||
+ | * [[https://conda-forge.org/packages/|conda-forge]] <= recommended source of packages | ||
+ | * [[https://anaconda.org/|default]] | ||
- | ==== Useful commands ==== | + | ==== Useful conda commands ==== |
- | There is also a summary available in the [[https://conda.io/docs/_downloads/conda-cheatsheet.pdf|Conda cheat sheet]] | + | This is just a subset of some of the commands, and a subset of their options! For a complete reference, use the [[https://docs.conda.io/projects/conda/en/stable/|official conda website]] |
- | * Get help | + | * **Get help** |
* ''conda -h'' | * ''conda -h'' | ||
* ''conda //command// %%--%%help'': help for a specific //command// | * ''conda //command// %%--%%help'': help for a specific //command// | ||
- | * Available environments | + | |
+ | * **Available environments** | ||
* ''conda env list'' | * ''conda env list'' | ||
- | * Create an environment | + | * The currently active environment has a "*****" character on its line<code>(base) $ conda env list |
- | * ''conda create -n new_environment_name package1 package2 packageN'' | + | # conda environments: |
- | * Delete all packages from an environment and the environment itself | + | # |
- | * ''conda remove -n crap_environment %%--%%all'' | + | base * /homedata/jypmce/miniconda3_2024-03</code> |
- | * Activate an existing environment (available in ''conda env list'') or go back to the default Python | + | |
+ | * **Activate** an existing environment (available in ''conda env list'') or **deactivate**, in order to go back to the default Python | ||
* ''conda activate existing_environment'' | * ''conda activate existing_environment'' | ||
- | * ''conda activate'' (without specifying an environment name) will activate the ''base'' environment | + | * ''conda activate'' (without specifying an environment name) will **activate the ''base'' Miniconda3 environment** |
* Go back to the default Python available on the system | * Go back to the default Python available on the system | ||
* ''conda deactivate'' | * ''conda deactivate'' | ||
- | * You can type ''which python'' to determine where is the current Python you are using | + | * You can type ''which python'' to determine where is the current ''python'' executable you are using is\\ You should **always know which Python you are using!** |
* <code>(base) $ which python | * <code>(base) $ which python | ||
/homedata/jypmce/miniconda3_2024-03/bin/python | /homedata/jypmce/miniconda3_2024-03/bin/python | ||
Line 376: | Line 465: | ||
/usr/bin/python</code> | /usr/bin/python</code> | ||
* Note: //before conda ''4.6''//, you had to type ''**source** activate existing_environment'' and ''**source** deactivate'' | * Note: //before conda ''4.6''//, you had to type ''**source** activate existing_environment'' and ''**source** deactivate'' | ||
- | * Get the list of installed packages, their version and where they come from (e.g. ''conda-forge'', ''pip'') | ||
- | * ''conda list'' | ||
- | * Searching packages | ||
- | * ''conda search %%--%%full-name package_exact_name''\\ ''conda search package_partial_name'' | ||
- | * See also the ''%%--%%channel'' option below | ||
- | * Installing and removing packages\\ Add ''-n target_environment'' to act on another environment than the active one | ||
- | * ''conda install package1 package2 packageN'' | ||
- | * Add ''%%--%%channel channel_url_or_alias'' to install from a specific //channel// | ||
- | * If there is no URL, conda will use the value of ''channel_alias'' that defaults to https://conda.anaconda.org/ | ||
- | * UV-CDAT relies on packages maintained in [[https://conda-forge.github.io|conda-forge]] ([[http://wesmckinney.com/blog/conda-forge-centos-moment/|why conda-forge?]]) and the installation commands should include the following channel option:\\ ''%%--%%channel conda-forge'' | ||
- | * ''conda remove package1 package2 packageN'' | ||
- | * Updating conda (update the //conda// package, not the full Miniconda) | ||
- | * ''conda update -n base -c defaults conda'' | ||
- | * Downgrading conda (required for installing UV-CDAT 2.10) | ||
- | * <code>bash-4.1$ conda install -n root "conda<4.3.13" | ||
- | Fetching package metadata ......... | ||
- | Solving package specifications: . | ||
- | Package plan for installation in environment /home/share/unix_files/cdat/miniconda2: | + | * Get the **list of installed packages**, their version, and where they come from (e.g. ''conda-forge'', ''pip'') |
+ | * ''conda list [-n existing_environment]'' | ||
- | The following packages will be DOWNGRADED due to dependency conflicts: | + | * **Create an environment** and install a few packages and their dependencies |
+ | * ''conda create -n new_environment_name package1 package2 packageN'' | ||
- | conda: 4.3.13-py27_0 --> 4.3.11-py27_0 | + | * **Delete an environment** |
+ | * ''conda remove -n crap_environment %%--%%all'' | ||
- | Proceed ([y]/n)? y | + | * **Find out if a package is available** on the //conda-forge// channel (or other channels) |
+ | * Note: you should always install a package with ''conda'' instead of ''pip'', if it is available on a //conda channel// | ||
+ | * ''conda search package_exact_name''\\ ''conda search '*package_partial_name*''' | ||
+ | * Note: ''search'' with wildcards fails in a //tcsh// shell, as of April 2024 ([[https://github.com/conda/conda/issues/9771|CSH install/search fail with '*' variable in an argument]]) | ||
- | conda-4.3.11-p 100% |###############################################################| Time: 0:00:00 4.28 MB/s | + | * **Install and remove packages**\\ Add ''-n target_environment'' to make changes in another environment than the active one |
+ | * ''conda install package1 package2 packageN'' | ||
+ | * ''conda'' should already be [[#fine-tuning_conda_to_use_only_conda-forge|configured to use conda-forge by default]] if you have read this page carefully | ||
+ | * Add ''-c channel_url_or_alias'' to install from a specific //channel// | ||
+ | * ''conda remove package1 package2 packageN'' | ||
- | bash-4.1$ conda info | + | * **Update** just ''conda'' (update the //conda// package, not the full Miniconda3 //base// environment) |
- | Current conda install: | + | * ''conda update -n base conda'' |
- | platform : linux-64 | + | * **Completely update an environment** (all the packages) |
- | conda version : 4.3.11 | + | * ''conda update -n existing_environment %%--%%all'' |
- | conda is private : False | + | |
- | conda-env version : 4.3.11</code> | + | |
- | * conda does not automatically clean the content of the ''pkgs'' directory! Use the following to //Remove index cache, lock files, tarballs, unused cache packages, and source cache// | + | * **Cleaning... <wrap em>Dangerous</wrap>**!\\ If you have several environments, ''conda clean'' may remove packages that are not needed in an environment, but are still used in other environments, and **you may end up with a broken installation...**\\ ''conda'' does not automatically clean the content of the ''pkgs'' directory!\\ Use the following to //Remove index cache, lock files, tarballs, unused cache packages, and source cache// |
* ''conda clean %%--%%all'' | * ''conda clean %%--%%all'' | ||
+ | |||
+ | ==== conda, pip and virtualenv ==== | ||
+ | |||
+ | * Read the [[https://docs.conda.io/projects/conda/en/stable/commands/#conda-vs-pip-vs-virtualenv-commands|Conda vs. pip vs. virtualenv commands]] section | ||
+ | |||
+ | * Read the [[https://docs.conda.io/projects/conda/en/stable/user-guide/tasks/manage-environments.html#using-pip-in-an-environment|Using pip in an environment]] | ||
==== Installation history ==== | ==== Installation history ==== | ||
+ | |||
+ | |||
+ | ===== Uninstalling Miniconda3 ===== | ||
+ | |||
+ | ==== Windows computer ==== | ||
+ | |||
+ | Miniconda3 can be removed like a regular Windows application | ||
+ | |||
+ | * Go to the //Apps & features// panel | ||
+ | * ''Start'' => ''Settings'' => ''Apps'' => ''Apps & features'' | ||
+ | * Select ''Miniconda3 py3xxxxx'' | ||
+ | * Click on ''Uninstall'' and follow the instructions displayed by the uninstaller window\\ This will remove: | ||
+ | * The (very big) directory that you specified when you installed //Miniconda3// | ||
+ | * The ''conda'' shortcuts from the ''Start Menu'' | ||
+ | |||
+ | ==== Linux-like computer ==== | ||