Differences

This shows you the differences between two versions of the page.

--- other:uvcdat:cdat_conda:cdat_8_2_1 [2021/03/03 14:27]
jypeter [Downloading cdms2/vcs test data]
+++ other:uvcdat:cdat_conda:cdat_8_2_1 [2021/03/08 15:35]
jypeter [Updating conda]
@@ Line 1: / Line 1: @@
-====== CDAT 8.2.1 installation notes ======
+====== Python distribution based on CDAT 8.2.1 notes ======
 [ [[.:index|Back to all versions]] ]
@@ Line 11: / Line 11: @@
 ===== Using JYP version =====
-If you mostly want to use the CDAT 8.2.1 installed by JYP, just use the following steps and then read the //What should I do now?// section below
+If you mostly want to use the **python distribution based on CDAT 8.2.1** installed by JYP, just use the following steps and then read the [[#what_should_i_do_now|What should I do now?]] section below
   * Initialize //conda// with:
@@ Line 23: / Line 23: @@
     * python 3.x: ''conda activate cdatm_py3''
-  * Type ''which python'' (or the ''wp'' alias) and make sure you get something like\\ ''[...]/miniconda3/envs/env_name/bin/python''
+  * Type ''which python'' (or the ''wp'' alias) and make sure you get something looking like
+    * ''[...]/miniconda3<possibly_some_version>/envs/<some_env_name>/bin/python''
+    * e.g. ''cdatm_py3'' environment at LSCE:\\ ''/home/share/unix_files/cdat/miniconda3_21-02/envs/cdatm_py3/bin/python''
@@ Line 40: / Line 42: @@
 ==== You want to replicate this installation on your own server or desktop/laptop ====
+Read the following sections of this web page
+There are some slight differences if you want to install this distribution:
+  * just for you. This is the usual case. Well, if you break something you can always remove everything and start from scratch
+  * for sharing a stable version with several other people
+The notes at the beginning of each section will tell you if you can skip the section if you are just installing for you
 ===== What's New in CDAT 8.2.1? =====
@@ Line 45: / Line 54: @@
   * Full [[https://uvcdat.llnl.gov/changelog.html#8.2.1|Change log]]
-Note: this particular CDAT installation at LSCE (and on other machines/servers) provides **many extra (non-CDAT) packages**. You can jump directly to the [[#extra_packages_list|Extra package list]] at the bottom of this page to see what is available
+Note: this particular CDAT installation at LSCE (and on other machines/servers) provides **many extra (non-CDAT) packages**. You can jump directly to the [[#extra_packages_list|Extra package list]] at the bottom of this page, if you want to see what is available
 ===== Installation with Miniconda3 =====
-We assume that Miniconda3 is already installed. Otherwise, follow the the [[other:uvcdat:cdat_conda:cdat_8_0_py2#installing_miniconda3|Installing Miniconda]] steps (and the //Post-Miniconda3 installation// section) we followed when installing CDAT 8.0
+==== Installing Miniconda3 ====
+If //Miniconda3// is not already installed, or if you want to start over with a brand new version, read [[other:uvcdat:cdat_conda:miniconda3_install|JYP steps for installing Miniconda3]].
-==== Updating conda ====
-The conda package itself can be updated (if need be) with\\ ''conda update -n base -c defaults conda''
 ==== Installing CDAT 8.2.1 ====
@@ Line 94: / Line 103: @@
 The installation line below is based on information from the [[https://github.com/CDAT/cdat/wiki/install#for-linux-python|CDAT wiki page]]. See also [[https://github.com/CDAT/cdat/issues/2265|Some questions and notes about 8.2.1 installation]]
+<code>
+bash-4.2$ conda config --env --add channels conda-forge
+bash-4.2$ conda config --set channel_priority strict
+bash-4.2$ cat .condarc
+channels:
+  - conda-forge
+  - defaults
+$ conda config --describe channel_priority
+$ conda create -n cdat-8.2.1_nompi_py3 -c conda-forge -c cdat/label/v8.2.1 cdat "libnetcdf=*=nompi_*" "mesalib=18.3.1" "python=3"</code>
 <code>$ conda create -n cdat-8.2.1_py3 -c conda-forge -c cdat/label/v8.2.1 cdat "libnetcdf=*=mpi_openmpi_*" "mesalib=18.3.1" "python=3"</code>
 Resulting list of installed packages
-  * ''conda list -n cdat-8.2.1_py3 > cdat_8.2.1_installed_packages.txt''
+  * ''conda list -n cdat-8.2.1_nompi_py3 > cdat_8.2.1_nompi_21-03-05.txt''
-  * {{ :other:uvcdat:cdat_conda:cdat_8.2.1_installed_packages.txt |}}
+  * no mpi: {{ :other:uvcdat:cdat_conda:cdat_8.2.1_nompi_21-03-05.txt |}}
+  * with mpî: {{ :other:uvcdat:cdat_conda:cdat_8.2.1_installed_packages.txt |}}
 [[https://github.com/CDAT/cdat/issues/2265|Some extra installation notes]]
@@ Line 165: / Line 185: @@
 </code>
-===== Customizing UV-CDAT for LSCE =====
+===== Customizing CDAT for LSCE =====
 ==== A quick test of cdms2 and vcs ====
@@ Line 196: / Line 216: @@
 ==== Packages that have no dependency problems ====
-After cloning, we are ready to install some extra packages that may be requested by LSCE users
+After [[#cloning_the_base_cdat_environment_before_adding_specific_packages_for_lsce|cloning]], we are ready to install some extra (e.g. not directly related to CDAT) packages that may be useful to LSCE users
-  * We first try to install together as many packages as possible that don't require other channels than //conda-forge//, and that don't request a downgrade of what is already installed
+  * We first install together as many packages as possible that don't require other channels than //conda-forge//, and that don't request a significant downgrade of what is already installed
-  * We then install individual extra packages with ''conda install'' or ''pip install''
+    * We use the //dry-run// option of conda in order to check the version (theoretically the latest) and origin (theoretically //conda-forge//) of the packages that would be installed. Or maybe the result will be that there are too many conflicts and the installation cannot proceed with the requested packages combination
+    * conda may take a very long time to report that there are too many conflicts and we have started using [[https://github.com/mamba-org/mamba|mamba]] that uses different solving libraries and is much (**much!**) faster than conda
+    * if the dry-run reports too many conflicts or requests too many downgrades, we try to remove some (not really important) packages and check if it works better
+      * the last resort is too create a new environment with a smaller set of packages if a user really needs a conflicting package
+  * We later install individual extra packages with ''conda install'', ''mamba install'' or ''pip install''
-<code># You can use the following to keep a trace of what will be installed
+=== Pre-installation check with the dry-run option ===
-#$ conda install --dry-run -n cdatm19_py3 -c conda-forge pillow pandas statsmodels seaborn scikit-image seawater gsw netcdf4 pyferret basemap-data-hires xlsxwriter cmocean rpy2 gdal windspharm  > somewhere/extra_packages.txt
-# Install...
+<code>/usr/bin/time mamba install -n cdatm19_nompi_py3 --dry-run -c conda-forge basemap basemap-data-hires cartopy cmocean cmor ferret_datasets gdal gdal glances gsw iris joblib netcdf4 palettable pandas pillow pyferret rpy2 scikit-image scikit-learn seaborn seawater shapely spyder statsmodels windspharm wrf-python xarray xlsxwriter
-$ conda install -n cdatm19_py3 -c conda-forge pillow pandas statsmodels seaborn scikit-image seawater gsw netcdf4 pyferret ferret_datasets basemap-data-hires xlsxwriter cmocean rpy2 gdal windspharm
-[...]
-# Use a similar install line if you want to install the same packages in the python3 version</code>
+[... lots of information about what would be done followed by some information about how long it took ...]
+  Summary:
+  Install: 167 packages
+  Downgrade: 3 packages
+  Total download: 474 MB
+DryRunExit: Dry run. Exiting.
+.31user 0.51system 0:11.45elapsed 77%CPU</code>
+=== Actual installation ===
+<code>/usr/bin/time mamba install -n cdatm19_nompi_py3 -c conda-forge basemap basemap-data-hires cartopy cmocean cmor ferret_datasets gdal gdal glances gsw iris joblib netcdf4 palettable pandas pillow pyferret rpy2 scikit-image scikit-learn seaborn seawater shapely spyder statsmodels windspharm wrf-python xarray xlsxwriter</code>
+Resulting list of installed packages
+  * ''conda list -n cdatm19_nompi_py3 > cdatm19_nompi_py3_21-03-06.txt''
+  * no mpi: {{ :other:uvcdat:cdat_conda:cdatm19_nompi_py3_21-03-06.txt |}}
+=== Extra packages installed with pip ===
-List of installed packages: {{ :other:uvcdat:cdat_conda:lsce-extra_01_install_190304.txt |}}
-Packages installed with //pip//
   * [[https://earthsystemcog.org/projects/wip/CMIP6DataRequest|dreqPy]]: CMIP6 Data Request Python API
     * ''conda activate cdatm19_py3''\\ ''pip install dreqPy''
@@ Line 222: / Line 262: @@
     * ''conda activate cdatm19_py3''\\ ''pip install ferretmagic''
-The following packages have no dependency problems and were installed (or updated) //later//
+=== Packages with no dependency problems and were added (or updated) later ===
   * [[https://cmor.llnl.gov/|CMOR]]: CMOR (//Climate Model Output Rewriter//) is used to produce CF-compliant netCDF files

PMIP3 wiki

User Tools

Site Tools

Differences

Page Tools