This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
other:uvcdat:cdat_conda:cdat_8_2_1 [2021/03/03 15:39] jypeter [Packages that have no dependency problems] |
other:uvcdat:cdat_conda:cdat_8_2_1 [2021/03/06 21:16] jypeter [Packages that have no dependency problems] Added the package list for nompi |
||
---|---|---|---|
Line 94: | Line 94: | ||
The installation line below is based on information from the [[https://github.com/CDAT/cdat/wiki/install#for-linux-python|CDAT wiki page]]. See also [[https://github.com/CDAT/cdat/issues/2265|Some questions and notes about 8.2.1 installation]] | The installation line below is based on information from the [[https://github.com/CDAT/cdat/wiki/install#for-linux-python|CDAT wiki page]]. See also [[https://github.com/CDAT/cdat/issues/2265|Some questions and notes about 8.2.1 installation]] | ||
+ | <code> | ||
+ | bash-4.2$ conda config --env --add channels conda-forge | ||
+ | bash-4.2$ conda config --set channel_priority strict | ||
+ | bash-4.2$ cat .condarc | ||
+ | channels: | ||
+ | - conda-forge | ||
+ | - defaults | ||
+ | $ conda config --describe channel_priority | ||
+ | |||
+ | $ conda create -n cdat-8.2.1_nompi_py3 -c conda-forge -c cdat/label/v8.2.1 cdat "libnetcdf=*=nompi_*" "mesalib=18.3.1" "python=3"</code> | ||
<code>$ conda create -n cdat-8.2.1_py3 -c conda-forge -c cdat/label/v8.2.1 cdat "libnetcdf=*=mpi_openmpi_*" "mesalib=18.3.1" "python=3"</code> | <code>$ conda create -n cdat-8.2.1_py3 -c conda-forge -c cdat/label/v8.2.1 cdat "libnetcdf=*=mpi_openmpi_*" "mesalib=18.3.1" "python=3"</code> | ||
Resulting list of installed packages | Resulting list of installed packages | ||
- | * ''conda list -n cdat-8.2.1_py3 > cdat_8.2.1_installed_packages.txt'' | + | * ''conda list -n cdat-8.2.1_nompi_py3 > cdat_8.2.1_nompi_21-03-05.txt'' |
- | * {{ :other:uvcdat:cdat_conda:cdat_8.2.1_installed_packages.txt |}} | + | * no mpi: {{ :other:uvcdat:cdat_conda:cdat_8.2.1_nompi_21-03-05.txt |}} |
+ | * with mpî: {{ :other:uvcdat:cdat_conda:cdat_8.2.1_installed_packages.txt |}} | ||
[[https://github.com/CDAT/cdat/issues/2265|Some extra installation notes]] | [[https://github.com/CDAT/cdat/issues/2265|Some extra installation notes]] | ||
Line 165: | Line 176: | ||
</code> | </code> | ||
- | ===== Customizing UV-CDAT for LSCE ===== | + | ===== Customizing CDAT for LSCE ===== |
==== A quick test of cdms2 and vcs ==== | ==== A quick test of cdms2 and vcs ==== | ||
Line 196: | Line 207: | ||
==== Packages that have no dependency problems ==== | ==== Packages that have no dependency problems ==== | ||
- | After [[#cloning_the_base_cdat_environment_before_adding_specific_packages_for_lsce|cloning]], we are ready to install some extra packages that may be requested by LSCE users | + | After [[#cloning_the_base_cdat_environment_before_adding_specific_packages_for_lsce|cloning]], we are ready to install some extra (e.g. not directly related to CDAT) packages that may be useful to LSCE users |
- | * We first try to install together as many packages as possible that don't require other channels than //conda-forge//, and that don't request a downgrade of what is already installed | + | * We first install together as many packages as possible that don't require other channels than //conda-forge//, and that don't request a significant downgrade of what is already installed |
- | * We then install individual extra packages with ''conda install'' or ''pip install'' | + | * We use the //dry-run// option of conda in order to check the version (theoretically the latest) and origin (theoretically //conda-forge//) of the packages that would be installed. Or maybe the result will be that there are too many conflicts and the installation cannot proceed with the requested packages combination |
+ | * conda may take a very long time to report that there are too many conflicts and we have started using [[https://github.com/mamba-org/mamba|mamba]] that uses different solving libraries and is much (**much!**) faster than conda | ||
+ | * if the dry-run reports too many conflicts or requests too many downgrades, we try to remove some (not really important) packages and check if it works better | ||
+ | * the last resort is too create a new environment with a smaller set of packages if a user really needs a conflicting package | ||
+ | * We later install individual extra packages with ''conda install'', ''mamba install'' or ''pip install'' | ||
- | <code># You can use the following to keep a trace of what will be installed | + | === Pre-installation check with the dry-run option === |
- | #$ conda install --dry-run -n cdatm19_py3 -c conda-forge pillow pandas statsmodels seaborn scikit-image seawater gsw netcdf4 pyferret basemap-data-hires xlsxwriter cmocean rpy2 gdal windspharm > somewhere/extra_packages.txt | + | |
- | # Install... | + | <code>/usr/bin/time mamba install -n cdatm19_nompi_py3 --dry-run -c conda-forge basemap basemap-data-hires cartopy cmocean cmor ferret_datasets gdal gdal glances gsw iris joblib netcdf4 palettable pandas pillow pyferret rpy2 scikit-image scikit-learn seaborn seawater shapely spyder statsmodels windspharm wrf-python xarray xlsxwriter |
- | $ conda install -n cdatm19_py3 -c conda-forge pillow pandas statsmodels seaborn scikit-image seawater gsw netcdf4 pyferret ferret_datasets basemap-data-hires xlsxwriter cmocean rpy2 gdal windspharm | + | |
- | [...] | + | |
- | # Use a similar install line if you want to install the same packages in the python3 version</code> | + | [... lots of information about what would be done followed by some information about how long it took ...] |
+ | |||
+ | Summary: | ||
+ | |||
+ | Install: 167 packages | ||
+ | Downgrade: 3 packages | ||
+ | |||
+ | Total download: 474 MB | ||
+ | |||
+ | DryRunExit: Dry run. Exiting. | ||
+ | |||
+ | 8.31user 0.51system 0:11.45elapsed 77%CPU</code> | ||
+ | |||
+ | === Actual installation === | ||
+ | |||
+ | <code>/usr/bin/time mamba install -n cdatm19_nompi_py3 -c conda-forge basemap basemap-data-hires cartopy cmocean cmor ferret_datasets gdal gdal glances gsw iris joblib netcdf4 palettable pandas pillow pyferret rpy2 scikit-image scikit-learn seaborn seawater shapely spyder statsmodels windspharm wrf-python xarray xlsxwriter</code> | ||
+ | |||
+ | Resulting list of installed packages | ||
+ | * ''conda list -n cdatm19_nompi_py3 > cdatm19_nompi_py3_21-03-06.txt'' | ||
+ | * no mpi: {{ :other:uvcdat:cdat_conda:cdatm19_nompi_py3_21-03-06.txt |}} | ||
+ | |||
+ | === Extra packages installed with pip === | ||
- | List of installed packages: {{ :other:uvcdat:cdat_conda:lsce-extra_01_install_190304.txt |}} | ||
- | Packages installed with //pip// | ||
* [[https://earthsystemcog.org/projects/wip/CMIP6DataRequest|dreqPy]]: CMIP6 Data Request Python API | * [[https://earthsystemcog.org/projects/wip/CMIP6DataRequest|dreqPy]]: CMIP6 Data Request Python API | ||
* ''conda activate cdatm19_py3''\\ ''pip install dreqPy'' | * ''conda activate cdatm19_py3''\\ ''pip install dreqPy'' | ||
Line 222: | Line 253: | ||
* ''conda activate cdatm19_py3''\\ ''pip install ferretmagic'' | * ''conda activate cdatm19_py3''\\ ''pip install ferretmagic'' | ||
- | The following packages have no dependency problems and were installed (or updated) //later// | + | === Packages with no dependency problems and were added (or updated) later === |
* [[https://cmor.llnl.gov/|CMOR]]: CMOR (//Climate Model Output Rewriter//) is used to produce CF-compliant netCDF files | * [[https://cmor.llnl.gov/|CMOR]]: CMOR (//Climate Model Output Rewriter//) is used to produce CF-compliant netCDF files |