User Tools

Site Tools


other:uvcdat:cdat_conda:index

This is an old revision of the document!


UV-CDAT conda-based versions installed by JYP

This page describes the conda-based versions of UV-CDAT, 2.8.0 and later, installed by JYP

For technical installation details, check Installing and maintaining UV-CDAT with conda

For versions up to 2.1, check the UV-CDAT versions installed by JYP

2.8.0

Installation with Anaconda2

Installing with Miniconda2 rather than Anaconda2 will save disk space!

# Keep a trace of what will be installed, with/without conda-forge
conda create --dry-run -n uvcdat-2.8.0 -c uvcdat uvcdat=2.8.0 > /home/scratch01/jypeter/uvcdat-2.8.0_install_dry_no-forge_160220.txt
conda create --dry-run -n uvcdat-2.8.0 -c conda-forge -c uvcdat uvcdat=2.8.0 > /home/scratch01/jypeter/uvcdat-2.8.0_install_dry_with-forge_160220.txt

# Do the actual installation (with conda-forge)
conda create -n uvcdat-2.8.0 -c conda-forge -c uvcdat uvcdat=2.8.0

Lists of installed packages: with conda-forge or without conda-forge

Size of /home/share/unix_files/cdat/anaconda2:

  • before installation: 2.1G
  • after installation (with conda-forge): 4.6G

Installation with Miniconda2

Note: after installation, we use only 2.7G (details below), compared to the 4.6G of the Anaconda2 installation

# Keep a trace of what will be installed (with conda-forge)
conda create --dry-run -n uvcdat-2.8.0 -c conda-forge -c uvcdat uvcdat=2.8.0 > /home/scratch01/jypeter/uvcdat-2.8.0_install_dry_with-forge_160220.txt

# Do the actual installation
conda create -n uvcdat-2.8.0 -c conda-forge -c uvcdat uvcdat=2.8.0

Lists of installed packages: List of installed packages

bash-4.1$ conda info --envs
# conda environments:
#
uvcdat-2.8.0             /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0
root                  *  /home/share/unix_files/cdat/miniconda2

bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2
2.7G    /home/share/unix_files/cdat/miniconda2

bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2/envs/*
2.0G    /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0

bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2/pkgs
2.6G    /home/share/unix_files/cdat/miniconda2/pkgs

bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2/*
4.4M    /home/share/unix_files/cdat/miniconda2/bin
4.0K    /home/share/unix_files/cdat/miniconda2/conda-bld
276K    /home/share/unix_files/cdat/miniconda2/conda-meta
2.0G    /home/share/unix_files/cdat/miniconda2/envs
20K     /home/share/unix_files/cdat/miniconda2/etc
3.8M    /home/share/unix_files/cdat/miniconda2/include
99M     /home/share/unix_files/cdat/miniconda2/lib
4.0K    /home/share/unix_files/cdat/miniconda2/LICENSE.txt
665M    /home/share/unix_files/cdat/miniconda2/pkgs
16K     /home/share/unix_files/cdat/miniconda2/share
8.0K    /home/share/unix_files/cdat/miniconda2/ssl

bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0/*
56M     /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0/bin
1.9M    /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0/conda-meta
48K     /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0/etc
420K    /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0/imports
67M     /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0/include
1.7G    /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0/lib
24K     /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0/man
3.1M    /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0/mkspecs
16M     /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0/mod
3.5M    /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0/plugins
57M     /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0/share
4.3M    /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0/ssl
2.7M    /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0/tests
12K     /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0/var
107M    /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0/vistrails

Cloning uvcdat to add specific packages for LSCE

Note: using hard links, cloning a full environment only adds an extra 100M of disk space

bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2
2.7G    /home/share/unix_files/cdat/miniconda2

bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2/envs/*
2.0G    /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0

bash-4.1$ conda create -n cdatm14 --clone uvcdat-2.8.0
[...]

bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2
2.8G    /home/share/unix_files/cdat/miniconda2

bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2/envs/*
2.0G    /home/share/unix_files/cdat/miniconda2/envs/cdatm14
82M     /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0

Packages that have no dependency problems

After cloning, we are ready to install some extra packages that may be requested by LSCE users

# Keep a trace of what will be installed
bash-4.1$ conda install --dry-run -n cdatm14 -c conda-forge -c uvcdat -c vacumm pillow pandas statsmodels seaborn scikit-image seawater gsw netcdf4 pyferret vacumm > /home/scratch01/jypeter/lsce-extra_01_install_160223.txt

# Install...
bash-4.1$ conda install -n cdatm14 -c conda-forge -c uvcdat -c vacumm pillow pandas statsmodels seaborn scikit-image seawater gsw netcdf4 pyferret vacumm

# Check the disk space after installation
bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2
3.6G    /home/share/unix_files/cdat/miniconda2

bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2/envs/*
2.4G    /home/share/unix_files/cdat/miniconda2/envs/cdatm14
82M     /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0

# Check the disk space again after cleaning
bash-4.1$ conda clean all

bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2
2.9G    /home/share/unix_files/cdat/miniconda2

bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2/envs/*
2.4G    /home/share/unix_files/cdat/miniconda2/envs/cdatm14
82M     /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0

Lists of installed packages: lsce-extra_01_install_160223.txt

The following packages have no dependency problems and were installed later

  • High resolution data for basemap
    • conda install -n cdatm14 -c conda-forge basemap-data-hires

Other packages

There is no warranty that the packages listed below will work correctly, because it was required to bypass the compatibility checks in order to install them…
  • shapely request a downgrade of geos when installed from conda-forge (see github issue) but it will work when installed from the default repo
    • conda install -n cdatm14 shapely
    • We get a slightly older version (1.5.16-py27_0 instead of 1.5.17-np111py27_2)
    • Note: cartopy (below) requests the same downgrade of geos
  • cartopy requests a downgrade of geos and matplotlib (see github issue)
    • We install the requested dependencies and then we force the installation of cartopy
      (uvcdat-2.8.0) bash-4.1$ conda install -n cdatm14 -c conda-forge -c uvcdat libxslt lxml owslib pyepsg
      [...]
      The following NEW packages will be INSTALLED:
      
          libxslt: 1.1.29-3      conda-forge
          lxml:    3.7.2-py27_1  conda-forge
          owslib:  0.14.0-py27_0 conda-forge
          pyepsg:  0.3.1-py27_0  conda-forge
      [...]
      
      (uvcdat-2.8.0) bash-4.1$ conda install -n cdatm14 -c conda-forge cartopy
      [...]
      The following NEW packages will be INSTALLED:
      
          cartopy:    0.15.0-np111py27_0 conda-forge
      
      The following packages will be DOWNGRADED due to dependency conflicts:
      
          geos:       3.6.0-0            conda-forge --> 3.5.1-1           conda-forge
          matplotlib: 2.0.0-np111py27_2  conda-forge --> 1.5.3-np111py27_7 conda-forge
      
      Proceed ([y]/n)? n
      
      (uvcdat-2.8.0) bash-4.1$ conda install --force -n cdatm14 -c conda-forge cartopy
      [...]
      The following NEW packages will be INSTALLED:
      
          cartopy: 0.15.0-np111py27_0 conda-forge
      
      Proceed ([y]/n)? y
  • trying to install rpy2 will trigger the installation of R!
    • We force the installation of only rpy2 in order to use the R already installed at LSCE
      • conda install –force -n cdatm14 -c conda-forge -c uvcdat rpy2

Extra packages list

  • pillow: the friendly PIL (Python Imaging Library) fork
  • pandas: Python Data Analysis Library
  • statsmodels: a Python module that allows users to explore data, estimate statistical models, and perform statistical tests
  • seaborn: statistical data visualization
  • scikit-image: image processing in Python
  • seawater: Python re-write of the CSIRO seawater toolbox
  • gsw: Python implementation of the Thermodynamic Equation Of Seawater
  • netcdf4: a Python interface to the netCDF C library
  • pyferret: Ferret encapsulated in Python
  • vacumm: Validation, Analysis, Comparison - Utilities written in Python to validate and analyze Multi-Model outputs, and compare them to observations
  • basemap-data-hires: high resolution data for basemap
  • shapely: a Python wrapper for GEOS for algebraic manipulation of geometry (manipulation and analysis of geometric objects in the Cartesian plane)
  • cartopy: a library providing cartographic tools for python
  • rpy2: providing simple and robust access to R from within Python

Environments summary

After following the steps above, we get the following environments:

  • uvcdat-2.8.0
  • cdatm14





[ PMIP3 Wiki Home ] - [ Help! ] - [ Wiki syntax ]

other/uvcdat/cdat_conda/index.1488209857.txt.gz · Last modified: 2017/02/27 15:37 by pmip3adm