UV-CDAT conda-based versions installed by JYP

This page describes the conda-based versions of UV-CDAT, 2.8.0 and later installed by JYP For versions up to 2.1, check the UV-CDAT versions installed by JYP

Useful links

Official CDAT site

CDAT wiki

CDAT @ github

cdms

cdtime

vcs

cdutil

genutil

2.8.0

Installation with Anaconda2

Installing with Miniconda2 rather than Anaconda2 will save disk space!

# Keep a trace of what will be installed, with/without conda-forge
conda create --dry-run -n uvcdat-2.8.0 -c uvcdat uvcdat=2.8.0 > /home/scratch01/jypeter/uvcdat-2.8.0_install_dry_no-forge_160220.txt
conda create --dry-run -n uvcdat-2.8.0 -c conda-forge -c uvcdat uvcdat=2.8.0 > /home/scratch01/jypeter/uvcdat-2.8.0_install_dry_with-forge_160220.txt
# Do the actual installation
conda create -n uvcdat-2.8.0 -c conda-forge -c uvcdat uvcdat=2.8.0

Lists of installed packages: with conda-forge or without conda-forge

Size of /home/share/unix_files/cdat/anaconda2:

before installation: 2.1G

after installation: 4.6G

Installation with Miniconda2

Note: after installation, we use only 2.7G (details below), compared to the 4.6G of the Anaconda2 installation

# Keep a trace of what will be installed (with conda-forge)
conda create --dry-run -n uvcdat-2.8.0 -c conda-forge -c uvcdat uvcdat=2.8.0 > /home/scratch01/jypeter/uvcdat-2.8.0_install_dry_with-forge_160220.txt
# Do the actual installation
conda create -n uvcdat-2.8.0 -c conda-forge -c uvcdat uvcdat=2.8.0

Lists of installed packages: List of installed packages

bash-4.1$ conda info --envs
# conda environments:
#
uvcdat-2.8.0 /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0
root * /home/share/unix_files/cdat/miniconda2
bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2
2.7G /home/share/unix_files/cdat/miniconda2
bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2/envs/*
2.0G /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0
bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2/pkgs
2.6G /home/share/unix_files/cdat/miniconda2/pkgs
bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2/*
4.4M /home/share/unix_files/cdat/miniconda2/bin
4.0K /home/share/unix_files/cdat/miniconda2/conda-bld
276K /home/share/unix_files/cdat/miniconda2/conda-meta
2.0G /home/share/unix_files/cdat/miniconda2/envs
20K /home/share/unix_files/cdat/miniconda2/etc
3.8M /home/share/unix_files/cdat/miniconda2/include
99M /home/share/unix_files/cdat/miniconda2/lib
4.0K /home/share/unix_files/cdat/miniconda2/LICENSE.txt
665M /home/share/unix_files/cdat/miniconda2/pkgs
16K /home/share/unix_files/cdat/miniconda2/share
8.0K /home/share/unix_files/cdat/miniconda2/ssl
bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0/*
56M /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0/bin
1.9M /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0/conda-meta
48K /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0/etc
420K /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0/imports
67M /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0/include
1.7G /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0/lib
24K /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0/man
3.1M /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0/mkspecs
16M /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0/mod
3.5M /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0/plugins
57M /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0/share
4.3M /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0/ssl
2.7M /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0/tests
12K /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0/var
107M /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0/vistrails

Cloning uvcdat to add specific packages for LSCE

Note: using hard links, cloning a full environment only adds an extra 100M of disk space

bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2
2.7G /home/share/unix_files/cdat/miniconda2
bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2/envs/*
2.0G /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0
bash-4.1$ conda create -n cdatm14 --clone uvcdat-2.8.0
[...]
bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2
2.8G /home/share/unix_files/cdat/miniconda2
bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2/envs/*
2.0G /home/share/unix_files/cdat/miniconda2/envs/cdatm14
82M /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0

Packages that have no dependency problems

After cloning, we are ready to install some extra packages that may be requested by LSCE users

# Keep a trace of what will be installed
bash-4.1$ conda install --dry-run -n cdatm14 -c conda-forge -c uvcdat -c vacumm pillow pandas statsmodels seaborn scikit-image seawater gsw netcdf4 pyferret vacumm > /home/scratch01/jypeter/lsce-extra_01_install_160223.txt
# Install...
bash-4.1$ conda install -n cdatm14 -c conda-forge -c uvcdat -c vacumm pillow pandas statsmodels seaborn scikit-image seawater gsw netcdf4 pyferret vacumm
# Check the disk space after installation
bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2
3.6G /home/share/unix_files/cdat/miniconda2
bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2/envs/*
2.4G /home/share/unix_files/cdat/miniconda2/envs/cdatm14
82M /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0
# Check the disk space again after cleaning
bash-4.1$ conda clean all
bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2
2.9G /home/share/unix_files/cdat/miniconda2
bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2/envs/*
2.4G /home/share/unix_files/cdat/miniconda2/envs/cdatm14
82M /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0

Lists of installed packages: lsce-extra_01_install_160223.txt

Other packages

There is no warranty that the packages listed below will work correctly, when it was required to bypass the compatibility checks in order to install them...

shapely request a downgrade of geos when installed from conda-forge (see github issue) but it will work when installed from the default repo

conda install -n cdatm14 shapely

We get a slightly older version (1.5.16-py27_0 instead of 1.5.17-np111py27_2)

Note: cartopy (below) requests the same downgrade of geos...

cartopy requests a downgrade of geos and matplotlib (see github issue)

We install the requested dependencies and then we force the installation of cartopy

(uvcdat-2.8.0) bash-4.1$ conda install -n cdatm14 -c conda-forge -c uvcdat libxslt lxml owslib pyepsg
[...]
The following NEW packages will be INSTALLED:
libxslt: 1.1.29-3 conda-forge
lxml: 3.7.2-py27_1 conda-forge
owslib: 0.14.0-py27_0 conda-forge
pyepsg: 0.3.1-py27_0 conda-forge
[...]
(uvcdat-2.8.0) bash-4.1$ conda install -n cdatm14 -c conda-forge cartopy
[...]
The following NEW packages will be INSTALLED:
cartopy: 0.15.0-np111py27_0 conda-forge
The following packages will be DOWNGRADED due to dependency conflicts:
geos: 3.6.0-0 conda-forge --> 3.5.1-1 conda-forge
matplotlib: 2.0.0-np111py27_2 conda-forge --> 1.5.3-np111py27_7 conda-forge
Proceed ([y]/n)? n
(uvcdat-2.8.0) bash-4.1$ conda install --force -n cdatm14 -c conda-forge cartopy
[...]
The following NEW packages will be INSTALLED:
cartopy: 0.15.0-np111py27_0 conda-forge
Proceed ([y]/n)? y

trying to install rpy2 will trigger the installation of R!

We force the installation of only rpy2 in order to use the R already installed at LSCE

conda install --force -n cdatm14 -c conda-forge -c uvcdat rpy2

Extra packages list

pillow: the friendly PIL (Python Imaging Library) fork

pandas: Python Data Analysis Library

statsmodels: a Python module that allows users to explore data, estimate statistical models, and perform statistical tests

seaborn: statistical data visualization

scikit-image: image processing in Python

seawater: Python re-write of the CSIRO seawater toolbox

gsw: Python implementation of the Thermodynamic Equation Of Seawater

netcdf4: a Python interface to the netCDF C library

pyferret: Ferret encapsulated in Python

vacumm: Validation, Analysis, Comparison - Utilities written in Python to validate and analyze Multi-Model outputs, and compare them to observations

shapely: a Python wrapper for GEOS for algebraic manipulation of geometry (manipulation and analysis of geometric objects in the Cartesian plane)

cartopy: a library providing cartographic tools for python

rpy2: providing simple and robust access to R from within Python

[ PMIP3 Wiki Home ] - [ Help! ] - [ Wiki syntax ]