This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
other:uvcdat:cdat_conda:index [2017/02/27 15:37] pmip3adm ↷ Page moved and renamed from other:uvcdat:cdat_conda to other:uvcdat:cdat_conda:index |
other:uvcdat:cdat_conda:index [2021/02/26 14:31] jypeter [UV-CDAT conda-based versions installed by JYP] Updated |
||
---|---|---|---|
Line 1: | Line 1: | ||
- | ====== UV-CDAT conda-based versions installed by JYP ====== | + | ====== CDAT versions installed by JYP ====== |
<WRAP center round important 60%> | <WRAP center round important 60%> | ||
- | This page describes the conda-based versions of UV-CDAT, 2.8.0 and later, installed by JYP | + | This page describes the **conda-based** versions of CDAT, ''2.8.0'' and later, installed by JYP |
- | For technical installation details, check [[.:conda_notes|Installing and maintaining UV-CDAT with conda]] | + | Follow the [[other:python:starting#ultra_quick-start_on_the_lsce_servers|Ultra quick-start on the LSCE servers]] instructions for actually **using CDAT at LSCE** |
- | For versions up to 2.1, check the [[.:installed|UV-CDAT versions installed by JYP]] | + | For technical installation details, check [[..:conda_notes|Installing and maintaining CDAT with conda]] |
+ | |||
+ | For versions up to 2.1, check the (deprecated) [[..:installed|CDAT versions installed by JYP]] page | ||
</WRAP> | </WRAP> | ||
+ | |||
+ | ===== Installed versions ===== | ||
+ | |||
+ | Listed with the most recent version at the top | ||
+ | |||
+ | * [[.::cdat_8_1|8.1]] | ||
+ | |||
+ | * [[.::cdat_8_0_py2|8.0]] | ||
+ | |||
+ | * [[.:cdat_2_12|2.12]] | ||
+ | |||
+ | * [[.:cdat_2_10|2.10]] | ||
+ | |||
+ | * [[.:cdat_2_8_0|2.8.0]] | ||
+ | |||
===== Useful links ===== | ===== Useful links ===== | ||
Line 19: | Line 36: | ||
* [[https://github.com/UV-CDAT/cdutil|cdutil]] | * [[https://github.com/UV-CDAT/cdutil|cdutil]] | ||
* [[https://github.com/UV-CDAT/genutil|genutil]] | * [[https://github.com/UV-CDAT/genutil|genutil]] | ||
- | |||
- | ===== 2.8.0 ===== | ||
- | |||
- | ==== Installation with Anaconda2 ==== | ||
- | |||
- | <WRAP center round alert 60%> | ||
- | [[#installation_with_miniconda2|Installing with Miniconda2]] rather than Anaconda2 will save disk space! | ||
- | </WRAP> | ||
- | |||
- | <code># Keep a trace of what will be installed, with/without conda-forge | ||
- | conda create --dry-run -n uvcdat-2.8.0 -c uvcdat uvcdat=2.8.0 > /home/scratch01/jypeter/uvcdat-2.8.0_install_dry_no-forge_160220.txt | ||
- | conda create --dry-run -n uvcdat-2.8.0 -c conda-forge -c uvcdat uvcdat=2.8.0 > /home/scratch01/jypeter/uvcdat-2.8.0_install_dry_with-forge_160220.txt | ||
- | |||
- | # Do the actual installation (with conda-forge) | ||
- | conda create -n uvcdat-2.8.0 -c conda-forge -c uvcdat uvcdat=2.8.0</code> | ||
- | |||
- | Lists of installed packages: {{ :other:uvcdat:uvcdat-2.8.0_install_dry_with-forge_160220.txt |with conda-forge}} or {{ :other:uvcdat:uvcdat-2.8.0_install_dry_no-forge_160220.txt |without conda-forge}} | ||
- | |||
- | Size of ''/home/share/unix_files/cdat/anaconda2'': | ||
- | * before installation: **2.1G** | ||
- | * after installation (with conda-forge): **4.6G** | ||
- | |||
- | ==== Installation with Miniconda2 ==== | ||
- | |||
- | Note: after installation, we use **//only// 2.7G** (details below), compared to the 4.6G of the Anaconda2 installation | ||
- | |||
- | <code># Keep a trace of what will be installed (with conda-forge) | ||
- | conda create --dry-run -n uvcdat-2.8.0 -c conda-forge -c uvcdat uvcdat=2.8.0 > /home/scratch01/jypeter/uvcdat-2.8.0_install_dry_with-forge_160220.txt | ||
- | |||
- | # Do the actual installation | ||
- | conda create -n uvcdat-2.8.0 -c conda-forge -c uvcdat uvcdat=2.8.0</code> | ||
- | |||
- | Lists of installed packages: {{ :other:uvcdat:uvcdat-2.8.0_install_dry_with-forge_160221.txt |List of installed packages}} | ||
- | |||
- | <code>bash-4.1$ conda info --envs | ||
- | # conda environments: | ||
- | # | ||
- | uvcdat-2.8.0 /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0 | ||
- | root * /home/share/unix_files/cdat/miniconda2 | ||
- | |||
- | bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2 | ||
- | 2.7G /home/share/unix_files/cdat/miniconda2 | ||
- | |||
- | bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2/envs/* | ||
- | 2.0G /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0 | ||
- | |||
- | bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2/pkgs | ||
- | 2.6G /home/share/unix_files/cdat/miniconda2/pkgs | ||
- | |||
- | bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2/* | ||
- | 4.4M /home/share/unix_files/cdat/miniconda2/bin | ||
- | 4.0K /home/share/unix_files/cdat/miniconda2/conda-bld | ||
- | 276K /home/share/unix_files/cdat/miniconda2/conda-meta | ||
- | 2.0G /home/share/unix_files/cdat/miniconda2/envs | ||
- | 20K /home/share/unix_files/cdat/miniconda2/etc | ||
- | 3.8M /home/share/unix_files/cdat/miniconda2/include | ||
- | 99M /home/share/unix_files/cdat/miniconda2/lib | ||
- | 4.0K /home/share/unix_files/cdat/miniconda2/LICENSE.txt | ||
- | 665M /home/share/unix_files/cdat/miniconda2/pkgs | ||
- | 16K /home/share/unix_files/cdat/miniconda2/share | ||
- | 8.0K /home/share/unix_files/cdat/miniconda2/ssl | ||
- | |||
- | bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0/* | ||
- | 56M /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0/bin | ||
- | 1.9M /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0/conda-meta | ||
- | 48K /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0/etc | ||
- | 420K /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0/imports | ||
- | 67M /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0/include | ||
- | 1.7G /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0/lib | ||
- | 24K /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0/man | ||
- | 3.1M /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0/mkspecs | ||
- | 16M /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0/mod | ||
- | 3.5M /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0/plugins | ||
- | 57M /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0/share | ||
- | 4.3M /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0/ssl | ||
- | 2.7M /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0/tests | ||
- | 12K /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0/var | ||
- | 107M /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0/vistrails</code> | ||
- | |||
- | ==== Cloning uvcdat to add specific packages for LSCE ==== | ||
- | |||
- | Note: using hard links, cloning a full environment only adds an extra 100M of disk space | ||
- | |||
- | <code>bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2 | ||
- | 2.7G /home/share/unix_files/cdat/miniconda2 | ||
- | |||
- | bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2/envs/* | ||
- | 2.0G /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0 | ||
- | |||
- | bash-4.1$ conda create -n cdatm14 --clone uvcdat-2.8.0 | ||
- | [...] | ||
- | |||
- | bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2 | ||
- | 2.8G /home/share/unix_files/cdat/miniconda2 | ||
- | |||
- | bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2/envs/* | ||
- | 2.0G /home/share/unix_files/cdat/miniconda2/envs/cdatm14 | ||
- | 82M /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0</code> | ||
- | |||
- | === Packages that have no dependency problems === | ||
- | |||
- | After cloning, we are ready to install some extra packages that may be requested by LSCE users | ||
- | |||
- | <code># Keep a trace of what will be installed | ||
- | bash-4.1$ conda install --dry-run -n cdatm14 -c conda-forge -c uvcdat -c vacumm pillow pandas statsmodels seaborn scikit-image seawater gsw netcdf4 pyferret vacumm > /home/scratch01/jypeter/lsce-extra_01_install_160223.txt | ||
- | |||
- | # Install... | ||
- | bash-4.1$ conda install -n cdatm14 -c conda-forge -c uvcdat -c vacumm pillow pandas statsmodels seaborn scikit-image seawater gsw netcdf4 pyferret vacumm | ||
- | |||
- | # Check the disk space after installation | ||
- | bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2 | ||
- | 3.6G /home/share/unix_files/cdat/miniconda2 | ||
- | |||
- | bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2/envs/* | ||
- | 2.4G /home/share/unix_files/cdat/miniconda2/envs/cdatm14 | ||
- | 82M /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0 | ||
- | |||
- | # Check the disk space again after cleaning | ||
- | bash-4.1$ conda clean all | ||
- | |||
- | bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2 | ||
- | 2.9G /home/share/unix_files/cdat/miniconda2 | ||
- | |||
- | bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2/envs/* | ||
- | 2.4G /home/share/unix_files/cdat/miniconda2/envs/cdatm14 | ||
- | 82M /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0</code> | ||
- | |||
- | Lists of installed packages: {{ :other:uvcdat:lsce-extra_01_install_160223.txt |}} | ||
- | |||
- | The following packages have no dependency problems and were installed //later// | ||
- | * High resolution data for ''basemap'' | ||
- | * ''conda install -n cdatm14 -c conda-forge basemap-data-hires'' | ||
- | |||
- | === Other packages === | ||
- | |||
- | <note warning>There is no warranty that the packages listed below will work correctly, because it was required to bypass the compatibility checks in order to install them...</note> | ||
- | |||
- | * ''shapely'' request a downgrade of ''geos'' when installed from //conda-forge// (see [[https://github.com/Toblerity/Shapely/issues/472|github issue]]) but it will work when installed from the default repo | ||
- | * ''conda install -n cdatm14 shapely'' | ||
- | * We get a slightly older version (//1.5.**16**-py27_0// instead of //1.5.**17**-np111py27_2//) | ||
- | * Note: ''cartopy'' (below) requests the same downgrade of ''geos''... | ||
- | * ''cartopy'' requests a downgrade of ''geos'' and ''matplotlib'' (see [[https://github.com/SciTools/cartopy/issues/853#issuecomment-282023158|github issue]]) | ||
- | * We install the requested dependencies and then we **force** the installation of ''cartopy''\\ <code>(uvcdat-2.8.0) bash-4.1$ conda install -n cdatm14 -c conda-forge -c uvcdat libxslt lxml owslib pyepsg | ||
- | [...] | ||
- | The following NEW packages will be INSTALLED: | ||
- | |||
- | libxslt: 1.1.29-3 conda-forge | ||
- | lxml: 3.7.2-py27_1 conda-forge | ||
- | owslib: 0.14.0-py27_0 conda-forge | ||
- | pyepsg: 0.3.1-py27_0 conda-forge | ||
- | [...] | ||
- | |||
- | (uvcdat-2.8.0) bash-4.1$ conda install -n cdatm14 -c conda-forge cartopy | ||
- | [...] | ||
- | The following NEW packages will be INSTALLED: | ||
- | |||
- | cartopy: 0.15.0-np111py27_0 conda-forge | ||
- | |||
- | The following packages will be DOWNGRADED due to dependency conflicts: | ||
- | |||
- | geos: 3.6.0-0 conda-forge --> 3.5.1-1 conda-forge | ||
- | matplotlib: 2.0.0-np111py27_2 conda-forge --> 1.5.3-np111py27_7 conda-forge | ||
- | |||
- | Proceed ([y]/n)? n | ||
- | |||
- | (uvcdat-2.8.0) bash-4.1$ conda install --force -n cdatm14 -c conda-forge cartopy | ||
- | [...] | ||
- | The following NEW packages will be INSTALLED: | ||
- | |||
- | cartopy: 0.15.0-np111py27_0 conda-forge | ||
- | |||
- | Proceed ([y]/n)? y</code> | ||
- | * trying to install ''rpy2'' will trigger the installation of ''R''! | ||
- | * We force the **installation of only rpy2** in order to use the ''R'' already installed at LSCE | ||
- | * ''conda install --force -n cdatm14 -c conda-forge -c uvcdat rpy2'' | ||
- | |||
- | ==== Extra packages list ==== | ||
- | |||
- | * [[https://python-pillow.org/|pillow]]: the friendly PIL (//Python Imaging Library//) fork | ||
- | * [[http://pandas.pydata.org/|pandas]]: Python Data Analysis Library | ||
- | * [[http://statsmodels.sourceforge.net/|statsmodels]]: a Python module that allows users to explore data, estimate statistical models, and perform statistical tests | ||
- | * [[https://seaborn.pydata.org/|seaborn]]: statistical data visualization | ||
- | * [[http://scikit-image.org/|scikit-image]]: image processing in Python | ||
- | * [[http://pythonhosted.org/seawater/|seawater]]: Python re-write of the CSIRO seawater toolbox | ||
- | * [[https://pypi.python.org/pypi/gsw/|gsw]]: Python implementation of the Thermodynamic Equation Of Seawater | ||
- | * [[http://unidata.github.io/netcdf4-python/|netcdf4]]: a Python interface to the netCDF C library | ||
- | * [[http://ferret.pmel.noaa.gov/Ferret/documentation/pyferret|pyferret]]: Ferret encapsulated in Python | ||
- | * [[http://www.ifremer.fr/vacumm/|vacumm]]: Validation, Analysis, Comparison - Utilities written in Python to validate and analyze Multi-Model outputs, and compare them to observations | ||
- | * [[https://github.com/conda-forge/basemap-data-hires-feedstock|basemap-data-hires]]: high resolution data for ''basemap'' | ||
- | |||
- | * [[http://toblerity.org/shapely/project.html|shapely]]: a Python wrapper for GEOS for algebraic manipulation of geometry (manipulation and analysis of geometric objects in the Cartesian plane) | ||
- | * [[http://scitools.org.uk/cartopy/|cartopy]]: a library providing cartographic tools for python | ||
- | * [[https://rpy2.readthedocs.io/en/version_2.8.x/|rpy2]]: providing simple and robust access to R from within Python | ||
- | |||
- | ==== Environments summary ==== | ||
- | |||
- | After following the steps above, we get the following environments: | ||
- | * uvcdat-2.8.0 | ||
- | * cdatm14 | ||
- | |||
/* standard page footer */ | /* standard page footer */ | ||