This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision | ||
other:uvcdat:cdat_conda:index [2017/02/27 15:37] pmip3adm ↷ Page moved and renamed from other:uvcdat:cdat_conda to other:uvcdat:cdat_conda:index |
other:uvcdat:cdat_conda:index [2024/03/22 09:56] (current) jypeter [Installed versions] Added link to the 2024-03 version |
||
---|---|---|---|
Line 1: | Line 1: | ||
- | ====== UV-CDAT conda-based versions installed by JYP ====== | + | ====== CDAT versions installed by JYP ====== |
<WRAP center round important 60%> | <WRAP center round important 60%> | ||
- | This page describes the conda-based versions of UV-CDAT, 2.8.0 and later, installed by JYP | + | This page describes the **conda-based** versions of CDAT, ''2.8.0'' and later, installed by JYP |
- | For technical installation details, check [[.:conda_notes|Installing and maintaining UV-CDAT with conda]] | + | Follow the [[other:python:starting#ultra_quick-start_on_the_lsce_servers|Ultra quick-start on the LSCE servers]] instructions for actually **using CDAT at LSCE** |
- | For versions up to 2.1, check the [[.:installed|UV-CDAT versions installed by JYP]] | + | For technical installation details, check [[..:conda_notes|Installing and maintaining CDAT with conda]] |
- | </WRAP> | + | |
- | + | ||
- | ===== Useful links ===== | + | |
- | + | ||
- | * [[http://uvcdat.llnl.gov/|Official CDAT site]] | + | |
- | * [[https://github.com/UV-CDAT/uvcdat/wiki/|CDAT wiki]] | + | |
- | * [[https://github.com/UV-CDAT|CDAT @ github]] | + | |
- | * [[https://github.com/UV-CDAT/cdms|cdms]] | + | |
- | * [[https://github.com/UV-CDAT/cdtime|cdtime]] | + | |
- | * [[https://github.com/UV-CDAT/vcs|vcs]] | + | |
- | * [[https://github.com/UV-CDAT/cdutil|cdutil]] | + | |
- | * [[https://github.com/UV-CDAT/genutil|genutil]] | + | |
- | + | ||
- | ===== 2.8.0 ===== | + | |
- | + | ||
- | ==== Installation with Anaconda2 ==== | + | |
- | <WRAP center round alert 60%> | + | For versions up to 2.1, check the (deprecated) [[..:installed|CDAT versions installed by JYP]] page |
- | [[#installation_with_miniconda2|Installing with Miniconda2]] rather than Anaconda2 will save disk space! | + | |
</WRAP> | </WRAP> | ||
- | <code># Keep a trace of what will be installed, with/without conda-forge | + | ===== Installed versions ===== |
- | conda create --dry-run -n uvcdat-2.8.0 -c uvcdat uvcdat=2.8.0 > /home/scratch01/jypeter/uvcdat-2.8.0_install_dry_no-forge_160220.txt | + | |
- | conda create --dry-run -n uvcdat-2.8.0 -c conda-forge -c uvcdat uvcdat=2.8.0 > /home/scratch01/jypeter/uvcdat-2.8.0_install_dry_with-forge_160220.txt | + | |
- | # Do the actual installation (with conda-forge) | + | Listed with the most recent version at the top |
- | conda create -n uvcdat-2.8.0 -c conda-forge -c uvcdat uvcdat=2.8.0</code> | + | |
- | Lists of installed packages: {{ :other:uvcdat:uvcdat-2.8.0_install_dry_with-forge_160220.txt |with conda-forge}} or {{ :other:uvcdat:uvcdat-2.8.0_install_dry_no-forge_160220.txt |without conda-forge}} | + | ==== Dated (but not outdated!) versions ==== |
- | Size of ''/home/share/unix_files/cdat/anaconda2'': | + | These versions are based on ''xcdat'' and lots of other useful packages, and the latest Python version available at the date of installation |
- | * before installation: **2.1G** | + | |
- | * after installation (with conda-forge): **4.6G** | + | |
- | ==== Installation with Miniconda2 ==== | + | * [[other:uvcdat:cdat_conda:cdat_2024_03|2024-03]] |
- | Note: after installation, we use **//only// 2.7G** (details below), compared to the 4.6G of the Anaconda2 installation | ||
- | <code># Keep a trace of what will be installed (with conda-forge) | + | ==== CDAT based versions ==== |
- | conda create --dry-run -n uvcdat-2.8.0 -c conda-forge -c uvcdat uvcdat=2.8.0 > /home/scratch01/jypeter/uvcdat-2.8.0_install_dry_with-forge_160220.txt | + | |
- | # Do the actual installation | + | These versions are based on **CDAT** packages (''cdms2'' and friends) that are unfortunately not compatible with //recent// versions of numpy |
- | conda create -n uvcdat-2.8.0 -c conda-forge -c uvcdat uvcdat=2.8.0</code> | + | |
- | Lists of installed packages: {{ :other:uvcdat:uvcdat-2.8.0_install_dry_with-forge_160221.txt |List of installed packages}} | + | * [[.:cdat_8_2_1|8.2.1]] |
- | <code>bash-4.1$ conda info --envs | + | * [[.:cdat_8_1|8.1]] |
- | # conda environments: | + | |
- | # | + | |
- | uvcdat-2.8.0 /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0 | + | |
- | root * /home/share/unix_files/cdat/miniconda2 | + | |
- | bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2 | + | * [[.::cdat_8_0_py2|8.0]] |
- | 2.7G /home/share/unix_files/cdat/miniconda2 | + | |
- | bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2/envs/* | + | * [[.:cdat_2_12|2.12]] |
- | 2.0G /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0 | + | |
- | bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2/pkgs | + | * [[.:cdat_2_10|2.10]] |
- | 2.6G /home/share/unix_files/cdat/miniconda2/pkgs | + | |
- | bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2/* | + | * [[.:cdat_2_8_0|2.8.0]] |
- | 4.4M /home/share/unix_files/cdat/miniconda2/bin | + | |
- | 4.0K /home/share/unix_files/cdat/miniconda2/conda-bld | + | |
- | 276K /home/share/unix_files/cdat/miniconda2/conda-meta | + | |
- | 2.0G /home/share/unix_files/cdat/miniconda2/envs | + | |
- | 20K /home/share/unix_files/cdat/miniconda2/etc | + | |
- | 3.8M /home/share/unix_files/cdat/miniconda2/include | + | |
- | 99M /home/share/unix_files/cdat/miniconda2/lib | + | |
- | 4.0K /home/share/unix_files/cdat/miniconda2/LICENSE.txt | + | |
- | 665M /home/share/unix_files/cdat/miniconda2/pkgs | + | |
- | 16K /home/share/unix_files/cdat/miniconda2/share | + | |
- | 8.0K /home/share/unix_files/cdat/miniconda2/ssl | + | |
- | bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0/* | ||
- | 56M /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0/bin | ||
- | 1.9M /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0/conda-meta | ||
- | 48K /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0/etc | ||
- | 420K /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0/imports | ||
- | 67M /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0/include | ||
- | 1.7G /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0/lib | ||
- | 24K /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0/man | ||
- | 3.1M /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0/mkspecs | ||
- | 16M /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0/mod | ||
- | 3.5M /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0/plugins | ||
- | 57M /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0/share | ||
- | 4.3M /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0/ssl | ||
- | 2.7M /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0/tests | ||
- | 12K /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0/var | ||
- | 107M /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0/vistrails</code> | ||
- | ==== Cloning uvcdat to add specific packages for LSCE ==== | + | ===== Useful links ===== |
- | Note: using hard links, cloning a full environment only adds an extra 100M of disk space | + | * [[http://uvcdat.llnl.gov/|Official CDAT site]] |
- | + | * [[https://github.com/UV-CDAT/uvcdat/wiki/|CDAT wiki]] | |
- | <code>bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2 | + | * [[https://github.com/UV-CDAT|CDAT @ github]] |
- | 2.7G /home/share/unix_files/cdat/miniconda2 | + | * [[https://github.com/UV-CDAT/cdms|cdms]] |
- | + | * [[https://github.com/UV-CDAT/cdtime|cdtime]] | |
- | bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2/envs/* | + | * [[https://github.com/UV-CDAT/vcs|vcs]] |
- | 2.0G /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0 | + | * [[https://github.com/UV-CDAT/cdutil|cdutil]] |
- | + | * [[https://github.com/UV-CDAT/genutil|genutil]] | |
- | bash-4.1$ conda create -n cdatm14 --clone uvcdat-2.8.0 | + | |
- | [...] | + | |
- | + | ||
- | bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2 | + | |
- | 2.8G /home/share/unix_files/cdat/miniconda2 | + | |
- | + | ||
- | bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2/envs/* | + | |
- | 2.0G /home/share/unix_files/cdat/miniconda2/envs/cdatm14 | + | |
- | 82M /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0</code> | + | |
- | + | ||
- | === Packages that have no dependency problems === | + | |
- | + | ||
- | After cloning, we are ready to install some extra packages that may be requested by LSCE users | + | |
- | + | ||
- | <code># Keep a trace of what will be installed | + | |
- | bash-4.1$ conda install --dry-run -n cdatm14 -c conda-forge -c uvcdat -c vacumm pillow pandas statsmodels seaborn scikit-image seawater gsw netcdf4 pyferret vacumm > /home/scratch01/jypeter/lsce-extra_01_install_160223.txt | + | |
- | + | ||
- | # Install... | + | |
- | bash-4.1$ conda install -n cdatm14 -c conda-forge -c uvcdat -c vacumm pillow pandas statsmodels seaborn scikit-image seawater gsw netcdf4 pyferret vacumm | + | |
- | + | ||
- | # Check the disk space after installation | + | |
- | bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2 | + | |
- | 3.6G /home/share/unix_files/cdat/miniconda2 | + | |
- | + | ||
- | bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2/envs/* | + | |
- | 2.4G /home/share/unix_files/cdat/miniconda2/envs/cdatm14 | + | |
- | 82M /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0 | + | |
- | + | ||
- | # Check the disk space again after cleaning | + | |
- | bash-4.1$ conda clean all | + | |
- | + | ||
- | bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2 | + | |
- | 2.9G /home/share/unix_files/cdat/miniconda2 | + | |
- | + | ||
- | bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2/envs/* | + | |
- | 2.4G /home/share/unix_files/cdat/miniconda2/envs/cdatm14 | + | |
- | 82M /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0</code> | + | |
- | + | ||
- | Lists of installed packages: {{ :other:uvcdat:lsce-extra_01_install_160223.txt |}} | + | |
- | + | ||
- | The following packages have no dependency problems and were installed //later// | + | |
- | * High resolution data for ''basemap'' | + | |
- | * ''conda install -n cdatm14 -c conda-forge basemap-data-hires'' | + | |
- | + | ||
- | === Other packages === | + | |
- | + | ||
- | <note warning>There is no warranty that the packages listed below will work correctly, because it was required to bypass the compatibility checks in order to install them...</note> | + | |
- | + | ||
- | * ''shapely'' request a downgrade of ''geos'' when installed from //conda-forge// (see [[https://github.com/Toblerity/Shapely/issues/472|github issue]]) but it will work when installed from the default repo | + | |
- | * ''conda install -n cdatm14 shapely'' | + | |
- | * We get a slightly older version (//1.5.**16**-py27_0// instead of //1.5.**17**-np111py27_2//) | + | |
- | * Note: ''cartopy'' (below) requests the same downgrade of ''geos''... | + | |
- | * ''cartopy'' requests a downgrade of ''geos'' and ''matplotlib'' (see [[https://github.com/SciTools/cartopy/issues/853#issuecomment-282023158|github issue]]) | + | |
- | * We install the requested dependencies and then we **force** the installation of ''cartopy''\\ <code>(uvcdat-2.8.0) bash-4.1$ conda install -n cdatm14 -c conda-forge -c uvcdat libxslt lxml owslib pyepsg | + | |
- | [...] | + | |
- | The following NEW packages will be INSTALLED: | + | |
- | + | ||
- | libxslt: 1.1.29-3 conda-forge | + | |
- | lxml: 3.7.2-py27_1 conda-forge | + | |
- | owslib: 0.14.0-py27_0 conda-forge | + | |
- | pyepsg: 0.3.1-py27_0 conda-forge | + | |
- | [...] | + | |
- | + | ||
- | (uvcdat-2.8.0) bash-4.1$ conda install -n cdatm14 -c conda-forge cartopy | + | |
- | [...] | + | |
- | The following NEW packages will be INSTALLED: | + | |
- | + | ||
- | cartopy: 0.15.0-np111py27_0 conda-forge | + | |
- | + | ||
- | The following packages will be DOWNGRADED due to dependency conflicts: | + | |
- | + | ||
- | geos: 3.6.0-0 conda-forge --> 3.5.1-1 conda-forge | + | |
- | matplotlib: 2.0.0-np111py27_2 conda-forge --> 1.5.3-np111py27_7 conda-forge | + | |
- | + | ||
- | Proceed ([y]/n)? n | + | |
- | + | ||
- | (uvcdat-2.8.0) bash-4.1$ conda install --force -n cdatm14 -c conda-forge cartopy | + | |
- | [...] | + | |
- | The following NEW packages will be INSTALLED: | + | |
- | + | ||
- | cartopy: 0.15.0-np111py27_0 conda-forge | + | |
- | + | ||
- | Proceed ([y]/n)? y</code> | + | |
- | * trying to install ''rpy2'' will trigger the installation of ''R''! | + | |
- | * We force the **installation of only rpy2** in order to use the ''R'' already installed at LSCE | + | |
- | * ''conda install --force -n cdatm14 -c conda-forge -c uvcdat rpy2'' | + | |
- | + | ||
- | ==== Extra packages list ==== | + | |
- | + | ||
- | * [[https://python-pillow.org/|pillow]]: the friendly PIL (//Python Imaging Library//) fork | + | |
- | * [[http://pandas.pydata.org/|pandas]]: Python Data Analysis Library | + | |
- | * [[http://statsmodels.sourceforge.net/|statsmodels]]: a Python module that allows users to explore data, estimate statistical models, and perform statistical tests | + | |
- | * [[https://seaborn.pydata.org/|seaborn]]: statistical data visualization | + | |
- | * [[http://scikit-image.org/|scikit-image]]: image processing in Python | + | |
- | * [[http://pythonhosted.org/seawater/|seawater]]: Python re-write of the CSIRO seawater toolbox | + | |
- | * [[https://pypi.python.org/pypi/gsw/|gsw]]: Python implementation of the Thermodynamic Equation Of Seawater | + | |
- | * [[http://unidata.github.io/netcdf4-python/|netcdf4]]: a Python interface to the netCDF C library | + | |
- | * [[http://ferret.pmel.noaa.gov/Ferret/documentation/pyferret|pyferret]]: Ferret encapsulated in Python | + | |
- | * [[http://www.ifremer.fr/vacumm/|vacumm]]: Validation, Analysis, Comparison - Utilities written in Python to validate and analyze Multi-Model outputs, and compare them to observations | + | |
- | * [[https://github.com/conda-forge/basemap-data-hires-feedstock|basemap-data-hires]]: high resolution data for ''basemap'' | + | |
- | + | ||
- | * [[http://toblerity.org/shapely/project.html|shapely]]: a Python wrapper for GEOS for algebraic manipulation of geometry (manipulation and analysis of geometric objects in the Cartesian plane) | + | |
- | * [[http://scitools.org.uk/cartopy/|cartopy]]: a library providing cartographic tools for python | + | |
- | * [[https://rpy2.readthedocs.io/en/version_2.8.x/|rpy2]]: providing simple and robust access to R from within Python | + | |
- | + | ||
- | ==== Environments summary ==== | + | |
- | + | ||
- | After following the steps above, we get the following environments: | + | |
- | * uvcdat-2.8.0 | + | |
- | * cdatm14 | + | |
/* standard page footer */ | /* standard page footer */ |