User Tools

Site Tools


other:uvcdat:cdat_conda:index

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
other:uvcdat:cdat_conda:index [2017/02/24 14:57]
jypeter
other:uvcdat:cdat_conda:index [2024/03/22 09:56] (current)
jypeter [Installed versions] Added link to the 2024-03 version
Line 1: Line 1:
-====== ​UV-CDAT conda-based ​versions installed by JYP ======+====== CDAT versions installed by JYP ======
  
 <WRAP center round important 60%> <WRAP center round important 60%>
-This page describes the conda-based versions of UV-CDAT, 2.8.0 and later installed by JYP+This page describes the **conda-based** versions of CDAT, ''​2.8.0'' ​and laterinstalled by JYP
  
-For versions up to 2.1, check the [[.:installed|UV-CDAT versions installed by JYP]] +Follow ​the [[other:python:starting#​ultra_quick-start_on_the_lsce_servers|Ultra quick-start on the LSCE servers]] instructions for actually ​**using CDAT at LSCE**
-</​WRAP>​ +
- +
-===== Useful links ===== +
- +
-  * [[http://​uvcdat.llnl.gov/​|Official CDAT site]] +
-  * [[https://​github.com/​UV-CDAT/​uvcdat/​wiki/​|CDAT wiki]] +
-  * [[https://​github.com/​UV-CDAT|CDAT @ github]] +
-    ​[[https://​github.com/​UV-CDAT/​cdms|cdms]] +
-    ​[[https://​github.com/​UV-CDAT/​cdtime|cdtime]] +
-    ​[[https://​github.com/​UV-CDAT/​vcs|vcs]] +
-    ​[[https://​github.com/​UV-CDAT/​cdutil|cdutil]] +
-    * [[https://​github.com/​UV-CDAT/​genutil|genutil]] +
- +
-===== 2.8.0 =====+
  
-==== Installation ​with Anaconda2 ====+For technical installation details, check [[..:​conda_notes|Installing and maintaining CDAT with conda]]
  
-<WRAP center round alert 60%> +For versions up to 2.1, check the (deprecated) ​[[..:​installed|CDAT versions installed by JYP]] page
-[[#​installation_with_miniconda2|Installing with Miniconda2]] rather than Anaconda2 will save disk space!+
 </​WRAP>​ </​WRAP>​
  
-<​code>#​ Keep a trace of what will be installed, with/​without conda-forge +===== Installed versions =====
-conda create --dry-run -n uvcdat-2.8.0 -c uvcdat uvcdat=2.8.0 > /​home/​scratch01/​jypeter/​uvcdat-2.8.0_install_dry_no-forge_160220.txt +
-conda create --dry-run -n uvcdat-2.8.0 -c conda-forge -c uvcdat uvcdat=2.8.0 > /​home/​scratch01/​jypeter/​uvcdat-2.8.0_install_dry_with-forge_160220.txt+
  
-# Do the actual installation +Listed with the most recent version at the top
-conda create -n uvcdat-2.8.0 -c conda-forge -c uvcdat uvcdat=2.8.0</​code>​+
  
-Lists of installed packages: {{ :​other:​uvcdat:​uvcdat-2.8.0_install_dry_with-forge_160220.txt |with conda-forge}} or {{ :​other:​uvcdat:​uvcdat-2.8.0_install_dry_no-forge_160220.txt |without conda-forge}}+==== Dated (but not outdated!) versions ====
  
-Size of ''​/​home/​share/​unix_files/​cdat/​anaconda2''​+These versions are based on ''​xcdat'' ​and lots of other useful packages, and the latest Python version available at the date of installation
-  * before installation:​ **2.1G** +
-  * after installation: **4.6G**+
  
-==== Installation with Miniconda2 ====+  * [[other:​uvcdat:​cdat_conda:​cdat_2024_03|2024-03]]
  
-Note: after installation,​ we use **//only// 2.7G** (details below), compared to the 4.6G of the Anaconda2 installation 
  
-<​code>#​ Keep a trace of what will be installed (with conda-forge) +==== CDAT based versions ====
-conda create --dry-run -n uvcdat-2.8.0 -c conda-forge -c uvcdat uvcdat=2.8.0 > /​home/​scratch01/​jypeter/​uvcdat-2.8.0_install_dry_with-forge_160220.txt+
  
-# Do the actual installation +These versions are based on **CDAT** packages (''​cdms2''​ and friends) that are unfortunately not compatible with //recent// versions of numpy
-conda create -n uvcdat-2.8.0 -c conda-forge -c uvcdat uvcdat=2.8.0<​/code>+
  
-Lists of installed packages{{ :​other:​uvcdat:​uvcdat-2.8.0_install_dry_with-forge_160221.txt |List of installed packages}}+  * [[.:cdat_8_2_1|8.2.1]]
  
-<​code>​bash-4.1$ conda info --envs +  * [[.:cdat_8_1|8.1]]
-# conda environments: +
-+
-uvcdat-2.8.0             /​home/​share/​unix_files/​cdat/​miniconda2/​envs/​uvcdat-2.8.0 +
-root                  *  /​home/​share/​unix_files/​cdat/​miniconda2+
  
-bash-4.1$ du -sh /​home/​share/​unix_files/​cdat/​miniconda2 +  * [[.::​cdat_8_0_py2|8.0]]
-2.7G    /​home/​share/​unix_files/​cdat/​miniconda2+
  
-bash-4.1$ du -sh /​home/​share/​unix_files/​cdat/​miniconda2/​envs/​* +  ​[[.:cdat_2_12|2.12]]
-2.0G    /​home/​share/​unix_files/​cdat/​miniconda2/​envs/​uvcdat-2.8.0+
  
-bash-4.1$ du -sh /​home/​share/​unix_files/​cdat/​miniconda2/​pkgs +  * [[.:cdat_2_10|2.10]]
-2.6G    /​home/​share/​unix_files/​cdat/​miniconda2/​pkgs+
  
-bash-4.1$ du -sh /​home/​share/​unix_files/​cdat/​miniconda2/​* +  ​[[.:​cdat_2_8_0|2.8.0]]
-4.4M    /​home/​share/​unix_files/​cdat/​miniconda2/​bin +
-4.0K    /​home/​share/​unix_files/​cdat/​miniconda2/​conda-bld +
-276K    /​home/​share/​unix_files/​cdat/​miniconda2/​conda-meta +
-2.0G    /​home/​share/​unix_files/​cdat/​miniconda2/​envs +
-20K     /​home/​share/​unix_files/​cdat/​miniconda2/​etc +
-3.8M    /​home/​share/​unix_files/​cdat/​miniconda2/​include +
-99M     /​home/​share/​unix_files/​cdat/​miniconda2/​lib +
-4.0K    /​home/​share/​unix_files/​cdat/​miniconda2/​LICENSE.txt +
-665M    /​home/​share/​unix_files/​cdat/​miniconda2/​pkgs +
-16K     /​home/​share/​unix_files/​cdat/​miniconda2/​share +
-8.0K    /​home/​share/​unix_files/​cdat/​miniconda2/​ssl+
  
-bash-4.1$ du -sh /​home/​share/​unix_files/​cdat/​miniconda2/​envs/​uvcdat-2.8.0/​* 
-56M     /​home/​share/​unix_files/​cdat/​miniconda2/​envs/​uvcdat-2.8.0/​bin 
-1.9M    /​home/​share/​unix_files/​cdat/​miniconda2/​envs/​uvcdat-2.8.0/​conda-meta 
-48K     /​home/​share/​unix_files/​cdat/​miniconda2/​envs/​uvcdat-2.8.0/​etc 
-420K    /​home/​share/​unix_files/​cdat/​miniconda2/​envs/​uvcdat-2.8.0/​imports 
-67M     /​home/​share/​unix_files/​cdat/​miniconda2/​envs/​uvcdat-2.8.0/​include 
-1.7G    /​home/​share/​unix_files/​cdat/​miniconda2/​envs/​uvcdat-2.8.0/​lib 
-24K     /​home/​share/​unix_files/​cdat/​miniconda2/​envs/​uvcdat-2.8.0/​man 
-3.1M    /​home/​share/​unix_files/​cdat/​miniconda2/​envs/​uvcdat-2.8.0/​mkspecs 
-16M     /​home/​share/​unix_files/​cdat/​miniconda2/​envs/​uvcdat-2.8.0/​mod 
-3.5M    /​home/​share/​unix_files/​cdat/​miniconda2/​envs/​uvcdat-2.8.0/​plugins 
-57M     /​home/​share/​unix_files/​cdat/​miniconda2/​envs/​uvcdat-2.8.0/​share 
-4.3M    /​home/​share/​unix_files/​cdat/​miniconda2/​envs/​uvcdat-2.8.0/​ssl 
-2.7M    /​home/​share/​unix_files/​cdat/​miniconda2/​envs/​uvcdat-2.8.0/​tests 
-12K     /​home/​share/​unix_files/​cdat/​miniconda2/​envs/​uvcdat-2.8.0/​var 
-107M    /​home/​share/​unix_files/​cdat/​miniconda2/​envs/​uvcdat-2.8.0/​vistrails</​code>​ 
  
-==== Cloning uvcdat to add specific packages for LSCE ====+===== Useful links =====
  
-Note: using hard links, cloning a full environment only adds an extra 100M of disk space +  ​* [[http://uvcdat.llnl.gov/|Official CDAT site]] 
- +  * [[https://​github.com/​UV-CDAT/uvcdat/wiki/|CDAT wiki]] 
-<​code>​bash-4.1$ du -sh /​home/​share/​unix_files/​cdat/​miniconda2 +  * [[https://​github.com/​UV-CDAT|CDAT @ github]] 
-2.7G    /​home/​share/​unix_files/​cdat/​miniconda2 +    * [[https://github.com/UV-CDAT/cdms|cdms]] 
- +    * [[https://github.com/UV-CDAT/cdtime|cdtime]] 
-bash-4.1$ du -sh /​home/​share/​unix_files/​cdat/​miniconda2/​envs/​* +    * [[https://​github.com/UV-CDAT/vcs|vcs]] 
-2.0G    /​home/​share/​unix_files/​cdat/​miniconda2/​envs/​uvcdat-2.8.0 +    * [[https://​github.com/​UV-CDAT/cdutil|cdutil]] 
- +    * [[https://github.com/UV-CDAT/genutil|genutil]]
-bash-4.1$ conda create -n cdatm14 --clone uvcdat-2.8.0 +
-[...] +
- +
-bash-4.1$ du -sh /home/​share/​unix_files/​cdat/​miniconda2 +
-2.8G    /​home/​share/​unix_files/​cdat/​miniconda2 +
- +
-bash-4.1$ du -sh /​home/​share/​unix_files/​cdat/​miniconda2/​envs/​* +
-2.0G    /​home/​share/​unix_files/​cdat/​miniconda2/​envs/​cdatm14 +
-82M     /​home/​share/​unix_files/​cdat/​miniconda2/​envs/uvcdat-2.8.0</code> +
- +
-=== Packages that have no dependency problems === +
- +
-After cloning, we are ready to install some extra packages that may be requested by LSCE users +
- +
-<​code>#​ Keep a trace of what will be installed +
-bash-4.1$ conda install --dry-run -n cdatm14 -c conda-forge -c uvcdat -c vacumm pillow pandas statsmodels seaborn scikit-image seawater gsw netcdf4 pyferret vacumm > /​home/​scratch01/​jypeter/​lsce-extra_01_install_160223.txt +
- +
-# Install... +
-bash-4.1$ conda install -n cdatm14 -c conda-forge -c uvcdat -c vacumm pillow pandas statsmodels seaborn scikit-image seawater gsw netcdf4 pyferret vacumm +
- +
-# Check the disk space after installation +
-bash-4.1$ du -sh /​home/​share/​unix_files/​cdat/​miniconda2 +
-3.6G    /​home/​share/​unix_files/​cdat/​miniconda2 +
- +
-bash-4.1$ du -sh /​home/​share/​unix_files/​cdat/​miniconda2/​envs/​* +
-2.4G    /​home/​share/​unix_files/​cdat/​miniconda2/​envs/​cdatm14 +
-82M     /​home/​share/​unix_files/​cdat/​miniconda2/​envs/​uvcdat-2.8.0 +
- +
-# Check the disk space again after cleaning +
-bash-4.1$ conda clean all +
- +
-bash-4.1$ du -sh /​home/​share/​unix_files/​cdat/​miniconda2 +
-2.9G    /​home/​share/​unix_files/​cdat/​miniconda2 +
- +
-bash-4.1$ du -sh /​home/​share/​unix_files/​cdat/​miniconda2/​envs/​* +
-2.4G    /​home/​share/​unix_files/​cdat/​miniconda2/​envs/​cdatm14 +
-82M     /​home/​share/​unix_files/​cdat/​miniconda2/​envs/​uvcdat-2.8.0</​code>​ +
- +
-Lists of installed packages: {{ :​other:​uvcdat:​lsce-extra_01_install_160223.txt ​|}} +
- +
-The following packages have no dependency problems and were installed //later// +
-  * High resolution data for ''​basemap''​ +
-    * ''​conda install -n cdatm14 -c conda-forge basemap-data-hires''​ +
- +
-=== Other packages === +
- +
-<note warning>​There is no warranty that the packages listed below will work correctly, when it was required to bypass the compatibility checks in order to install them...</​note>​ +
- +
-  * ''​shapely''​ request a downgrade of ''​geos''​ when installed from //​conda-forge//​ (see [[https://​github.com/​Toblerity/Shapely/issues/472|github issue]]) but it will work when installed from the default repo +
-    * ''​conda install -n cdatm14 shapely''​ +
-    * We get a slightly older version (//​1.5.**16**-py27_0//​ instead of //​1.5.**17**-np111py27_2//​) +
-    * Note: ''​cartopy''​ (below) requests the same downgrade of ''​geos''​... +
-  * ''​cartopy''​ requests a downgrade of ''​geos''​ and ''​matplotlib''​ (see [[https://​github.com/​SciTools/​cartopy/​issues/​853#​issuecomment-282023158|github ​issue]]) +
-    ​* We install the requested dependencies and then we **force** the installation of ''​cartopy''​\\ <​code>​(uvcdat-2.8.0) bash-4.1$ conda install -n cdatm14 -c conda-forge -c uvcdat libxslt lxml owslib pyepsg +
-[...] +
-The following NEW packages will be INSTALLED:​ +
- +
-    libxslt: 1.1.29-3 ​     conda-forge +
-    lxml:    3.7.2-py27_1 ​ conda-forge +
-    owslib: ​ 0.14.0-py27_0 conda-forge +
-    pyepsg: ​ 0.3.1-py27_0 ​ conda-forge +
-[...] +
- +
-(uvcdat-2.8.0) bash-4.1$ conda install -n cdatm14 -c conda-forge cartopy +
-[...] +
-The following NEW packages will be INSTALLED:​ +
- +
-    cartopy: ​   0.15.0-np111py27_0 conda-forge +
- +
-The following packages will be DOWNGRADED due to dependency conflicts:​ +
- +
-    geos:       ​3.6.0-0 ​           conda-forge --> 3.5.1-1 ​          ​conda-forge +
-    matplotlib: 2.0.0-np111py27_2 ​ conda-forge --> 1.5.3-np111py27_7 conda-forge +
- +
-Proceed ([y]/n)? n +
- +
-(uvcdat-2.8.0) bash-4.1$ conda install --force -n cdatm14 -c conda-forge cartopy +
-[...] +
-The following NEW packages will be INSTALLED:​ +
- +
-    cartopy: 0.15.0-np111py27_0 conda-forge +
- +
-Proceed ([y]/n)? y</​code>​ +
-  * trying to install ''​rpy2''​ will trigger the installation of ''​R''​! +
-    * We force the **installation of only rpy2** in order to use the ''​R''​ already installed at LSCE +
-      * ''​conda install --force -n cdatm14 -c conda-forge -c uvcdat rpy2''​ +
- +
-=== Extra packages list === +
- +
-  ​* [[https://python-pillow.org/|pillow]]: the friendly PIL (//Python Imaging Library//) fork +
-  * [[http://​pandas.pydata.org/|pandas]]: Python Data Analysis Library +
-  * [[http://​statsmodels.sourceforge.net/​|statsmodels]]:​ a Python module that allows users to explore data, estimate statistical models, and perform statistical tests +
-  ​* [[https://seaborn.pydata.org/​|seaborn]]:​ statistical data visualization +
-  * [[http://scikit-image.org/|scikit-image]]: image processing in Python +
-  * [[http://​pythonhosted.org/​seawater/​|seawater]]:​ Python re-write of the CSIRO seawater toolbox +
-  ​* [[https://pypi.python.org/​pypi/​gsw/​|gsw]]:​ Python implementation of the Thermodynamic Equation Of Seawater +
-  * [[http://​unidata.github.io/netcdf4-python/|netcdf4]]: a Python interface to the netCDF C library +
-  * [[http://​ferret.pmel.noaa.gov/​Ferret/​documentation/​pyferret|pyferret]]:​ Ferret encapsulated in Python +
-  * [[http://​www.ifremer.fr/​vacumm/​|vacumm]]:​ Validation, Analysis, Comparison - Utilities written in Python to validate and analyze Multi-Model outputs, and compare them to observations +
-  ​* [[https://​github.com/​conda-forge/basemap-data-hires-feedstock|basemap-data-hires]]: high resolution data for ''​basemap''​ +
- +
-  * [[http://​toblerity.org/​shapely/​project.html|shapely]]:​ a Python wrapper for GEOS for algebraic manipulation of geometry (manipulation and analysis of geometric objects in the Cartesian plane) +
-  * [[http://​scitools.org.uk/​cartopy/​|cartopy]]:​ a library providing cartographic tools for python +
-  ​* [[https://rpy2.readthedocs.io/en/​version_2.8.x/|rpy2]]: providing simple and robust access to R from within Python+
  
 /* standard page footer */ /* standard page footer */
other/uvcdat/cdat_conda/index.1487948235.txt.gz · Last modified: 2017/02/24 14:57 by jypeter