Differences

This shows you the differences between two versions of the page.

--- other:uvcdat:cdat_conda:cdat_2_8_0 [2017/03/03 11:35] – Added wget, and notes about CMOR jypeter
+++ other:uvcdat:cdat_conda:cdat_2_8_0 [2017/08/31 14:04] (current) – [Packages that have no dependency problems] Added a note about PMIP read permission problem jypeter
@@ Line 2: / Line 2: @@
 [ [[.:index|Back to all versions]] ]
+<WRAP center round tip 60%>
+Follow the instructions about the [[other:python:starting#conda-based_versions_of_uv-cdat|conda-based versions of UV-CDAT initialization]] for actually using an installed version of ''2.8.0''
+</WRAP>
+===== What's New? =====
+  * [[https://github.com/UV-CDAT/uvcdat/releases/tag/v2.8|2.8 announcement]] and features summary ([[https://github.com/UV-CDAT/uvcdat/releases|all versions]])
+  * Full [[https://uv-cdat.llnl.gov/changelog.html#2.8|Change log]]
 ===== Installation with Anaconda2 =====
 <WRAP center round alert 60%>
-[[#installation_with_miniconda2|Installing with Miniconda2]] rather than Anaconda2 will save disk space!
+[[#installation_with_miniconda2|Installing with Miniconda2]] rather than Anaconda2 is **much better** because it will save disk space!
 </WRAP>
@@ Line 23: / Line 33: @@
 ===== Installation with Miniconda2 =====
+<note important>\\ We assume that [[other:uvcdat:conda_notes#miniconda|Miniconda2]] is already installed</note>
 Note: after installation, we use **//only// 2.7G** (details below), compared to the 4.6G of the Anaconda2 installation
@@ Line 78: / Line 90: @@
 K     /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0/var
 M    /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0/vistrails</code>
+===== Downgrading conda =====
+Note: on June 1st 2017, we used the following step in order to **down**grade conda and be ready to install UV-CDAT 2.10 that required an older version of conda (strictly older than 4.3.13) than the one used for 2.8
+<code>bash-4.1$ conda install -n root "conda<4.3.13"
+Fetching package metadata .........
+Solving package specifications: .
+Package plan for installation in environment /home/share/unix_files/cdat/miniconda2:
+The following packages will be DOWNGRADED due to dependency conflicts:
+    conda: 4.3.13-py27_0 --> 4.3.11-py27_0
+Proceed ([y]/n)? y
+conda-4.3.11-p 100% |###############################################################| Time: 0:00:00   4.28 MB/s
+bash-4.1$ conda info
+Current conda install:
+               platform : linux-64
+          conda version : 4.3.11
+       conda is private : False
+      conda-env version : 4.3.11</code>
 ===== Cloning uvcdat to add specific packages for LSCE =====
+<WRAP center round tip 60%>
+This section is about the creation of the **cdatm14** environment
+Notes about [[https://wiki.lsce.ipsl.fr/pmip3/doku.php/other:python:starting#conda-based_versions_of_uv-cdat|actually using the cdatm14 conda-based python]]
+</WRAP>
 Note: using hard links, cloning a full environment only adds an extra 100M of disk space
@@ Line 98: / Line 143: @@
 .0G    /home/share/unix_files/cdat/miniconda2/envs/cdatm14
 M     /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0</code>
+==== Downloading cdms2/vcs test data ====
+You should download the test data and use it in the example scripts that you want to distribute, and scripts you write for reporting the errors you find (if any...)
+<code>(cdatm14) bash-4.1$ python -c 'import vcs; vcs.download_sample_data_files(); print "\nFinished downloading sample data to", vcs.sample_data'
+Finished downloading sample data to /home/share/unix_files/cdat/miniconda2/envs/cdatm14/share/uvcdat/sample_data
+(cdatm14) bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2/envs/cdatm14/share/uvcdat/sample_data
+M    /home/share/unix_files/cdat/miniconda2/envs/cdatm14/share/uvcdat/sample_data</code>
 ==== Packages that have no dependency problems ====
@@ Line 132: / Line 189: @@
   * High resolution data for ''basemap''
     * ''conda install -n cdatm14 -c conda-forge basemap-data-hires''
-  * wget
+  * [[https://github.com/PCMDI/pcmdi_metrics|PCMDI metrics package]] (PMP)
-    * ''conda install -n cdatm14 wget'' gets version **2.2**
+    * ''conda install -n cdatm14 -c conda-forge -c pcmdi pcmdi_metrics''
-    * Used ''conda install -n cdatm14 -c daler wget=3.2'' in order to get version **3.2**, the same as the one on Pypi
+    * Note: There was a [[https://github.com/PCMDI/pcmdi_metrics/issues/496|read permission problem on the files]] - <wrap em>Solved in 2.10</wrap>
+  * [[https://xlsxwriter.readthedocs.io/|XlsxWriter]]: a Python module for creating Excel XLSX files
+    * ''conda install -n cdatm14 xlsxwriter''
+    * Note: <wrap hi>this is a dependency of ''dreqPy''</wrap>
+  * [[https://earthsystemcog.org/projects/wip/CMIP6DataRequest|dreqPy]]: CMIP6 Data Request Python API
+    * ''pip install dreqPy''
+    * Update with: ''pip install %%--%%upgrade dreqPy''
+      * Get version number with:\\ <code>$ drq -v
+dreqPy version 01.00.06 [Version 01.00.06]</code>
+  * [[https://cmor.llnl.gov/|CMOR]]: CMOR (//Climate Model Output Rewriter//) is used to produce CF-compliant netCDF files
+    * ''conda install -n cdatm14 -c conda-forge -c pcmdi -c uvcdat cmor''
+      * Did not complete the installation yet, because it requested a downgrade of libnetcdf (see [[https://github.com/PCMDI/cmor/issues/138|github issue]])
+      * Installation issue was solved in ''cmor-3.2.3''
+  * [[https://github.com/pierre-rouanet/dtw|dtw]]: DTW (Dynamic Time Warping) python module
+    * ''pip install dtw''
+    * Requested by Sentia Goursaud
 ==== TODO ====
+//Add here packages that would be useful but have some problems that prevent their installation//
-  * CMOR
+  * [[https://earthsystemcog.org/projects/esmpy/|ESMPy]]: ESMF Python Regridding Interface - <wrap hi>Works in [[.:cdat_2_10|2.10]]!</wrap>
-    * ''conda install -n cdatm14 -c conda-forge -c pcmdi -c uvcdat cmor''
+    * ''conda install -n cdatm14 -c conda-forge esmpy''
-      * Did not complete the installation yet, because it requested a downgrade of libnetcdf (see [[https://github.com/PCMDI/cmor/issues/138|github issue]])
+    * Installation of esmpy ''7.0.0'' did not report errors, **but** esmpy seemed to have libraries based importation problems and has been removed
+      * ''conda -n cdatm14 remove esmpy''
+  * [[http://www.gdal.org/|GDAL]]: Geospatial Data Abstraction Library - <wrap hi>Works in [[.:cdat_2_10|2.10]]!</wrap>
+    * Cannot be installed due to too many dependency problems, and because it requires the downgrade of geos and rpy2
 ==== Other packages ====
@@ Line 184: / Line 260: @@
   * trying to install ''rpy2'' will trigger the installation of ''R''!
     * We force the **installation of only rpy2** in order to use the ''R'' already installed at LSCE
-      * ''conda install --force -n cdatm14 -c conda-forge -c uvcdat rpy2''
+      * ''conda install %%--%%force -n cdatm14 -c conda-forge -c uvcdat rpy2''
 ===== Extra packages list =====
@@ Line 199: / Line 275: @@
   * [[http://www.ifremer.fr/vacumm/|vacumm]]: Validation, Analysis, Comparison - Utilities written in Python to validate and analyze Multi-Model outputs, and compare them to observations
   * [[https://github.com/conda-forge/basemap-data-hires-feedstock|basemap-data-hires]]: high resolution data for ''basemap''
+  * [[https://github.com/PCMDI/pcmdi_metrics|PCMDI metrics package]] (PMP): objectively compare results from climate models with observations using well-established statistical tests
+  * [[https://xlsxwriter.readthedocs.io/|XlsxWriter]]: a Python module for creating Excel XLSX files
+  * [[https://earthsystemcog.org/projects/wip/CMIP6DataRequest|dreqPy]]: CMIP6 Data Request Python API
+  * [[https://cmor.llnl.gov/|CMOR]]:  CMOR (//Climate Model Output Rewriter//) is used to produce CF-compliant netCDF files
+  * [[https://github.com/pierre-rouanet/dtw|dtw]]: DTW (Dynamic Time Warping) python module
   * [[http://toblerity.org/shapely/project.html|shapely]]: a Python wrapper for GEOS for algebraic manipulation of geometry (manipulation and analysis of geometric objects in the Cartesian plane)
   * [[http://scitools.org.uk/cartopy/|cartopy]]: a library providing cartographic tools for python
   * [[https://rpy2.readthedocs.io/en/version_2.8.x/|rpy2]]: providing simple and robust access to R from within Python
+==== Removed packages ====
   * [[https://pypi.python.org/pypi/wget|wget]]: pure python download utility
-    * Had to install from an alternate repo in order to get the latest version
+    * Used ''conda install -n cdatm14 -c daler wget=3.2'' in order to get version **3.2**, the same as the one on Pypi
+    * **REMOVED** because it did not work correctly in the script where it was needed: it emulated wget with //ftp// and failed in our case after downloading the first file
 ===== Environments summary =====
-After following the steps above, we get the following environments:
+After following the steps above, we get the following environments. Use the ''conda info --envs'' command to get the up-to-date list of available environments
-  * uvcdat-2.8.0
-  * cdatm14
+^  Environment\\ name  ^  Server  ^  ''conda list''  ^
+|  uvcdat-2.8.0  |  LSCE\\ ciclad  |  {{ :other:uvcdat:cdat_conda:conda_list_lsce_uvcdat-2.8.0_170310.txt |package list}}  |
+|  cdatm14  |  LSCE\\ ciclad  |  {{ :other:uvcdat:cdat_conda:conda_list_lsce_cdatm14_170310.txt|package list}}  |
 /* standard page footer */