Differences

This shows you the differences between two versions of the page.

--- other:uvcdat:cdat_conda:cdat_8_0_py2 [2019/03/01 16:22]
jypeter Added a small 'nightly' version installation section
+++ other:uvcdat:cdat_conda:cdat_8_0_py2 [2019/03/04 17:23]
jypeter
@@ Line 258: / Line 258: @@
 === Python 2.7 version ===
-<code>$ conda create -n cdat-nightly_py2 -c cdat/label/nightly-c conda-forge -c cdat python=2.7 cdat</code>
+<code>$ conda create -n cdat-nightly_py2 -c cdat/label/nightly -c conda-forge -c cdat python=2.7 cdat</code>
+<wrap hi>Not tested!</wrap> Can this be updated with ''conda update -n cdat-nightly_py2 -c cdat/label/nightly -c conda-forge -c cdat --all'' ???
 === Python 3.6 version ===
@@ Line 275: / Line 277: @@
 Note: using hard links, cloning a full environment //only// adds an extra 582M of disk space
-<code>$ du -sh /home/share/unix_files/cdat/miniconda3/envs/*
+<code>$ conda create -n cdatm17_py2 --clone cdat-8.0_py2
-.8G    /home/share/unix_files/cdat/miniconda3/envs/cdat-8.0_py2
-$ conda create -n cdatm17_py2 --clone cdat-8.0_py2
 Source:      /home/share/unix_files/cdat/miniconda3/envs/cdat-8.0_py2
 Destination: /home/share/unix_files/cdat/miniconda3/envs/cdatm17_py2
@@ Line 286: / Line 286: @@
 $ du -sh /home/share/unix_files/cdat/miniconda3/envs/*
-.8G    /home/share/unix_files/cdat/miniconda3/envs/cdat-8.0_py2
+.5G    /home/share/unix_files/cdat/miniconda3/envs/cdat-8.0_py2
 M    /home/share/unix_files/cdat/miniconda3/envs/cdatm17_py2
+M    /home/share/unix_files/cdat/miniconda3/envs/cdat-nightly_py2
 </code>
+==== cdat nightly case ====
+''conda create -n cdatm-nightly_py2 %%--%%clone cdat-nightly_py2''
 ===== Getting ready for a moving default CDAT =====
+<note warning>This step should probably be listed at the **end**, especially in a multi-user environment! If there is already a ''cdatm'' link, make sure that the new version is stable and working correctly before updating the ''cdatm'' link</note>
 We create a **//cdatm// symbolic link** in the ''envs'' directory, that has a //stable name// but can be moved to point to the latest default CDAT. In that case, most users can just activate this //cdatm// version and always get the latest stable version
@@ Line 334: / Line 341: @@
 After cloning, we are ready to install some extra packages that may be requested by LSCE users
+  * We first try to install together as many packages as possible that don't require other channels than //conda-forge//, and that don't request a downgrade of what is already installed
+  * We then install individual extra packages with ''conda install'' or ''pip install''
 <code># Keep a trace of what will be installed
-bash-4.1$ conda install --dry-run -n cdatm15 -c conda-forge -c uvcdat -c pcmdi pillow pandas statsmodels seaborn scikit-image seawater gsw netcdf4 pyferret basemap-data-hires pcmdi_metrics xlsxwriter cmor shapely cartopy rpy2 > /home/scratch01/jypeter/lsce-extra_01_install_170808.txt
+$ conda install --dry-run -n cdatm17_py2 -c conda-forge pillow pandas statsmodels seaborn scikit-image seawater gsw netcdf4 pyferret basemap-data-hires xlsxwriter cmocean rpy2 gdal windspharm  > /home/scratch01/jypeter/lsce-extra_01_install_190304.txt
 # Install...
-bash-4.1$ conda install -n cdatm15 -c conda-forge -c uvcdat -c pcmdi pillow pandas statsmodels seaborn scikit-image seawater gsw netcdf4 pyferret basemap-data-hires pcmdi_metrics xlsxwriter cmor shapely cartopy rpy2
+$ conda install -n cdatm17_py2 -c conda-forge pillow pandas statsmodels seaborn scikit-image seawater gsw netcdf4 pyferret basemap-data-hires xlsxwriter cmocean rpy2 gdal windspharm
 [...]
+# Install extra packages in the nightly version (if you need 'nightly')
+$ conda install -n cdatm-nightly_py2 -c conda-forge pillow pandas statsmodels seaborn scikit-image seawater gsw netcdf4 pyferret basemap-data-hires xlsxwriter cmocean rpy2 gdal windspharm
 # Check the disk space after installation
-bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2
+$ du -sh /home/share/unix_files/cdat/miniconda3
-G	/home/share/unix_files/cdat/miniconda2
+.5G    /home/share/unix_files/cdat/miniconda3
-bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2/envs/*
+$ du -sh /home/share/unix_files/cdat/miniconda3/envs/*
-	/home/share/unix_files/cdat/miniconda2/envs/cdatm
+.5G    /home/share/unix_files/cdat/miniconda3/envs/cdat-8.0_py2
-.8G	/home/share/unix_files/cdat/miniconda2/envs/cdatm14
+.2G    /home/share/unix_files/cdat/miniconda3/envs/cdatm17_py2
-.7G	/home/share/unix_files/cdat/miniconda2/envs/cdatm15
+M    /home/share/unix_files/cdat/miniconda3/envs/cdatm-nightly_py2
-M	/home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.10
+M    /home/share/unix_files/cdat/miniconda3/envs/cdat-nightly_py2</code>
-.0G	/home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0
-# Check the disk space again after cleaning
+List of installed packages: {{ :other:uvcdat:cdat_conda:lsce-extra_01_install_190304.txt |}}
-bash-4.1$ conda clean --all
-bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2
-.9G	/home/share/unix_files/cdat/miniconda2</code>
-List of installed packages: {{ :other:uvcdat:cdat_conda:lsce-extra_01_install_170808.txt |}}
-Disk space used after adding the extra packages, in <wrap em>in the  new directory hierarchy</wrap> (''cdat/miniconda2b'', on the obelix servers)
-<code>bash-4.2$ du -sh /home/share/unix_files/cdat/miniconda2b
-.4G	/home/share/unix_files/cdat/miniconda2b
-bash-4.2$ du -sh /home/share/unix_files/cdat/miniconda2b/envs/*
-	/home/share/unix_files/cdat/miniconda2b/envs/cdatm
-.8G	/home/share/unix_files/cdat/miniconda2b/envs/cdatm15
-M	/home/share/unix_files/cdat/miniconda2b/envs/uvcdat-2.10
-# Check the disk space again after cleaning
-bash-4.2$ conda clean --all
-bash-4.2$ du -sh /home/share/unix_files/cdat/miniconda2b
-.4G	/home/share/unix_files/cdat/miniconda2b</code>
 Packages installed with //pip//
@@ Line 382: / Line 372: @@
     * Update with: ''pip install %%--%%upgrade dreqPy''
       * Get version number with:\\ <code>$ drq -v
-dreqPy version 01.00.06 [Version 01.00.06]</code>
+dreqPy version 01.00.29 [Version 01.00.29]</code>
-  * [[https://github.com/pierre-rouanet/dtw|dtw]]: DTW (Dynamic Time Warping) python module
-    * ''pip install dtw''
-    * Requested by Sentia Goursaud
 The following packages have no dependency problems and were installed (or updated) //later//
-  * [[http://www.earthsystemmodeling.org/esmf_releases/non_public/ESMF_7_0_0/esmpy_doc/html/index.html|ESMPy]]: ESMF Python Regridding Interface
-    * We used the following to get a more up-to-date version of esmpy than the one bundled with 2.10
-    * <code>bash-4.1$ conda list | grep -i esm
-esmf                      7.0.0                         6    conda-forge
-esmpy                     7.0.0                    py27_1    conda-forge
-bash-4.1$ conda install -n cdatm15 -c nesii/channel/dev-esmf -c conda-forge esmpy=7.1.0.dev32
+  * [[http://www.ifremer.fr/vacumm/|vacumm]]: Validation, Analysis, Comparison - Utilities written in Python to validate and analyze Multi-Model outputs, and compare them to observations
-[...]
+    * ''conda install -n cdatm17_py2 -c conda-forge -c vacumm vacumm''
-bash-4.1$ conda list | grep -i esm
-esmf                      7.1.0.dev32                   1    nesii/channel/dev-esmf
-esmpy                     7.1.0.dev32              py27_1    nesii/channel/dev-esmf</code>
-  * [[http://matplotlib.org/cmocean/|cmocean]]: beautiful colormaps for oceanography
-    * ''conda install -n cdatm15 -c conda-forge cmocean''
-  * [[http://scitools.org.uk/iris/index.html|iris]]: A Python library for Meteorology and Climatology
-    * ''conda install -n cdatm15 -c conda-forge -c scitools iris''
-  * [[http://www.gdal.org/|OSGeo/GDAL]]: Geospatial Data Abstraction Library. GDAL is a translator library for raster and vector geospatial data formats
-    * ''conda install -c conda-forge gdal''
-    * [[https://pcjericks.github.io/py-gdalogr-cookbook/|Python GDAL/OGR Cookbook]]
   * [[https://github.com/stefraynaud/spanlib|spanlib]]: Spectral Analysis Library
-    * ''conda install -n cdatm15 -c stefraynaud -c conda-forge spanlib''
+    * ''conda install -n cdatm17_py2 -c stefraynaud -c conda-forge spanlib''
     * Test: ''python -c 'from spanlib.analyzer import Analyzer' ''
+==== TODO ====
+//Add here packages that would be useful and have not been installed yet, or have some problems that prevent their installation//
+  * [[http://scitools.org.uk/iris/index.html|iris]]: A Python library for Meteorology and Climatology
+    * ''conda install -n cdatxxx -c conda-forge iris''
   * [[http://wrf-python.readthedocs.io/en/latest/|wrf-python]]: A collection of diagnostic and interpolation routines for use with output from the Weather Research and Forecasting (WRF-ARW) Model
     * ''conda install -n cdatm15 -c conda-forge wrf-python''
   * [[https://glances.readthedocs.io/en/stable/index.html|glances]]: a cross-platform monitoring tool (similar to ''top'')
     * ''conda install -n cdatm15 -c conda-forge glances''
     * [[https://github.com/nicolargo/glances|glances@github]]
-  * [[https://ajdawson.github.io/windspharm/latest/|windspharm]]: spherical harmonic wind analysis in Python
-    * ''conda install -n cdatm15 -c conda-forge windspharm''
-    * [[https://github.com/ajdawson/windspharm|windspharm@github]]
-==== TODO ====
-//Add here packages that would be useful but have some problems that prevent their installation//
-  * [[http://www.ifremer.fr/vacumm/|vacumm]]: Validation, Analysis, Comparison - Utilities written in Python to validate and analyze Multi-Model outputs, and compare them to observations
-    * As of August 08 2017, installing vacumm in 2.10 requires downgrading to 2.10 (see [[https://github.com/VACUMM/vacumm/issues/9|github issue]])!
-    * <wrap hi>Problem solved</wrap> with **vacumm 3.4.1-1** on August 31st! Installation was OK with\\ ''conda install -n cdatm15 -c conda-forge -c vacumm vacumm''
-  * Install PySPLIT for Sentia\\ [[https://github.com/mscross/pysplit|PySPLIT]] A package for generating HYSPLIT air parcel trajectories trajectories, performing moisture uptake analyses, expediting HYSPLIT cluster analysis, and for visualizing trajectories, clusters, and along-trajectory meteorological data\\ Works with [[https://ready.arl.noaa.gov/HYSPLIT.php|HYSPLiT]]
-    - ''conda install -n cdatm15 -c conda-forge fiona geopandas''
-    - ''pip install pysplit''
 ==== Other packages ====
@@ Line 455: / Line 420: @@
 We check if some of the installed files are missing read access for the //group// or //other//, and we manually change the permissions
-<code>(cdatm15) bash-4.2$ find cdatm15 \! -perm /g+r,o+r -ls
+<code> >find /home/share/unix_files/cdat/miniconda3/envs \! -perm /g+r,o+r -ls
-50333969    4 -rw-------   2 jypeter  lsce          234 Apr  4 20:35 cdatm15/lib/python2.7/site-packages/cdp-1.0.3-py2.7.egg-info/PKG-INFO
+# Everything OK!</code>
-50333971    4 -rw-------   2 jypeter  lsce          291 Apr  4 20:35 cdatm15/lib/python2.7/site-packages/cdp-1.0.3-py2.7.egg-info/SOURCES.txt
-50333962    4 -rw-------   2 jypeter  lsce            4 Apr  4 20:35 cdatm15/lib/python2.7/site-packages/cdp-1.0.3-py2.7.egg-info/top_level.txt
-50333961    4 -rw-------   2 jypeter  lsce            1 Apr  4 20:35 cdatm15/lib/python2.7/site-packages/cdp-1.0.3-py2.7.egg-info/dependency_links.txt
-50333960    4 -rw-------   2 jypeter  lsce            1 Apr  4 20:35 cdatm15/lib/python2.7/site-packages/cdp-1.0.3-py2.7.egg-info/not-zip-safe
-50333987    8 -rw-------   2 jypeter  lsce         5149 Apr  4 20:35 cdatm15/lib/python2.7/site-packages/cdp/cdp_viewer.pyc
-50333986    8 -rw-------   2 jypeter  lsce         4208 Apr  4 20:35 cdatm15/lib/python2.7/site-packages/cdp/cdp_metric.py
-50333966    4 -rw-------   2 jypeter  lsce          150 Apr  4 20:35 cdatm15/lib/python2.7/site-packages/cdp/__init__.pyc
-50333968    4 -rw-------   2 jypeter  lsce          201 Apr  4 20:35 cdatm15/lib/python2.7/site-packages/cdp/cdp_provenance.py
-50333988    8 -rw-------   2 jypeter  lsce         5365 Apr  4 20:35 cdatm15/lib/python2.7/site-packages/cdp/cdp_metric.pyc
-50333982    4 -rw-------   2 jypeter  lsce         1821 Apr  4 20:35 cdatm15/lib/python2.7/site-packages/cdp/cdp_parser.py
-50333981    4 -rw-------   2 jypeter  lsce         1773 Apr  4 20:35 cdatm15/lib/python2.7/site-packages/cdp/cdp_parameter.py
-50333973    4 -rw-------   2 jypeter  lsce          670 Apr  4 20:35 cdatm15/lib/python2.7/site-packages/cdp/cdp_output.py
-50333980    4 -rw-------   2 jypeter  lsce         1642 Apr  4 20:35 cdatm15/lib/python2.7/site-packages/cdp/cdp_output.pyc
-50333979    4 -rw-------   2 jypeter  lsce         1474 Apr  4 20:35 cdatm15/lib/python2.7/site-packages/cdp/_cache.pyc
-50333967    4 -rw-------   2 jypeter  lsce          187 Apr  4 20:35 cdatm15/lib/python2.7/site-packages/cdp/cdp_tool.py
-50333989    0 -rw-------   2 jypeter  lsce            0 Apr  4 20:35 cdatm15/lib/python2.7/site-packages/cdp/__init__.py
-50333985    4 -rw-------   2 jypeter  lsce         3843 Apr  4 20:35 cdatm15/lib/python2.7/site-packages/cdp/cdp_viewer.py
-50333984    4 -rw-------   2 jypeter  lsce         2436 Apr  4 20:35 cdatm15/lib/python2.7/site-packages/cdp/cdp_parameter.pyc
-50333970    4 -rw-------   2 jypeter  lsce          288 Apr  4 20:35 cdatm15/lib/python2.7/site-packages/cdp/cdp_io.py
-50333974    4 -rw-------   2 jypeter  lsce          692 Apr  4 20:35 cdatm15/lib/python2.7/site-packages/cdp/cdp_tool.pyc
-50333978    4 -rw-------   2 jypeter  lsce          878 Apr  4 20:35 cdatm15/lib/python2.7/site-packages/cdp/cdp_io.pyc
-50333976    4 -rw-------   2 jypeter  lsce          734 Apr  4 20:35 cdatm15/lib/python2.7/site-packages/cdp/_cache.py
-50333983    4 -rw-------   2 jypeter  lsce         2269 Apr  4 20:35 cdatm15/lib/python2.7/site-packages/cdp/cdp_parser.pyc
-50333975    4 -rw-------   2 jypeter  lsce          724 Apr  4 20:35 cdatm15/lib/python2.7/site-packages/cdp/cdp_provenance.pyc
-(cdatm15) bash-4.2$ chmod -R a+r cdatm15/lib/python2.7/site-packages/cdp*
-(cdatm15) bash-4.2$ find cdatm15 \! -perm /g+r,o+r -ls</code>
@@ Line 508: / Line 447: @@
   * [[https://rpy2.readthedocs.io/en/version_2.8.x/|rpy2]]: providing simple and robust access to R from within Python
   * [[http://matplotlib.org/cmocean/|cmocean]]: beautiful colormaps for oceanography
-  * [[http://scitools.org.uk/iris/index.html|iris]]: A Python library for Meteorology and Climatology
   * [[http://www.gdal.org/|OSGeo/GDAL]]: Geospatial Data Abstraction Library. GDAL is a translator library for raster and vector geospatial data formats
+    * [[https://pcjericks.github.io/py-gdalogr-cookbook/|Python GDAL/OGR Cookbook]]
   * [[https://github.com/stefraynaud/spanlib|spanlib]]: Spectral Analysis Library
-  * [[http://wrf-python.readthedocs.io/en/latest/|wrf-python]]: A collection of diagnostic and interpolation routines for use with output from the Weather Research and Forecasting (WRF-ARW) Model
-  * [[https://glances.readthedocs.io/en/stable/index.html|glances]]: a cross-platform monitoring tool (similar to ''top'')
   * [[https://ajdawson.github.io/windspharm/latest/|windspharm]]: spherical harmonic wind analysis in Python
@@ Line 524: / Line 461: @@
 ^  Environment\\ name  ^  Server  ^  ''conda list''  ^
-|  uvcdat-2.10  |  LSCE  |  {{  |package list}}  |
+| cdat-8.0_py2 |  LSCE\\ ciclad  | |
-|  cdatm15  |  LSCE  |  {{ |package list}}  |
+| cdatm17_py2 |  LSCE\\ ciclad  | |
+| cdat-nightly_py2 |  LSCE\\ ciclad  | |
+| cdatm-nightly_py2 |  LSCE\\ ciclad  | |
 /* standard page footer */

PMIP3 wiki

User Tools

Site Tools

Differences

Page Tools