User Tools

Site Tools


other:uvcdat:cdat_conda:cdat_8_1

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
other:uvcdat:cdat_conda:cdat_8_1 [2019/03/12 14:52]
jypeter [Packages that have no dependency problems]
other:uvcdat:cdat_conda:cdat_8_1 [2021/03/26 09:10]
jypeter [You want to replicate this installation on your own server or desktop/laptop]
Line 4: Line 4:
  
 <WRAP center round tip 60%> <WRAP center round tip 60%>
-Follow the instructions ​about the [[other:​python:​starting#​conda-based_versions_of_uv-cdat|conda-based versions of UV-CDAT initialization]] for actually using an installed version of ''​8.1'​'+If you have questions ​about Python, read: 
 +  * [[other:​python:​starting|Working with Python]] 
 +  * [[other:​python:​jyp_steps|JYP's recommended steps for learning python]]!
 </​WRAP>​ </​WRAP>​
  
-===== What's New? =====+===== Using JYP version ===== 
 + 
 +If you mostly want to use the CDAT 8.1 installed by JYP, just use the following steps and then read the //What should I do now?// section below 
 + 
 +  * Initialize //conda// with: 
 + 
 +^  Server ​ ^  tcsh  ^  bash  ^ 
 +|  LSCE  | ''​source ~jypeter/​.conda3_jyp.csh''​ | ''​source ~jypeter/​.conda3_jyp.sh''​ | 
 +|  ciclad ​ | ''​source ~jypmce/​.conda3_jyp.csh''​ | ''​source ~jypmce/​.conda3_jyp.sh''​ | 
 + 
 +  * Choose one of the [[#​environments_summary|installed environments]] and activate it with: ''​conda activate env_name''​ 
 +    * python 2.7.x: ''​conda activate cdatm_py2''​ 
 +    * python 3.x: ''​conda activate cdatm_py3''​ (not available on ciclad, but could be installed) 
 + 
 +  * Type ''​which python''​ (or the ''​wp''​ alias) and make sure you get something like\\ ''​[...]/​miniconda3/​envs/​env_name/​bin/​python''​ 
 + 
 + 
 +===== What should I do now? ===== 
 + 
 +==== You just want to use the installed version ==== 
 + 
 +You are ready if you have typed the ''​conda activate''​ command specified in the previous section! 
 +  * You may want to add the ''​source''​ line to your shell configuration file so that you don't have to remember it and type it each time you open a new terminal 
 +    * Note: it's better to type the ''​activate''​ command in a terminal only when you want to use this specific version of python, rather than having the ''​activate''​ in your shell configuration file. This may have some side effects in some cases 
 +  * Have a look at the [[other:​uvcdat:​cdat_conda:​cdat_8_1#​extra_packages_list|Extra package list]] at the bottom of this page to see what is available in this distribution 
 +  * If you have not done it yet, read (or at least have a look at the list of sections):​ 
 +    * [[other:​python:​starting|Working with Python]] 
 +    * [[other:​python:​jyp_steps|JYP'​s recommended steps for learning python]] 
 + 
 +==== You want to replicate this installation on your own server or desktop/​laptop ==== 
 + 
 +Read carefully the following sections of this page! 8-) 
 +===== What's New in CDAT 8.1? =====
  
   * [[https://​github.com/​cdat/​cdat/​releases/​tag/​v8.1|8.1 announcement]] and features summary ([[https://​github.com/​UV-CDAT/​uvcdat/​releases|all versions]])   * [[https://​github.com/​cdat/​cdat/​releases/​tag/​v8.1|8.1 announcement]] and features summary ([[https://​github.com/​UV-CDAT/​uvcdat/​releases|all versions]])
   * Full [[https://​uvcdat.llnl.gov/​changelog.html#​8.1|Change log]]   * Full [[https://​uvcdat.llnl.gov/​changelog.html#​8.1|Change log]]
 +
 +Note: this particular CDAT installation at LSCE (and on other machines/​servers) provides **many extra (non-CDAT) packages**. You can jump directly to the [[other:​uvcdat:​cdat_conda:​cdat_8_1#​extra_packages_list|Extra package list]] at the bottom of this page to see what is available
  
 ===== Installation with Miniconda3 ===== ===== Installation with Miniconda3 =====
Line 100: Line 136:
 ===== Customizing UV-CDAT for LSCE ===== ===== Customizing UV-CDAT for LSCE =====
  
 +==== Testing vcs ====
  
 +Start python and paste the following lines
 +
 +<​code>​import numpy as np, vcs
 +id100 = np.identity(100)
 +x = vcs.init()
 +x.plot(id100)
 +</​code>​
 +
 +If you get the kind of errors described in [[https://​github.com/​CDAT/​vcs/​issues/​393|x.plot crashes with OpenGL2 error]], the drivers on your server are probably not compatible with an interactive use of vcs... The workaround is to **install mesalib and work in //headless mode//** (e.g. without a display)...
 +
 +Install //mesalib// with ''​conda install -c cdat/​label/​v81 -c conda-forge mesalib vtk-cdat''​ and try pasting again the few code lines above in python. If things work as expected, no canvas will be opened, but you'll be able to save the canvas to a file with
 +
 +<​code>​x.png('​test_mesalib'​)
 +x.pdf('​test_mesalib'​)</​code>​
 +
 +Note: using //mesalib// for headless mode work makes it possible to work without an X server, and with no DISPLAY variable defined
 ==== Downloading cdms2/vcs test data ==== ==== Downloading cdms2/vcs test data ====
  
Line 127: Line 180:
  
 # Install... # Install...
-$ conda install -n cdatm18_py2 -c conda-forge pillow pandas statsmodels seaborn scikit-image seawater gsw netcdf4 pyferret basemap-data-hires xlsxwriter cmocean rpy2 gdal windspharm+$ conda install -n cdatm18_py2 -c conda-forge pillow pandas statsmodels seaborn scikit-image seawater gsw netcdf4 pyferret ​ferret_datasets ​basemap-data-hires xlsxwriter cmocean rpy2 gdal windspharm
 [...] [...]
  
Line 140: Line 193:
       * Get version number with:\\ <​code>​$ drq -v       * Get version number with:\\ <​code>​$ drq -v
 dreqPy version 01.00.29 [Version 01.00.29]</​code>​ dreqPy version 01.00.29 [Version 01.00.29]</​code>​
 +
 +  * [[https://​github.com/​PBrockmann/​ipython_ferretmagic|ipython_ferretmagic]]:​ IPython notebook extension for ferret
 +    * ''​conda activate cdatm18_py2''​\\ ''​pip install ferretmagic''​
  
 The following packages have no dependency problems and were installed (or updated) //later// The following packages have no dependency problems and were installed (or updated) //later//
 +
 +  * [[https://​cmor.llnl.gov/​|CMOR]]:​ CMOR (//Climate Model Output Rewriter//) is used to produce CF-compliant netCDF files
 +    * ''​conda install -n cdatm18_py2 -c conda-forge cmor''​
 +    * Get version number with: ''​python -c 'from cmor import *; print( (CMOR_VERSION_MAJOR,​ CMOR_VERSION_MINOR,​ CMOR_VERSION_PATCH) )' ''​
 +    * <wrap hi>​Warning!</​wrap>​ [[https://​github.com/​PCMDI/​cmor/​issues/​449|CMOR currently requires Python 2.7]]
  
   * [[http://​www.ifremer.fr/​vacumm/​|vacumm]]:​ Validation, Analysis, Comparison - Utilities written in Python to validate and analyze Multi-Model outputs, and compare them to observations   * [[http://​www.ifremer.fr/​vacumm/​|vacumm]]:​ Validation, Analysis, Comparison - Utilities written in Python to validate and analyze Multi-Model outputs, and compare them to observations
Line 150: Line 211:
     * ''​conda install -n cdatm18_py2 -c stefraynaud -c conda-forge spanlib''​     * ''​conda install -n cdatm18_py2 -c stefraynaud -c conda-forge spanlib''​
     * Test: ''​python -c 'from spanlib.analyzer import Analyzer'​ ''​     * Test: ''​python -c 'from spanlib.analyzer import Analyzer'​ ''​
 +
 +  * [[http://​scitools.org.uk/​cartopy/​|cartopy]]:​ a Python package designed for geospatial data processing in order to produce maps and other geospatial data analyses
 +    * ''​conda install -n cdatm18_py2 -c conda-forge cartopy''​
 +
 +  * [[https://​joblib.readthedocs.io/​en/​latest/​|joblib]]:​ running Python functions as pipeline jobs
 +    * ''​conda install -n cdatm18_py2 -c conda-forge joblib''​
  
   * [[https://​climaf.readthedocs.io/​en/​latest/​|CliMAF]]:​ a Climate Model Assessment Framework   * [[https://​climaf.readthedocs.io/​en/​latest/​|CliMAF]]:​ a Climate Model Assessment Framework
     * [[other:​uvcdat:​cdat_conda:​climaf_install|Installation notes]]     * [[other:​uvcdat:​cdat_conda:​climaf_install|Installation notes]]
 +
 +  * [[https://​jiffyclub.github.io/​palettable/​|Palettable]]:​ Color palettes for Python
 +    * ''​conda install -n cdatm18_py2 -c conda-forge palettable''​
  
 ==== TODO ==== ==== TODO ====
Line 177: Line 247:
  
 Some packages change more often than others, and can be easily updated the following way: Some packages change more often than others, and can be easily updated the following way:
- 
-  * [[https://​cmor.llnl.gov/​|CMOR]] 
-    * Update with: ''​conda update -n cdatm15 -c conda-forge -c pcmdi -c uvcdat cmor''​ 
-      * Get version number with: ''​python -c 'from cmor import *; print CMOR_VERSION_MAJOR,​ CMOR_VERSION_MINOR,​ CMOR_VERSION_PATCH'​ ''​ 
  
   * [[https://​earthsystemcog.org/​projects/​wip/​CMIP6DataRequest|dreqPy]]   * [[https://​earthsystemcog.org/​projects/​wip/​CMIP6DataRequest|dreqPy]]
Line 207: Line 273:
   * [[http://​www.ifremer.fr/​vacumm/​|vacumm]]:​ Validation, Analysis, Comparison - Utilities written in Python to validate and analyze Multi-Model outputs, and compare them to observations   * [[http://​www.ifremer.fr/​vacumm/​|vacumm]]:​ Validation, Analysis, Comparison - Utilities written in Python to validate and analyze Multi-Model outputs, and compare them to observations
   * [[http://​unidata.github.io/​netcdf4-python/​|netcdf4]]:​ a Python interface to the netCDF C library   * [[http://​unidata.github.io/​netcdf4-python/​|netcdf4]]:​ a Python interface to the netCDF C library
-  * [[http://​ferret.pmel.noaa.gov/​Ferret/​documentation/​pyferret|pyferret]]:​ Ferret encapsulated in +  * [[http://​ferret.pmel.noaa.gov/​Ferret/​documentation/​pyferret|pyferret]]:​ Ferret encapsulated in Python 
 +  * [[https://​github.com/​PBrockmann/​ipython_ferretmagic|ipython_ferretmagic]]:​ IPython notebook extension for ferret
   * [[https://​github.com/​conda-forge/​basemap-data-hires-feedstock|basemap-data-hires]]:​ high resolution data for ''​basemap''​   * [[https://​github.com/​conda-forge/​basemap-data-hires-feedstock|basemap-data-hires]]:​ high resolution data for ''​basemap''​
   * [[https://​github.com/​PCMDI/​pcmdi_metrics|PCMDI metrics package]] (PMP): objectively compare results from climate models with observations using well-established statistical tests   * [[https://​github.com/​PCMDI/​pcmdi_metrics|PCMDI metrics package]] (PMP): objectively compare results from climate models with observations using well-established statistical tests
Line 216: Line 283:
   * [[https://​github.com/​pierre-rouanet/​dtw|dtw]]:​ DTW (Dynamic Time Warping) python module   * [[https://​github.com/​pierre-rouanet/​dtw|dtw]]:​ DTW (Dynamic Time Warping) python module
   * [[http://​toblerity.org/​shapely/​project.html|shapely]]:​ a Python wrapper for GEOS for algebraic manipulation of geometry (manipulation and analysis of geometric objects in the Cartesian plane)   * [[http://​toblerity.org/​shapely/​project.html|shapely]]:​ a Python wrapper for GEOS for algebraic manipulation of geometry (manipulation and analysis of geometric objects in the Cartesian plane)
-  * [[http://​scitools.org.uk/​cartopy/​|cartopy]]:​ a library providing cartographic tools for python+  * [[http://​scitools.org.uk/​cartopy/​|cartopy]]:​ a Python package designed ​for geospatial data processing in order to produce maps and other geospatial data analyses
   * [[https://​rpy2.readthedocs.io/​en/​version_2.8.x/​|rpy2]]:​ providing simple and robust access to R from within Python   * [[https://​rpy2.readthedocs.io/​en/​version_2.8.x/​|rpy2]]:​ providing simple and robust access to R from within Python
   * [[http://​matplotlib.org/​cmocean/​|cmocean]]:​ beautiful colormaps for oceanography   * [[http://​matplotlib.org/​cmocean/​|cmocean]]:​ beautiful colormaps for oceanography
Line 224: Line 291:
   * [[https://​ajdawson.github.io/​windspharm/​latest/​|windspharm]]:​ spherical harmonic wind analysis in Python   * [[https://​ajdawson.github.io/​windspharm/​latest/​|windspharm]]:​ spherical harmonic wind analysis in Python
   * [[https://​climaf.readthedocs.io/​en/​latest/​|CliMAF]]:​ a Climate Model Assessment Framework   * [[https://​climaf.readthedocs.io/​en/​latest/​|CliMAF]]:​ a Climate Model Assessment Framework
 +  * [[https://​joblib.readthedocs.io/​en/​latest/​|joblib]]:​ running Python functions as pipeline jobs
 +  * [[https://​jiffyclub.github.io/​palettable/​|Palettable]]:​ Color palettes for Python
  
 ==== Removed packages ==== ==== Removed packages ====
Line 233: Line 302:
 After following the steps above, we get the following environments. Use the ''​conda env list''​ command (same result as ''​conda info %%--%%envs''​) to get the up-to-date list of available environments. After following the steps above, we get the following environments. Use the ''​conda env list''​ command (same result as ''​conda info %%--%%envs''​) to get the up-to-date list of available environments.
  
-Use ''​conda list -n env_name''​ to get a detailed list of which packages/​modules are installed+Use ''​conda list -n env_name''​ to get a detailed list of which packages/​modules are installed, or check the [[other:​uvcdat:​conda_notes#​installation_history|conda Installation history]] section to get more details
  
-^  Environment\\ name  ^  Server ​ ^   ​Summary ​ ​^ ​ ''​conda list'' ​ ^ +^  Environment\\ name  ^  Server ​ ^   ​Summary ​ ^ 
-| cdat-8.1_py2 |  LSCE\\ ciclad ​ | CDAT 8.1 & Python 2.7 +| cdat-8.1_py2 |  LSCE\\ ciclad ​ | CDAT 8.1 & Python 2.7 | 
-| cdat-8.1_py3 |  LSCE  | CDAT 8.1 & Python 3.6 +| cdat-8.1_py3 |  LSCE  | CDAT 8.1 & Python 3.6 | 
-| cdatm18_py2 |  LSCE\\ ciclad ​ | CDAT 8.1 & P 2.7 //JYP version// ​+| cdatm18_py2\\ or //​cdatm_py2// ​|  LSCE\\ ciclad ​ | CDAT 8.1 & P 2.7 //JYP version// | 
-| cdatm18_py3 |  LSCE  | CDAT 8.1 & P 3.7 //JYP version// ​+| cdatm18_py3\\ or //​cdatm_py3// ​|  LSCE  | CDAT 8.1 & P 3.7 //JYP version// | 
-| cdat-nightly_py2 |  LSCE\\ ciclad ​ | CDAT nightly & Python 2.7 +| cdat-nightly_py2 |  LSCE\\ ciclad ​ | CDAT nightly & Python 2.7 | 
-| cdatm-nightly_py2 |  LSCE\\ ciclad ​ | CDAT nightly & Python 3.6 | |+| cdatm-nightly_py2 |  LSCE\\ ciclad ​ | CDAT nightly & P 2.7 //JYP version// ​|
  
 /* standard page footer */ /* standard page footer */
other/uvcdat/cdat_conda/cdat_8_1.txt · Last modified: 2021/03/26 09:10 by jypeter