This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision | ||
other:uvcdat:cdat_conda:cdat_8_0_py2 [2019/03/01 16:12] jypeter |
other:uvcdat:cdat_conda:cdat_8_0_py2 [2021/02/27 13:55] (current) jypeter [Installing Miniconda3] |
||
---|---|---|---|
Line 18: | Line 18: | ||
==== Installing Miniconda3 ==== | ==== Installing Miniconda3 ==== | ||
- | We have installed a new version of Miniconda3 (steps below adapted from the [[other:uvcdat:conda_notes#installing_miniconda|installing miniconda]] instructions) | + | * You don't need to be (and **you should not be**) //root// when you install Miniconda3. You just need enough disk space where you have write access |
+ | * If you are installing Miniconda3 in a Linux environment on a [[other:win10wsl|Windows 10 computer using Windows Subsystem for Linux (WSL)]], pay special attention to the instructions on the **WSL** lines | ||
+ | |||
+ | The steps below are adapted from the older (but with lots of extra and possibly useful details) [[other:uvcdat:conda_notes#installing_miniconda|installing miniconda]] instructions | ||
+ | |||
+ | * Get the [[https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh|Python 3.8 Linux 64-bit (bash installer)]] (unless there is a [[https://conda.io/miniconda.html|more recent version?]]) | ||
+ | * Find some temporary space on Linux (or Windows, if you are using WSL) | ||
+ | * e.g. Linux at LSCE:\\ ''cd /home/scratch01/<your_login>'' | ||
+ | * e.g. WSL, assuming that there is a ''C:\Scratch\<your_login>'' directory:\\ ''cd /mnt/c/Scratch/<your_login>'' | ||
+ | * Use ''wget'' to download the installer:\\ ''wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh'' (90 Mb, 26 Feb 2021) | ||
- | * Get the [[https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh|Python 3.7 Linux 64-bit (bash installer)]] (unless there is a [[https://conda.io/miniconda.html|more recent version?]]) | ||
- | * Find some temporary space on Linux or Windows | ||
- | * Linux:''cd /home/scratch01/jypeter'' | ||
- | * WSL: ''cd /mnt/h/Scratch/jypeter/'' (assuming that there is a ''H:\Scratch\jypeter'' directory) | ||
- | * ''wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh'' (67 Mo, 15 Feb 2019) | ||
* Execute the installer | * Execute the installer | ||
* ''bash Miniconda3-latest-Linux-x86_64.sh'' | * ''bash Miniconda3-latest-Linux-x86_64.sh'' | ||
* Accept the license | * Accept the license | ||
- | * Specify an explicit installation path **outside of the home directory**, with enough disk space (more than 3G if you are going to install UV-CDAT and some extra packages), preferably on a disk that is not backed up: | + | * Note: at the end of the installation (next step), **answer //no// to the following question**, so that the installer does not change your existing shell initialization files!\\ ''Do you wish the installer to initialize Miniconda3 by running conda init? [yes|no]'' => **NO** |
- | * Linux LSCE: ''/home/share/unix_files/cdat/miniconda3'' | + | * Specify an explicit installation path **outside of your //home// directory**, with enough disk space (more than 3 Gb if you are going to install CDAT and some extra packages), preferably on a disk that is not backed up: |
- | * Linux ciclad: ''/data/jypmce/cdat/miniconda3'' | + | * Installations by //JYP//: |
- | * WSL: installing to a directory that is not in ''/home/'' <wrap hi>does not work</wrap> (e.g. ''/mnt/h/CDAT/miniconda3'',assuming there is a ''H:\CDAT\'' directory, does not work)\\ Need to accept the installation in the default location: ''/home/jypeter/miniconda3'' | + | * Linux at LSCE: ''/home/share/unix_files/cdat/miniconda3'', or another subdirectory of ''/home/share/unix_files/cdat/'' |
- | * Resulting ''miniconda3'' directory size is 302M\\ <code> >du -sh miniconda3 | + | * Linux on ciclad: ''/data/jypmce/cdat/miniconda3'', or another sub-directory of ''/data/jypmce/cdat/'' |
- | 302M miniconda3 | + | * WSL: installing to a directory that is not in ''/home/'' <wrap hi>does not work</wrap> (e.g. ''/mnt/h/CDAT/miniconda3'',assuming there is a ''H:\CDAT\'' directory, does not work)\\ You need to accept the installation in the default location: ''/home/<your_login>/miniconda3'' |
- | >cd miniconda3 | + | * The resulting ''miniconda3'' directory size is 342M\\ <code> > du -sh miniconda3 |
- | /home/share/unix_files/cdat/miniconda3 | + | 342M miniconda3 |
- | >du -sh * | + | |
- | 6.3M bin | + | > cd miniconda3 |
- | 2.0M compiler_compat | + | |
- | 524K conda-meta | + | > du -sh * |
- | 4.0K envs | + | 20M bin |
- | 28K etc | + | 0 compiler_compat |
- | 5.2M include | + | 4.0K condabin |
- | 146M lib | + | 684K conda-meta |
- | 8.0K LICENSE.txt | + | 0 envs |
- | 142M pkgs | + | 16K etc |
- | 928K share | + | 5.5M include |
- | 92K ssl | + | 4.0K info |
- | 12K x86_64-conda_cos6-linux-gnu</code> | + | 198M lib |
- | * Accept the automatic update of the //PATH// variable in ''.bashrc'' (note: the installer automatically creates a backup of ''.bashrc''). This will some extra lines at the end of ''.bashrc'' used by the //old-style conda initialization// (we will later switch to the //new-style//). The added lines should look like: | + | 12K LICENSE.txt |
- | * ''# added by Miniconda3 installer''\\ ''export PATH="/home/share/unix_files/cdat/miniconda3/bin:$PATH"'' | + | 114M pkgs |
- | * ''# added by Miniconda4.xxx installer''\\ ''[some really fancy bash shell commands]'' | + | 604K share |
- | * Remove the installer: ''rm Miniconda3-latest-Linux-x86_64.sh'' | + | 4.0K shell |
- | * Open a new terminal (if you use bash) or open a new terminal and type ''bash'' | + | 0 ssl |
- | * check if the python from the conda distribution has been correctly installed\\ <code> > bash | + | 0 x86_64-conda_cos6-linux-gnu</code> |
+ | |||
+ | * Initialize the newly installed conda environment: | ||
+ | * bash shell: ''source <installation_path>/miniconda3/etc/profile.d/conda.sh'' | ||
+ | * tcsh shell: ''source <installation_path>/miniconda3/etc/profile.d/conda.csh'' | ||
+ | |||
+ | * Check if you can use the ''conda'' command, and use it to initialize the //base// environment | ||
+ | * <code>$ which conda | ||
+ | <installation_path>/miniconda3/condabin/conda | ||
$ which python | $ which python | ||
- | /home/share/unix_files/cdat/miniconda3/bin/python | + | /usr/bin/python |
- | $ which conda | + | $ conda activate |
- | /home/share/unix_files/cdat/miniconda3/bin/conda</code> | + | (base) $ which python |
- | * Update and clean the new installation\\ <code>$ conda update --all | + | <installation_path>/miniconda3/bin/python</code> |
- | $ du -sh /home/share/unix_files/cdat/miniconda3 | + | * Update the new installation |
- | 515M /home/share/unix_files/cdat/miniconda3 | + | * <code>$ conda update --all |
- | $ conda clean --all | + | |
- | $ du -sh /home/share/unix_files/cdat/miniconda3 | + | |
- | 270M /home/share/unix_files/cdat/miniconda3 | + | |
- | $ conda list | + | |
- | # packages in environment at /home/share/unix_files/cdat/miniconda3: | + | |
- | # | + | |
- | # Name Version Build Channel | + | |
- | [...] | + | |
- | python 3.7.1 h0371630_3 | + | |
- | readline 7.0 h7b6447c_5 | + | |
- | requests 2.20.0 py37_0 | + | |
[...]</code> | [...]</code> | ||
+ | * During the update, the ''miniconda3'' directory size goes from 432 Mb to 581 Mb. This directory will keep on growing, which is the reason why you should put it on a (preferably non backed up) disk where you have //enough// space | ||
+ | |||
+ | * Make sure we have the latest ''conda'' (just in case we did not get it with the //update//)\\ ''conda update -n base -c defaults conda'' | ||
+ | |||
+ | * Remove the installer:\\ ''rm Miniconda3-latest-Linux-x86_64.sh'' | ||
==== Post-Miniconda3 installation ==== | ==== Post-Miniconda3 installation ==== | ||
Line 248: | Line 254: | ||
<code>$ conda create -n cdat-8.0_py3 -c cdat/label/v80 -c conda-forge -c cdat python=3.6 cdat</code> | <code>$ conda create -n cdat-8.0_py3 -c cdat/label/v80 -c conda-forge -c cdat python=3.6 cdat</code> | ||
+ | |||
+ | ==== Installing CDAT nightly ==== | ||
+ | |||
+ | Notes: | ||
+ | |||
+ | * This page is about CDAT 8.0, but we have added a short nightly section here as a convenient shortcut. This should probably be moved to a stand-alone nightly page later | ||
+ | * [[https://github.com/CDAT/cdat/wiki/Obtain-CDAT-nightly-packages|Official nightly page]] | ||
+ | |||
+ | === Python 2.7 version === | ||
+ | |||
+ | <code>$ conda create -n cdat-nightly_py2 -c cdat/label/nightly -c conda-forge -c cdat python=2.7 cdat</code> | ||
+ | |||
+ | <wrap hi>Not tested!</wrap> Can this be updated with ''conda update -n cdat-nightly_py2 -c cdat/label/nightly -c conda-forge -c cdat --all'' ??? | ||
+ | |||
+ | === Python 3.6 version === | ||
+ | |||
+ | <code>$ conda create -n cdat-nightly_py3 -c cdat/label/nightly -c conda-forge -c cdat python=3.6 cdat</code> | ||
===== Cloning cdat to add specific packages for LSCE ===== | ===== Cloning cdat to add specific packages for LSCE ===== | ||
Line 260: | Line 283: | ||
Note: using hard links, cloning a full environment //only// adds an extra 582M of disk space | Note: using hard links, cloning a full environment //only// adds an extra 582M of disk space | ||
- | <code>$ du -sh /home/share/unix_files/cdat/miniconda3/envs/* | + | <code>$ conda create -n cdatm17_py2 --clone cdat-8.0_py2 |
- | 2.8G /home/share/unix_files/cdat/miniconda3/envs/cdat-8.0_py2 | + | |
- | $ conda create -n cdatm17_py2 --clone cdat-8.0_py2 | + | |
Source: /home/share/unix_files/cdat/miniconda3/envs/cdat-8.0_py2 | Source: /home/share/unix_files/cdat/miniconda3/envs/cdat-8.0_py2 | ||
Destination: /home/share/unix_files/cdat/miniconda3/envs/cdatm17_py2 | Destination: /home/share/unix_files/cdat/miniconda3/envs/cdatm17_py2 | ||
Line 271: | Line 292: | ||
$ du -sh /home/share/unix_files/cdat/miniconda3/envs/* | $ du -sh /home/share/unix_files/cdat/miniconda3/envs/* | ||
- | 2.8G /home/share/unix_files/cdat/miniconda3/envs/cdat-8.0_py2 | + | 2.5G /home/share/unix_files/cdat/miniconda3/envs/cdat-8.0_py2 |
- | 582M /home/share/unix_files/cdat/miniconda3/envs/cdatm17_py2 | + | 392M /home/share/unix_files/cdat/miniconda3/envs/cdatm17_py2 |
+ | 839M /home/share/unix_files/cdat/miniconda3/envs/cdat-nightly_py2 | ||
</code> | </code> | ||
+ | |||
+ | ==== cdat nightly case ==== | ||
+ | |||
+ | ''conda create -n cdatm-nightly_py2 %%--%%clone cdat-nightly_py2'' | ||
===== Getting ready for a moving default CDAT ===== | ===== Getting ready for a moving default CDAT ===== | ||
+ | |||
+ | <note warning>This step should probably be listed at the **end**, especially in a multi-user environment! If there is already a ''cdatm'' link, make sure that the new version is stable and working correctly before updating the ''cdatm'' link</note> | ||
We create a **//cdatm// symbolic link** in the ''envs'' directory, that has a //stable name// but can be moved to point to the latest default CDAT. In that case, most users can just activate this //cdatm// version and always get the latest stable version | We create a **//cdatm// symbolic link** in the ''envs'' directory, that has a //stable name// but can be moved to point to the latest default CDAT. In that case, most users can just activate this //cdatm// version and always get the latest stable version | ||
Line 319: | Line 347: | ||
After cloning, we are ready to install some extra packages that may be requested by LSCE users | After cloning, we are ready to install some extra packages that may be requested by LSCE users | ||
+ | |||
+ | * We first try to install together as many packages as possible that don't require other channels than //conda-forge//, and that don't request a downgrade of what is already installed | ||
+ | * We then install individual extra packages with ''conda install'' or ''pip install'' | ||
<code># Keep a trace of what will be installed | <code># Keep a trace of what will be installed | ||
- | bash-4.1$ conda install --dry-run -n cdatm15 -c conda-forge -c uvcdat -c pcmdi pillow pandas statsmodels seaborn scikit-image seawater gsw netcdf4 pyferret basemap-data-hires pcmdi_metrics xlsxwriter cmor shapely cartopy rpy2 > /home/scratch01/jypeter/lsce-extra_01_install_170808.txt | + | $ conda install --dry-run -n cdatm17_py2 -c conda-forge pillow pandas statsmodels seaborn scikit-image seawater gsw netcdf4 pyferret basemap-data-hires xlsxwriter cmocean rpy2 gdal windspharm > /home/scratch01/jypeter/lsce-extra_01_install_190304.txt |
# Install... | # Install... | ||
- | bash-4.1$ conda install -n cdatm15 -c conda-forge -c uvcdat -c pcmdi pillow pandas statsmodels seaborn scikit-image seawater gsw netcdf4 pyferret basemap-data-hires pcmdi_metrics xlsxwriter cmor shapely cartopy rpy2 | + | $ conda install -n cdatm17_py2 -c conda-forge pillow pandas statsmodels seaborn scikit-image seawater gsw netcdf4 pyferret basemap-data-hires xlsxwriter cmocean rpy2 gdal windspharm |
[...] | [...] | ||
+ | |||
+ | # Install extra packages in the nightly version (if you need 'nightly') | ||
+ | $ conda install -n cdatm-nightly_py2 -c conda-forge pillow pandas statsmodels seaborn scikit-image seawater gsw netcdf4 pyferret basemap-data-hires xlsxwriter cmocean rpy2 gdal windspharm | ||
# Check the disk space after installation | # Check the disk space after installation | ||
- | bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2 | + | $ du -sh /home/share/unix_files/cdat/miniconda3 |
- | 11G /home/share/unix_files/cdat/miniconda2 | + | 7.5G /home/share/unix_files/cdat/miniconda3 |
- | bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2/envs/* | + | $ du -sh /home/share/unix_files/cdat/miniconda3/envs/* |
- | 0 /home/share/unix_files/cdat/miniconda2/envs/cdatm | + | 2.5G /home/share/unix_files/cdat/miniconda3/envs/cdat-8.0_py2 |
- | 2.8G /home/share/unix_files/cdat/miniconda2/envs/cdatm14 | + | 1.2G /home/share/unix_files/cdat/miniconda3/envs/cdatm17_py2 |
- | 3.7G /home/share/unix_files/cdat/miniconda2/envs/cdatm15 | + | 938M /home/share/unix_files/cdat/miniconda3/envs/cdatm-nightly_py2 |
- | 574M /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.10 | + | 450M /home/share/unix_files/cdat/miniconda3/envs/cdat-nightly_py2</code> |
- | 2.0G /home/share/unix_files/cdat/miniconda2/envs/uvcdat-2.8.0 | + | |
- | + | ||
- | # Check the disk space again after cleaning | + | |
- | bash-4.1$ conda clean --all | + | |
- | bash-4.1$ du -sh /home/share/unix_files/cdat/miniconda2 | + | List of installed packages: {{ :other:uvcdat:cdat_conda:lsce-extra_01_install_190304.txt |}} |
- | 9.9G /home/share/unix_files/cdat/miniconda2</code> | + | |
- | + | ||
- | List of installed packages: {{ :other:uvcdat:cdat_conda:lsce-extra_01_install_170808.txt |}} | + | |
- | + | ||
- | Disk space used after adding the extra packages, in <wrap em>in the new directory hierarchy</wrap> (''cdat/miniconda2b'', on the obelix servers) | + | |
- | + | ||
- | <code>bash-4.2$ du -sh /home/share/unix_files/cdat/miniconda2b | + | |
- | 6.4G /home/share/unix_files/cdat/miniconda2b | + | |
- | + | ||
- | bash-4.2$ du -sh /home/share/unix_files/cdat/miniconda2b/envs/* | + | |
- | 0 /home/share/unix_files/cdat/miniconda2b/envs/cdatm | + | |
- | 3.8G /home/share/unix_files/cdat/miniconda2b/envs/cdatm15 | + | |
- | 630M /home/share/unix_files/cdat/miniconda2b/envs/uvcdat-2.10 | + | |
- | + | ||
- | # Check the disk space again after cleaning | + | |
- | bash-4.2$ conda clean --all | + | |
- | + | ||
- | bash-4.2$ du -sh /home/share/unix_files/cdat/miniconda2b | + | |
- | 5.4G /home/share/unix_files/cdat/miniconda2b</code> | + | |
Packages installed with //pip// | Packages installed with //pip// | ||
Line 367: | Line 378: | ||
* Update with: ''pip install %%--%%upgrade dreqPy'' | * Update with: ''pip install %%--%%upgrade dreqPy'' | ||
* Get version number with:\\ <code>$ drq -v | * Get version number with:\\ <code>$ drq -v | ||
- | dreqPy version 01.00.06 [Version 01.00.06]</code> | + | dreqPy version 01.00.29 [Version 01.00.29]</code> |
- | * [[https://github.com/pierre-rouanet/dtw|dtw]]: DTW (Dynamic Time Warping) python module | + | |
- | * ''pip install dtw'' | + | |
- | * Requested by Sentia Goursaud | + | |
The following packages have no dependency problems and were installed (or updated) //later// | The following packages have no dependency problems and were installed (or updated) //later// | ||
- | * [[http://www.earthsystemmodeling.org/esmf_releases/non_public/ESMF_7_0_0/esmpy_doc/html/index.html|ESMPy]]: ESMF Python Regridding Interface | ||
- | * We used the following to get a more up-to-date version of esmpy than the one bundled with 2.10 | ||
- | * <code>bash-4.1$ conda list | grep -i esm | ||
- | esmf 7.0.0 6 conda-forge | ||
- | esmpy 7.0.0 py27_1 conda-forge | ||
- | bash-4.1$ conda install -n cdatm15 -c nesii/channel/dev-esmf -c conda-forge esmpy=7.1.0.dev32 | + | * [[http://www.ifremer.fr/vacumm/|vacumm]]: Validation, Analysis, Comparison - Utilities written in Python to validate and analyze Multi-Model outputs, and compare them to observations |
- | [...] | + | * ''conda install -n cdatm17_py2 -c conda-forge -c vacumm vacumm'' |
- | bash-4.1$ conda list | grep -i esm | ||
- | esmf 7.1.0.dev32 1 nesii/channel/dev-esmf | ||
- | esmpy 7.1.0.dev32 py27_1 nesii/channel/dev-esmf</code> | ||
- | * [[http://matplotlib.org/cmocean/|cmocean]]: beautiful colormaps for oceanography | ||
- | * ''conda install -n cdatm15 -c conda-forge cmocean'' | ||
- | * [[http://scitools.org.uk/iris/index.html|iris]]: A Python library for Meteorology and Climatology | ||
- | * ''conda install -n cdatm15 -c conda-forge -c scitools iris'' | ||
- | * [[http://www.gdal.org/|OSGeo/GDAL]]: Geospatial Data Abstraction Library. GDAL is a translator library for raster and vector geospatial data formats | ||
- | * ''conda install -c conda-forge gdal'' | ||
- | * [[https://pcjericks.github.io/py-gdalogr-cookbook/|Python GDAL/OGR Cookbook]] | ||
* [[https://github.com/stefraynaud/spanlib|spanlib]]: Spectral Analysis Library | * [[https://github.com/stefraynaud/spanlib|spanlib]]: Spectral Analysis Library | ||
- | * ''conda install -n cdatm15 -c stefraynaud -c conda-forge spanlib'' | + | * ''conda install -n cdatm17_py2 -c stefraynaud -c conda-forge spanlib'' |
* Test: ''python -c 'from spanlib.analyzer import Analyzer' '' | * Test: ''python -c 'from spanlib.analyzer import Analyzer' '' | ||
- | * [[http://wrf-python.readthedocs.io/en/latest/|wrf-python]]: A collection of diagnostic and interpolation routines for use with output from the Weather Research and Forecasting (WRF-ARW) Model | + | |
- | * ''conda install -n cdatm15 -c conda-forge wrf-python'' | + | * [[https://climaf.readthedocs.io/en/latest/|CliMAF]]: a Climate Model Assessment Framework |
- | * [[https://glances.readthedocs.io/en/stable/index.html|glances]]: a cross-platform monitoring tool (similar to ''top'') | + | * [[other:uvcdat:cdat_conda:climaf_install|Installation notes]] |
- | * ''conda install -n cdatm15 -c conda-forge glances'' | + | |
- | * [[https://github.com/nicolargo/glances|glances@github]] | + | |
- | * [[https://ajdawson.github.io/windspharm/latest/|windspharm]]: spherical harmonic wind analysis in Python | + | |
- | * ''conda install -n cdatm15 -c conda-forge windspharm'' | + | |
- | * [[https://github.com/ajdawson/windspharm|windspharm@github]] | + | |
==== TODO ==== | ==== TODO ==== | ||
- | //Add here packages that would be useful but have some problems that prevent their installation// | + | //Add here packages that would be useful and have not been installed yet, or have some problems that prevent their installation// |
- | * [[http://www.ifremer.fr/vacumm/|vacumm]]: Validation, Analysis, Comparison - Utilities written in Python to validate and analyze Multi-Model outputs, and compare them to observations | + | * [[http://scitools.org.uk/iris/index.html|iris]]: A Python library for Meteorology and Climatology |
- | * As of August 08 2017, installing vacumm in 2.10 requires downgrading to 2.10 (see [[https://github.com/VACUMM/vacumm/issues/9|github issue]])! | + | * ''conda install -n cdatxxx -c conda-forge iris'' |
- | * <wrap hi>Problem solved</wrap> with **vacumm 3.4.1-1** on August 31st! Installation was OK with\\ ''conda install -n cdatm15 -c conda-forge -c vacumm vacumm'' | + | |
- | * Install PySPLIT for Sentia\\ [[https://github.com/mscross/pysplit|PySPLIT]] A package for generating HYSPLIT air parcel trajectories trajectories, performing moisture uptake analyses, expediting HYSPLIT cluster analysis, and for visualizing trajectories, clusters, and along-trajectory meteorological data\\ Works with [[https://ready.arl.noaa.gov/HYSPLIT.php|HYSPLiT]] | + | * [[http://wrf-python.readthedocs.io/en/latest/|wrf-python]]: A collection of diagnostic and interpolation routines for use with output from the Weather Research and Forecasting (WRF-ARW) Model |
- | - ''conda install -n cdatm15 -c conda-forge fiona geopandas'' | + | * ''conda install -n cdatxxx -c conda-forge wrf-python'' |
- | - ''pip install pysplit'' | + | |
+ | * [[https://glances.readthedocs.io/en/stable/index.html|glances]]: a cross-platform monitoring tool (similar to ''top'') | ||
+ | * ''conda install -n cdatxxx -c conda-forge glances'' | ||
+ | * [[https://github.com/nicolargo/glances|glances@github]] | ||
==== Other packages ==== | ==== Other packages ==== | ||
Line 428: | Line 417: | ||
* [[https://cmor.llnl.gov/|CMOR]] | * [[https://cmor.llnl.gov/|CMOR]] | ||
- | * Update with: ''conda update -n cdatm15 -c conda-forge -c pcmdi -c uvcdat cmor'' | + | * Update with: ''conda update -n cdatxxx -c conda-forge -c pcmdi -c uvcdat cmor'' |
* Get version number with: ''python -c 'from cmor import *; print CMOR_VERSION_MAJOR, CMOR_VERSION_MINOR, CMOR_VERSION_PATCH' '' | * Get version number with: ''python -c 'from cmor import *; print CMOR_VERSION_MAJOR, CMOR_VERSION_MINOR, CMOR_VERSION_PATCH' '' | ||
Line 434: | Line 423: | ||
* Update with: ''pip install %%--%%upgrade dreqPy'' | * Update with: ''pip install %%--%%upgrade dreqPy'' | ||
* Get version number with: ''drq -v'' | * Get version number with: ''drq -v'' | ||
+ | |||
+ | ==== Installing CLIMAF ==== | ||
+ | |||
+ | The [[https://climaf.readthedocs.io/|CLIMAF]] installation is a special case, because we may want to use it with the Python installed for CDAT, but it is not installed using conda | ||
+ | |||
+ | We follow the steps for the [[https://climaf.readthedocs.io/en/latest/installing.html#installing-climaf-if-you-are-not-at-cnrm-on-ciclad-or-climserv|installation of a non-standard server]] | ||
+ | |||
+ | * Go to the appropriate installation directory (we choose the directory where we have also installed miniconda3): | ||
+ | * LSCE: ''cd /home/share/unix_files/cdat'' | ||
+ | * ciclad: ''/data/jypmce/cdat/'' | ||
+ | * ''git clone https://github.com/senesis/climaf climaf'' | ||
+ | * Add CLIMAF to the PYTHONPATH environment variable | ||
+ | * LSCE & tcsh: ''setenv PYTHONPATH ${PYTHONPATH}:/home/share/unix_files/cdat/climaf'' | ||
+ | * LSCE & bash: ''export PYTHONPATH=${PYTHONPATH}:/home/share/unix_files/cdat/climaf'' | ||
+ | * ciclad: same as above, using ''/data/jypmce/cdat/climaf'' | ||
+ | * Test the installation | ||
+ | * make sure the [[https://climaf.readthedocs.io/en/latest/requirements.html#requirements|Requirements]] are available: | ||
+ | * LSCE: ''module load netcdf/4 nco cdo ncl ncview'' | ||
+ | * ciclad: ''module load netcdf4 nco cdo ncl/6.3.0'' | ||
+ | * optional: start an X server | ||
+ | * <code>cd climaf/testing | ||
+ | ./test_install.sh</code> | ||
+ | * Once the tests are OK, add the required //module load// and environment variables definitions to the ''.basrc'' and ''.cshrc'' files. <wrap hi>Note: maybe these lines should go to the ''.profile'' and ''.login'' files...(check [[https://www.gnu.org/software/bash/manual/html_node/Bash-Startup-Files.html|Bash Startup Files]])</wrap> | ||
+ | * LSCE: add to your .cshrc | ||
+ | * <code># Prerequisites for using CLIMAF | ||
+ | module load netcdf/4 nco cdo ncl ncview | ||
+ | |||
+ | # CLIMAF environment variables | ||
+ | setenv CLIMAF /home/share/unix_files/cdat/climaf | ||
+ | setenv PYTHONPATH "${PYTHONPATH}:${CLIMAF}" | ||
+ | #setenv PYTHONPATH $CLIMAF | ||
+ | setenv PATH "${PATH}:${CLIMAF}/bin" | ||
+ | setenv CLIMAF_CACHE "/home/scratch01/${USER}/climaf_cache" | ||
+ | setenv TMPDIR $CLIMAF_CACHE | ||
+ | setenv CLIMAF_LOG_DIR $CLIMAF_CACHE | ||
+ | |||
+ | #conda activate cdatm-nightly_py2</code> | ||
+ | * ciclad: add to your .bashrc | ||
+ | * <code># Prerequisites for using CLIMAF when not using the | ||
+ | # default 'module load climaf' | ||
+ | module load netcdf4 nco cdo ncl/6.3.0 | ||
+ | |||
+ | # CLIMAF environment variables | ||
+ | export CLIMAF=/data/jypmce/cdat/climaf | ||
+ | #export PYTHONPATH=$PYTHONPATH:$CLIMAF | ||
+ | export PYTHONPATH=$CLIMAF | ||
+ | export PATH=$PATH:$CLIMAF/bin | ||
+ | export CLIMAF_CACHE=/data/jypmce/climaf_cache | ||
+ | export TMPDIR=$CLIMAF_CACHE | ||
+ | export CLIMAF_LOG_DIR=$CLIMAF_CACHE | ||
+ | |||
+ | #conda activate cdatm-nightly_py2 | ||
+ | </code> | ||
+ | * Open a new shell and <wrap em>run the tests again</wrap>. If there is an error that was not there, check what you have added to the shell init files | ||
+ | * [[other:uvcdat:cdat_conda:climaf_test|Test CLIMAF on ciclad]] | ||
==== Cleaning up things ==== | ==== Cleaning up things ==== | ||
Line 440: | Line 484: | ||
We check if some of the installed files are missing read access for the //group// or //other//, and we manually change the permissions | We check if some of the installed files are missing read access for the //group// or //other//, and we manually change the permissions | ||
- | <code>(cdatm15) bash-4.2$ find cdatm15 \! -perm /g+r,o+r -ls | + | <code> >find /home/share/unix_files/cdat/miniconda3/envs \! -perm /g+r,o+r -ls |
- | 50333969 4 -rw------- 2 jypeter lsce 234 Apr 4 20:35 cdatm15/lib/python2.7/site-packages/cdp-1.0.3-py2.7.egg-info/PKG-INFO | + | # Everything OK!</code> |
- | 50333971 4 -rw------- 2 jypeter lsce 291 Apr 4 20:35 cdatm15/lib/python2.7/site-packages/cdp-1.0.3-py2.7.egg-info/SOURCES.txt | + | |
- | 50333962 4 -rw------- 2 jypeter lsce 4 Apr 4 20:35 cdatm15/lib/python2.7/site-packages/cdp-1.0.3-py2.7.egg-info/top_level.txt | + | |
- | 50333961 4 -rw------- 2 jypeter lsce 1 Apr 4 20:35 cdatm15/lib/python2.7/site-packages/cdp-1.0.3-py2.7.egg-info/dependency_links.txt | + | |
- | 50333960 4 -rw------- 2 jypeter lsce 1 Apr 4 20:35 cdatm15/lib/python2.7/site-packages/cdp-1.0.3-py2.7.egg-info/not-zip-safe | + | |
- | 50333987 8 -rw------- 2 jypeter lsce 5149 Apr 4 20:35 cdatm15/lib/python2.7/site-packages/cdp/cdp_viewer.pyc | + | |
- | 50333986 8 -rw------- 2 jypeter lsce 4208 Apr 4 20:35 cdatm15/lib/python2.7/site-packages/cdp/cdp_metric.py | + | |
- | 50333966 4 -rw------- 2 jypeter lsce 150 Apr 4 20:35 cdatm15/lib/python2.7/site-packages/cdp/__init__.pyc | + | |
- | 50333968 4 -rw------- 2 jypeter lsce 201 Apr 4 20:35 cdatm15/lib/python2.7/site-packages/cdp/cdp_provenance.py | + | |
- | 50333988 8 -rw------- 2 jypeter lsce 5365 Apr 4 20:35 cdatm15/lib/python2.7/site-packages/cdp/cdp_metric.pyc | + | |
- | 50333982 4 -rw------- 2 jypeter lsce 1821 Apr 4 20:35 cdatm15/lib/python2.7/site-packages/cdp/cdp_parser.py | + | |
- | 50333981 4 -rw------- 2 jypeter lsce 1773 Apr 4 20:35 cdatm15/lib/python2.7/site-packages/cdp/cdp_parameter.py | + | |
- | 50333973 4 -rw------- 2 jypeter lsce 670 Apr 4 20:35 cdatm15/lib/python2.7/site-packages/cdp/cdp_output.py | + | |
- | 50333980 4 -rw------- 2 jypeter lsce 1642 Apr 4 20:35 cdatm15/lib/python2.7/site-packages/cdp/cdp_output.pyc | + | |
- | 50333979 4 -rw------- 2 jypeter lsce 1474 Apr 4 20:35 cdatm15/lib/python2.7/site-packages/cdp/_cache.pyc | + | |
- | 50333967 4 -rw------- 2 jypeter lsce 187 Apr 4 20:35 cdatm15/lib/python2.7/site-packages/cdp/cdp_tool.py | + | |
- | 50333989 0 -rw------- 2 jypeter lsce 0 Apr 4 20:35 cdatm15/lib/python2.7/site-packages/cdp/__init__.py | + | |
- | 50333985 4 -rw------- 2 jypeter lsce 3843 Apr 4 20:35 cdatm15/lib/python2.7/site-packages/cdp/cdp_viewer.py | + | |
- | 50333984 4 -rw------- 2 jypeter lsce 2436 Apr 4 20:35 cdatm15/lib/python2.7/site-packages/cdp/cdp_parameter.pyc | + | |
- | 50333970 4 -rw------- 2 jypeter lsce 288 Apr 4 20:35 cdatm15/lib/python2.7/site-packages/cdp/cdp_io.py | + | |
- | 50333974 4 -rw------- 2 jypeter lsce 692 Apr 4 20:35 cdatm15/lib/python2.7/site-packages/cdp/cdp_tool.pyc | + | |
- | 50333978 4 -rw------- 2 jypeter lsce 878 Apr 4 20:35 cdatm15/lib/python2.7/site-packages/cdp/cdp_io.pyc | + | |
- | 50333976 4 -rw------- 2 jypeter lsce 734 Apr 4 20:35 cdatm15/lib/python2.7/site-packages/cdp/_cache.py | + | |
- | 50333983 4 -rw------- 2 jypeter lsce 2269 Apr 4 20:35 cdatm15/lib/python2.7/site-packages/cdp/cdp_parser.pyc | + | |
- | 50333975 4 -rw------- 2 jypeter lsce 724 Apr 4 20:35 cdatm15/lib/python2.7/site-packages/cdp/cdp_provenance.pyc | + | |
- | (cdatm15) bash-4.2$ chmod -R a+r cdatm15/lib/python2.7/site-packages/cdp* | + | |
- | (cdatm15) bash-4.2$ find cdatm15 \! -perm /g+r,o+r -ls</code> | + | |
Line 493: | Line 511: | ||
* [[https://rpy2.readthedocs.io/en/version_2.8.x/|rpy2]]: providing simple and robust access to R from within Python | * [[https://rpy2.readthedocs.io/en/version_2.8.x/|rpy2]]: providing simple and robust access to R from within Python | ||
* [[http://matplotlib.org/cmocean/|cmocean]]: beautiful colormaps for oceanography | * [[http://matplotlib.org/cmocean/|cmocean]]: beautiful colormaps for oceanography | ||
- | * [[http://scitools.org.uk/iris/index.html|iris]]: A Python library for Meteorology and Climatology | ||
* [[http://www.gdal.org/|OSGeo/GDAL]]: Geospatial Data Abstraction Library. GDAL is a translator library for raster and vector geospatial data formats | * [[http://www.gdal.org/|OSGeo/GDAL]]: Geospatial Data Abstraction Library. GDAL is a translator library for raster and vector geospatial data formats | ||
+ | * [[https://pcjericks.github.io/py-gdalogr-cookbook/|Python GDAL/OGR Cookbook]] | ||
* [[https://github.com/stefraynaud/spanlib|spanlib]]: Spectral Analysis Library | * [[https://github.com/stefraynaud/spanlib|spanlib]]: Spectral Analysis Library | ||
- | * [[http://wrf-python.readthedocs.io/en/latest/|wrf-python]]: A collection of diagnostic and interpolation routines for use with output from the Weather Research and Forecasting (WRF-ARW) Model | ||
- | * [[https://glances.readthedocs.io/en/stable/index.html|glances]]: a cross-platform monitoring tool (similar to ''top'') | ||
* [[https://ajdawson.github.io/windspharm/latest/|windspharm]]: spherical harmonic wind analysis in Python | * [[https://ajdawson.github.io/windspharm/latest/|windspharm]]: spherical harmonic wind analysis in Python | ||
Line 509: | Line 525: | ||
^ Environment\\ name ^ Server ^ ''conda list'' ^ | ^ Environment\\ name ^ Server ^ ''conda list'' ^ | ||
- | | uvcdat-2.10 | LSCE | {{ |package list}} | | + | | cdat-8.0_py2 | LSCE | | |
- | | cdatm15 | LSCE | {{ |package list}} | | + | | cdatm17_py2 | LSCE | | |
+ | | cdat-nightly_py2 | LSCE\\ ciclad | | | ||
+ | | cdatm-nightly_py2 | LSCE\\ ciclad | | | ||
/* standard page footer */ | /* standard page footer */ |