This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
other:uvcdat:cdat_conda:cdat_8_2_1 [2021/06/09 14:52] jypeter [Extra packages list] Cleaned the list, added python-cdo |
other:uvcdat:cdat_conda:cdat_8_2_1 [2022/03/25 09:01] jypeter [Extra packages list] Added nccmp |
||
---|---|---|---|
Line 227: | Line 227: | ||
=== Pre-installation check with the dry-run option === | === Pre-installation check with the dry-run option === | ||
- | <code>/usr/bin/time mamba install -n cdatm19_nompi_py3 --dry-run -c conda-forge basemap basemap-data-hires cartopy cmocean cmor ferret_datasets gdal gdal geopandas glances gsw iris joblib netcdf4 palettable pandas pillow pyferret rpy2 scikit-image scikit-learn seaborn seawater shapely spyder statsmodels windspharm wrf-python xarray xlsxwriter | + | <code>/usr/bin/time mamba install -n cdatm19_nompi_py3 --dry-run -c conda-forge basemap basemap-data basemap-data-hires cartopy cmocean cmor ferret_datasets gdal gdal geopandas glances gsw iris joblib netcdf4 palettable pandas pillow pyferret rpy2 scikit-image scikit-learn seaborn seawater shapely spyder statsmodels windspharm wrf-python xarray xlsxwriter |
[... lots of information about what would be done followed by some information about how long it took ...] | [... lots of information about what would be done followed by some information about how long it took ...] | ||
Line 244: | Line 244: | ||
=== Actual installation === | === Actual installation === | ||
- | <code>/usr/bin/time mamba install -n cdatm19_nompi_py3 -c conda-forge basemap basemap-data-hires cartopy cmocean cmor ferret_datasets gdal gdal geopandas glances gsw iris joblib netcdf4 palettable pandas pillow pyferret rpy2 scikit-image scikit-learn seaborn seawater shapely spyder statsmodels windspharm wrf-python xarray xlsxwriter</code> | + | <code>/usr/bin/time mamba install -n cdatm19_nompi_py3 -c conda-forge basemap basemap-data basemap-data-hires cartopy cmocean cmor ferret_datasets gdal gdal geopandas glances gsw iris joblib netcdf4 palettable pandas pillow pyferret rpy2 scikit-image scikit-learn seaborn seawater shapely spyder statsmodels windspharm wrf-python xarray xlsxwriter</code> |
Resulting list of installed packages | Resulting list of installed packages | ||
Line 269: | Line 269: | ||
* Get version number with: ''python -c 'from cmor import *; print( (CMOR_VERSION_MAJOR, CMOR_VERSION_MINOR, CMOR_VERSION_PATCH) )' '' | * Get version number with: ''python -c 'from cmor import *; print( (CMOR_VERSION_MAJOR, CMOR_VERSION_MINOR, CMOR_VERSION_PATCH) )' '' | ||
* <wrap hi>Warning!</wrap> [[https://github.com/PCMDI/cmor/issues/449|CMOR currently requires Python 2.7]] | * <wrap hi>Warning!</wrap> [[https://github.com/PCMDI/cmor/issues/449|CMOR currently requires Python 2.7]] | ||
- | |||
- | * [[http://www.ifremer.fr/vacumm/|vacumm]]: Validation, Analysis, Comparison - Utilities written in Python to validate and analyze Multi-Model outputs, and compare them to observations | ||
- | * ''conda install -n cdatm19_py3 -c conda-forge -c vacumm vacumm'' | ||
- | * <wrap hi>Warning!</wrap> [[https://github.com/VACUMM/vacumm/issues/15|standard install of vacumm does not work for Python3]] | ||
- | |||
- | * [[https://github.com/stefraynaud/spanlib|spanlib]]: Spectral Analysis Library | ||
- | * ''conda install -n cdatm19_py3 -c stefraynaud -c conda-forge spanlib'' | ||
- | * Test: ''python -c 'from spanlib.analyzer import Analyzer' '' | ||
* [[http://scitools.org.uk/cartopy/|cartopy]]: a Python package designed for geospatial data processing in order to produce maps and other geospatial data analyses | * [[http://scitools.org.uk/cartopy/|cartopy]]: a Python package designed for geospatial data processing in order to produce maps and other geospatial data analyses | ||
Line 286: | Line 278: | ||
* [[https://climaf.readthedocs.io/en/latest/|CliMAF]]: a Climate Model Assessment Framework | * [[https://climaf.readthedocs.io/en/latest/|CliMAF]]: a Climate Model Assessment Framework | ||
* [[other:uvcdat:cdat_conda:climaf_install|Installation notes]] | * [[other:uvcdat:cdat_conda:climaf_install|Installation notes]] | ||
- | |||
- | * [[https://jiffyclub.github.io/palettable/|Palettable]]: Color palettes for Python | ||
- | * ''conda install -n cdatm19_py3 -c conda-forge palettable'' | ||
==== TODO ==== | ==== TODO ==== | ||
Line 332: | Line 321: | ||
FIXME: need to clean the list below, and move the packages to the ordered list further down | FIXME: need to clean the list below, and move the packages to the ordered list further down | ||
- | * [[http://www.ifremer.fr/vacumm/|vacumm]]: Validation, Analysis, Comparison - Utilities written in Python to validate and analyze Multi-Model outputs, and compare them to observations | ||
* [[http://unidata.github.io/netcdf4-python/|netcdf4]]: a Python interface to the netCDF C library | * [[http://unidata.github.io/netcdf4-python/|netcdf4]]: a Python interface to the netCDF C library | ||
* [[https://github.com/PBrockmann/ipython_ferretmagic|ipython_ferretmagic]]: IPython notebook extension for ferret | * [[https://github.com/PBrockmann/ipython_ferretmagic|ipython_ferretmagic]]: IPython notebook extension for ferret | ||
- | * [[https://github.com/conda-forge/basemap-data-hires-feedstock|basemap-data-hires]]: high resolution data for ''basemap'' | ||
* [[https://github.com/PCMDI/pcmdi_metrics|PCMDI metrics package]] (PMP): objectively compare results from climate models with observations using well-established statistical tests | * [[https://github.com/PCMDI/pcmdi_metrics|PCMDI metrics package]] (PMP): objectively compare results from climate models with observations using well-established statistical tests | ||
* [[https://xlsxwriter.readthedocs.io/|XlsxWriter]]: a Python module for creating Excel XLSX files | * [[https://xlsxwriter.readthedocs.io/|XlsxWriter]]: a Python module for creating Excel XLSX files | ||
Line 341: | Line 328: | ||
* [[https://earthsystemcog.org/projects/wip/CMIP6DataRequest|dreqPy]]: CMIP6 Data Request Python API | * [[https://earthsystemcog.org/projects/wip/CMIP6DataRequest|dreqPy]]: CMIP6 Data Request Python API | ||
* [[https://cmor.llnl.gov/|CMOR]]: CMOR (//Climate Model Output Rewriter//) is used to produce CF-compliant netCDF files | * [[https://cmor.llnl.gov/|CMOR]]: CMOR (//Climate Model Output Rewriter//) is used to produce CF-compliant netCDF files | ||
- | * [[https://github.com/pierre-rouanet/dtw|dtw]]: DTW (Dynamic Time Warping) python module | ||
* [[http://toblerity.org/shapely/project.html|shapely]]: a Python wrapper for GEOS for algebraic manipulation of geometry (manipulation and analysis of geometric objects in the Cartesian plane) | * [[http://toblerity.org/shapely/project.html|shapely]]: a Python wrapper for GEOS for algebraic manipulation of geometry (manipulation and analysis of geometric objects in the Cartesian plane) | ||
- | * [[http://scitools.org.uk/cartopy/|cartopy]]: a Python package designed for geospatial data processing in order to produce maps and other geospatial data analyses | ||
- | * [[https://github.com/stefraynaud/spanlib|spanlib]]: Spectral Analysis Library | ||
* [[https://ajdawson.github.io/windspharm/latest/|windspharm]]: spherical harmonic wind analysis in Python | * [[https://ajdawson.github.io/windspharm/latest/|windspharm]]: spherical harmonic wind analysis in Python | ||
* [[https://climaf.readthedocs.io/en/latest/|CliMAF]]: a Climate Model Assessment Framework | * [[https://climaf.readthedocs.io/en/latest/|CliMAF]]: a Climate Model Assessment Framework | ||
Line 355: | Line 339: | ||
+ | * [[https://matplotlib.org/basemap/|basemap]]: a library for plotting 2D data on maps in Python | ||
+ | * [[missing|basemap-data]] and [[https://github.com/conda-forge/basemap-data-hires-feedstock|basemap-data-hires]]: (high resolution) data for ''basemap'' | ||
+ | * [[http://scitools.org.uk/cartopy/|cartopy]]: a Python package designed for geospatial data processing in order to produce maps and other geospatial data analyses | ||
+ | * see also ''iris'' | ||
* [[https://github.com/Try2Code/cdo-bindings|python-cdo]]: Python scripting interface of [[https://code.mpimet.mpg.de/projects/cdo/wiki/Cdo#Documentation|cdo]] | * [[https://github.com/Try2Code/cdo-bindings|python-cdo]]: Python scripting interface of [[https://code.mpimet.mpg.de/projects/cdo/wiki/Cdo#Documentation|cdo]] | ||
* ''python-cdo'' will install the ''cdo'' package (providing the ''cdo ''executable) as a dependency | * ''python-cdo'' will install the ''cdo'' package (providing the ''cdo ''executable) as a dependency | ||
- | * see also [[https://code.mpimet.mpg.de/projects/cdo/wiki/Cdo%7Brbpy%7D#Python|Using CDO from python or ruby]] | + | * see also [[https://code.mpimet.mpg.de/projects/cdo/wiki/Cdo%7Brbpy%7D|Using CDO from python or ruby]] |
- | * [[http://matplotlib.org/cmocean/|cmocean]]: beautiful colormaps for oceanography | + | * [[https://github.com/ecmwf/cfgrib|cfgrib]]: Python interface to map GRIB files to the Unidata's Common Data Model v4 following the CF Conventions |
+ | * see also ''eccodes'' and ''pygrib'' | ||
+ | * [[https://clustergram.readthedocs.io/|clustergram]]: visualization and diagnostics for cluster analysis | ||
+ | * [[https://www.fabiocrameri.ch/colourmaps/|cmcrameri]]: Crameri's Scientific colour maps ''[color]'' | ||
+ | * the colormaps are also available in [[https://jiffyclub.github.io/palettable/scientific/|palettable.scientific]] | ||
+ | * [[http://matplotlib.org/cmocean/|cmocean]]: beautiful colormaps for oceanography ''[color]'' | ||
+ | * [[https://dynamictimewarping.github.io/|python-dtw]]: implementation of Dynamic Time Warping-type (DTW) | ||
+ | * Note: older installations provided [[https://github.com/pierre-rouanet/dtw|dtw]]: DTW (Dynamic Time Warping) python module | ||
+ | * [[https://confluence.ecmwf.int/display/ECC/|ecCodes]]: ecCodes is a package developed by ECMWF which provides an application programming interface and a set of tools for decoding and encoding GRIB and BUFR messages | ||
+ | * need to install ''eccodes'' and ''python-eccodes'' ([[https://github.com/ecmwf/eccodes-python/issues/56|details]]) | ||
+ | * see also ''cfgrib'' and ''pygrib'' | ||
+ | * [[https://docs.esmvaltool.org/en/latest/|ESMValTool]]: Earth System Model Evaluation Tool | ||
+ | * not installed yet (March 2022) because it required download of 302 Mb extra packages and a downgrade of gdal | ||
* [[http://ferret.pmel.noaa.gov/Ferret/documentation/pyferret|pyferret]] and ''ferret_datasets'': Ferret encapsulated in Python | * [[http://ferret.pmel.noaa.gov/Ferret/documentation/pyferret|pyferret]] and ''ferret_datasets'': Ferret encapsulated in Python | ||
* OSGeo/[[http://www.gdal.org/|gdal]]: Geospatial Data Abstraction Library. GDAL is a translator library for raster and vector geospatial data formats | * OSGeo/[[http://www.gdal.org/|gdal]]: Geospatial Data Abstraction Library. GDAL is a translator library for raster and vector geospatial data formats | ||
Line 364: | Line 364: | ||
* [[https://github.com/TEOS-10/GSW-python|gsw]]: Python implementation of the Thermodynamic Equation Of Seawater | * [[https://github.com/TEOS-10/GSW-python|gsw]]: Python implementation of the Thermodynamic Equation Of Seawater | ||
* see also //seawater// | * see also //seawater// | ||
+ | * [[https://intake.readthedocs.io/|intake]]: a lightweight package for finding, investigating, loading and disseminating data | ||
+ | * [[https://intake-esm.readthedocs.io/|intake-esm]]: data cataloging utility built on top of intake, pandas, and xarray | ||
+ | * [[https://scitools-iris.readthedocs.io/en/stable/|iris]]: a powerful, format-agnostic, community-driven Python package for analysing and visualising Earth science data | ||
+ | * see also ''cartopy'' | ||
+ | * install sample data with ''conda install -c conda-forge iris-sample-data'' | ||
* [[https://joblib.readthedocs.io/en/latest/|joblib]]: running Python functions as pipeline jobs | * [[https://joblib.readthedocs.io/en/latest/|joblib]]: running Python functions as pipeline jobs | ||
+ | * [[https://gitlab.com/remikz/nccmp|nccmp]]: compare two NetCDF files bitwise, semantically or with a user defined tolerance (absolute or relative percentage) | ||
+ | * can probably also be done with [[https://code.mpimet.mpg.de/projects/cdo/embedded/index.html#x1-580002.1.3|cdo diff]] | ||
* [[https://opencv.org/|OpenCV]]: OpenCV (Open Source Computer Vision Library) is an open-source library that includes several hundreds of computer vision algorithms. See also [[https://www.geeksforgeeks.org/opencv-python-tutorial/|OpenCV Python Tutorial]] and **scikit-image** | * [[https://opencv.org/|OpenCV]]: OpenCV (Open Source Computer Vision Library) is an open-source library that includes several hundreds of computer vision algorithms. See also [[https://www.geeksforgeeks.org/opencv-python-tutorial/|OpenCV Python Tutorial]] and **scikit-image** | ||
- | * [[https://jiffyclub.github.io/palettable/|Palettable]]: Color palettes for Python | + | * [[https://jiffyclub.github.io/palettable/|Palettable]]: Color palettes for Python ''[color]'' |
* [[http://pandas.pydata.org/|pandas]]: Python Data Analysis Library | * [[http://pandas.pydata.org/|pandas]]: Python Data Analysis Library | ||
* [[https://geopandas.org/|geopandas]]: an open source project to make working with geospatial data in python easier | * [[https://geopandas.org/|geopandas]]: an open source project to make working with geospatial data in python easier | ||
* [[https://peakutils.readthedocs.io/|PeakUtils]]: utilities related to the detection of peaks on 1D data | * [[https://peakutils.readthedocs.io/|PeakUtils]]: utilities related to the detection of peaks on 1D data | ||
* [[https://python-pillow.org/|pillow]]: the friendly PIL (//Python Imaging Library//) fork | * [[https://python-pillow.org/|pillow]]: the friendly PIL (//Python Imaging Library//) fork | ||
+ | * [[https://proplot.readthedocs.io/en/latest/|proplot]]: a lightweight **matplotlib wrapper** for making beautiful, publication-quality graphics | ||
+ | * [[https://github.com/jswhit/pygrib|pygrib]]: high-level interface to the ECWMF ECCODES C library for reading GRIB files | ||
+ | * see also ''eccodes'' and ''cfgrib'' | ||
+ | * [[https://pyleoclim-util.readthedocs.io/|pyleoclim]]: a Python package designed for the analysis of paleoclimate data | ||
+ | * **Wait** till it can be installed with ''conda'' ([[https://github.com/LinkedEarth/Pyleoclim_util/discussions/205|Why I'm not installing Pyleoclim yet]]) | ||
* [[https://rpy2.github.io/|rpy2]]: an interface to R running embedded in a Python process | * [[https://rpy2.github.io/|rpy2]]: an interface to R running embedded in a Python process | ||
- | * [[http://scikit-image.org/|scikit-image]]: image processing in Python | + | * [[http://scikit-image.org/|scikit-image]]: a collection of algorithms for image processing in Python |
- | * [[https://scikit-learn.org/stable/index.html|scikit-learn]]: Machine Learning in Python | + | * [[https://scikit-learn.org/|scikit-learn]]: an open source machine learning library that supports supervised and unsupervised learning. It also provides various tools for model fitting, data preprocessing, model selection and evaluation, and many other utilities. |
* [[https://seaborn.pydata.org/|seaborn]]: statistical data visualization | * [[https://seaborn.pydata.org/|seaborn]]: statistical data visualization | ||
* [[http://pythonhosted.org/seawater/|seawater]]: Python re-write of the CSIRO seawater toolbox | * [[http://pythonhosted.org/seawater/|seawater]]: Python re-write of the CSIRO seawater toolbox | ||
* see also //gsw// | * see also //gsw// | ||
- | * [[http://statsmodels.sourceforge.net/|statsmodels]]: a Python module that allows users to explore data, estimate statistical models, and perform statistical tests | + | * [[https://www.statsmodels.org/|statsmodels]]: a Python module that provides classes and functions for the estimation of many different statistical models, as well as for conducting statistical tests, and statistical data exploration. |
+ | * [[https://github.com/SatAgro/suntime|suntime]]: simple sunset and sunrise time calculation python library | ||
+ | * **Warning**: not available in conda, use ''pip install suntime'' | ||
+ | * [[https://www.tensorflow.org/|tensorflow-mkl]]: an end-to-end open source machine learning platform | ||
+ | * [[https://anaconda.org/anaconda/tensorflow-mkl|tensorflow-mkl]] will install the **CPU**-based (**not GPU**) version | ||
+ | * [[https://xcdat.readthedocs.io/|xcdat]]: Xarray Extended with Climate Data Analysis Tools | ||
+ | * [[https://xoa.readthedocs.io/en/latest/|xoa]]: xarray-based ocean analysis library | ||
+ | * ''xoa'' is the successor of [[http://www.ifremer.fr/vacumm/|vacumm]] (vacumm does **not** support Python3) | ||
==== Removed packages ==== | ==== Removed packages ==== | ||
Line 383: | Line 402: | ||
* NO removed packages! | * NO removed packages! | ||
+ | ==== Packages NOT supported in Python3 ==== | ||
+ | |||
+ | * [[https://github.com/stefraynaud/spanlib|spanlib]]: Spectral Analysis Library | ||
+ | * [[http://www.ifremer.fr/vacumm/|vacumm]]: Validation, Analysis, Comparison - Utilities written in Python to validate and analyze Multi-Model outputs, and compare them to observations | ||
+ | * Check the ''xoa'' replacement package | ||
===== Environments summary ===== | ===== Environments summary ===== | ||