Differences

This shows you the differences between two versions of the page.

--- other:uvcdat:cdat_conda:cdat_2024_03 [2024/03/13 17:00] – created jypeter
+++ other:uvcdat:cdat_conda:cdat_2024_03 [2025/08/27 11:09] (current) – [Extra packages list] added pynco jypeter
@@ Line 47: / Line 47: @@
 Read the next sections of this web page
-There are some slight differences if you want to install this distribution:
+There are some slight differences depending on how and where you want to install this distribution:
-  * Only for you. This is the usual case. Well, if you break something you can always remove everything and start from scratch
+  * **Only for you**\\ This is the common case. Well, if you break something you can always remove everything and start from scratch
-  * For sharing a stable version with several other people. You should be careful not to break the installion!
+  * For **sharing a stable version with several other people**\\ You should be careful not to break a shared installation!
+    * Be extremely careful with packages that require downgrading other packages/dependencies (especially downgrading key packages like ''numpy'' and ''matplotlib'').
+    * Try to avoid packages only based on ''pip''
 The notes at the beginning of each section will tell you if you can skip the section if you are just installing for you
-===== What's New in CDAT 8.2.1? =====
+===== What's New in JYP's 2024-03 distribution? =====
-  * [[https://github.com/CDAT/cdat/releases/tag/v8.2.1|8.2.1 announcement]] and features summary ([[https://github.com/UV-CDAT/uvcdat/releases|all versions]])
+  * This is the first distribution not //tied// to CDAT packages
-  * Full [[https://uvcdat.llnl.gov/changelog.html#8.2.1|Change log]]
-Note: this particular CDAT installation at LSCE (and on other machines/servers) provides **many extra (non-CDAT) packages**. You can jump directly to the [[#extra_packages_list|Extra package list]] at the bottom of this page, if you want to see what is available
+  * There are lots (and lots) of extra packages that all have their own history. You can always visit the [[#extra_packages_list|Extra package list]] on this page, to get news from individual packages
 ===== Installation with Miniconda3 =====
@@ Line 66: / Line 67: @@
 ==== Installing Miniconda3 ====
-If //Miniconda3// is not already installed, or if you want to start over with a brand new version, read [[other:uvcdat:cdat_conda:miniconda3_install|JYP steps for installing Miniconda3]].
+If //Miniconda3// is not already installed, or if you want to start over with a brand new (or clean) version of //Miniconda3//, read [[other:uvcdat:cdat_conda:miniconda3_install|JYP steps for installing Miniconda3]].
@@ Line 318: / Line 319: @@
 ===== Extra packages list =====
+<note important>
+Do not forget to add the following packages to the future environments\\ \\
+  * [[https://anaconda.org/conda-forge/cxx-compiler/|cxx-compiler]]: a metapackage to obtain a C++ compiler
+  * [[https://f90nml.readthedocs.io/|f90nml]]: a Fortran namelist parser, generator, and editor
+  * [[https://flake8.pycqa.org/en/stable/|flake8]]: your Tool For Style Guide Enforcement
+  * [[https://loguru.readthedocs.io/|Loguru]]: a library which aims to bring enjoyable logging in Python
+  * [[https://pythonot.github.io/|pot]]: Python Optimal Transport\\ This open source Python library provides several solvers for optimization problems related to Optimal Transport for signal, image processing and machine learning
+  * [[https://pybind11.readthedocs.io/|pybind11]]: a lightweight header-only library that exposes C++ types in Python and vice versa
+  * [[https://docs.astral.sh/ruff/|ruff]]: An extremely fast Python linter and code formatter, written in Rust
+  * [[https://xrft.readthedocs.io/|xrft]]: Fourier transforms for xarray data
+  * [[https://zarr.readthedocs.io/|zarr]]: a file storage format for chunked, compressed, N-dimensional arrays based on an open-source specification
+</note>
 FIXME: need to clean the list below, and move the packages to the ordered list further down
@@ Line 325: / Line 341: @@
   * [[https://xlsxwriter.readthedocs.io/|XlsxWriter]]: a Python module for creating Excel XLSX files
     * Note: <wrap hi>this is a dependency of ''dreqPy''</wrap>
+    * See also ''pandas'', ''openpyxl'', ''xlrd''
   * [[https://earthsystemcog.org/projects/wip/CMIP6DataRequest|dreqPy]]: CMIP6 Data Request Python API
   * [[https://cmor.llnl.gov/|CMOR]]:  CMOR (//Climate Model Output Rewriter//) is used to produce CF-compliant netCDF files
@@ Line 345: / Line 362: @@
     * ''python-cdo'' will install the ''cdo'' package (providing the ''cdo ''executable) as a dependency
     * see also [[https://code.mpimet.mpg.de/projects/cdo/wiki/Cdo%7Brbpy%7D|Using CDO from python or ruby]]
+    * See also ''nco'', ''pynco''
   * [[https://cds.climate.copernicus.eu/api-how-to|cdsapi]]: The Climate Data Store (CDS) Application Program Interface (API) is a service providing programmatic access to CDS data
     * CDS = Copernicus [[https://cds.climate.copernicus.eu/|Climate Data Store]]
@@ Line 358: / Line 376: @@
     * the colormaps are also available in [[https://jiffyclub.github.io/palettable/scientific/|palettable.scientific]]
   * [[http://matplotlib.org/cmocean/|cmocean]]: beautiful colormaps for oceanography ''[color]''
+  * [[https://github.com/kelvinou01/conda-which|conda-which]]: find the package owning a file
+    * See the [[https://github.com/conda/conda/issues/14026|Is there an easy way to find the package "owning" a file ?]] GH issue
+  * [[https://www.cvxpy.org/|cvxpy]]: an open source Python-embedded modeling language for convex optimization problems
   * [[https://dash.plotly.com/|dash]] and [[https://github.com/plotly/jupyter-dash|jupyter-dash]]: the original low-code framework for rapidly building data apps in Python, R, Julia, and F#
     * see also ''plotly''
+  * [[https://docs.dask.org/|dask]]: a Python library for parallel and distributed computing
+    * dask.[[https://distributed.dask.org/|distributed]]: lightweight library for distributed computing in Python
+    * See also: flox
   * [[https://github.com/man-group/dtale|D-Tale]] brings you an easy way to view & analyze Pandas data structures. It integrates seamlessly with ipython notebooks & python/ipython terminals.
     * Install with: ''conda install dtale -c conda-forge''
@@ Line 385: / Line 409: @@
     * See also //scikit-learn//
   * [[https://github.com/TEOS-10/GSW-python|gsw]]: Python implementation of the Thermodynamic Equation Of Seawater
-    * see also //seawater//
+    * See also //seawater//
   * [[https://noaa-orr-erd.github.io/gridded/index.html|gridded]]: a single API for accessing / working with gridded model results on multiple grid types
     * Supports the [[http://cfconventions.org/|CF]], [[https://ugrid-conventions.github.io/ugrid-conventions/|UGRID]] and [[http://sgrid.github.io/sgrid/|SGRID]] conventions
+  * [[https://uber.github.io/h3-py/|h3-py]]: Uber’s H3 Hexagonal Hierarchical Geospatial Indexing System in Python
+    * Additional reading: [[https://towardsdatascience.com/uber-h3-for-data-analysis-with-python-1e54acdcc908|Uber H3 for Data Analysis with Python]]
+  * [[https://github.com/mapado/haversine|haversine]]: calculate the distance (in various units) between two points on Earth using their latitude and longitude
+    * See also ''[[https://scikit-learn.org/stable/modules/generated/sklearn.metrics.pairwise.haversine_distances.html|haversine_distances]] @ scikit-learn''
   * [[https://icclim.readthedocs.io/|icclim]]: icclim (Index Calculation for CLIMate) is a Python library to compute climate indices
   * [[https://intake.readthedocs.io/|intake]]: a lightweight package for finding, investigating, loading and disseminating data
@@ Line 404: / Line 432: @@
   * [[https://gitlab.com/remikz/nccmp|nccmp]]: compare two NetCDF files bitwise, semantically or with a user defined tolerance (absolute or relative percentage)
     * can probably also be done with [[https://code.mpimet.mpg.de/projects/cdo/embedded/index.html#x1-580002.1.3|cdo -v diffn file1.nc file2.nc]]
+  * [[https://nco.sourceforge.net/|nco]]: The NCO toolkit manipulates and analyzes data stored in netCDF-accessible formats
+    * Note: this only installs the [[https://nco.sourceforge.net/#Definition|nco executables]] (''ncks'', ''ncatted'', ...)! It is not really a Python package
+    * See also ''pynco'', ''cdo''
+  * [[https://github.com/pydata/numexpr|NumExpr]]: Fast numerical expression evaluator for NumPy
   * [[https://opencv.org/|OpenCV]]: OpenCV (Open Source Computer Vision Library) is an open-source library that includes several hundreds of computer vision algorithms. See also [[https://www.geeksforgeeks.org/opencv-python-tutorial/|OpenCV Python Tutorial]] and **scikit-image**
   * [[https://openpyxl.readthedocs.io/|openpyxl]]: a Python library to read/write Excel 2010 ''xlsx''/''xlsm'' files
+    * See also ''pandas'', ''XlsxWriter'', ''xlrd''
   * [[https://jiffyclub.github.io/palettable/|Palettable]]: Color palettes for Python ''[color]''
   * [[http://pandas.pydata.org/|pandas]]: Python Data Analysis Library
@@ Line 414: / Line 447: @@
     * see also ''python-kaleido'' and ''dash''
   * [[https://www.fatiando.org/pooch/|pooch]]: a friend to fetch your data files (makes it easy to download a file, without messing with ''requests'' and ''urllib'')
-  * [[https://proplot.readthedocs.io/en/latest/|proplot]]: a lightweight **matplotlib wrapper** for making beautiful, publication-quality graphics
+  * //proplot//: a lightweight matplotlib wrapper for making beautiful, publication-quality graphics
+    * **Replaced by ''ultraplot''**
   * [[https://psyplot.github.io/|psyplot]]: Interactive Data Visualization from Python and GUIs
+  * [[https://www.pygmt.org/latest/|PyGMT]]: a Python interface for the [[https://www.generic-mapping-tools.org/|Generic Mapping Tools]]
   * [[https://github.com/jswhit/pygrib|pygrib]]: high-level interface to the ECWMF ECCODES C library for reading GRIB files
     * see also ''eccodes'' and ''cfgrib''
   * [[https://pyleoclim-util.readthedocs.io/|pyleoclim]]: a Python package designed for the analysis of paleoclimate data
     * **Wait** till it can be installed with ''conda'' ([[https://github.com/LinkedEarth/Pyleoclim_util/discussions/205|Why I'm not installing Pyleoclim yet]])
+  * [[https://www.pymoo.org/|pymoo]]: multi-objective Optimization in Python
+  * [[https://pynco.readthedocs.io/|pynco]]:Python bindings for NCO
+    * See also ''nco'', ''cdo'', ''python-cdo''
   * [[https://requests.readthedocs.io/|requests]]: is an elegant and simple HTTP library for Python, built for human beings
     * See also ''pooch''
@@ Line 437: / Line 475: @@
   * [[https://www.tensorflow.org/|tensorflow-mkl]]: an end-to-end open source machine learning platform
     * [[https://anaconda.org/anaconda/tensorflow-mkl|tensorflow-mkl]] will install the **CPU**-based (**not GPU**) version
-  * [[https://github.com/tqdm|tqdm]]: make your loops show a smart progress meter
+  * [[https://github.com/tqdm/tqdm|tqdm]]: make your loops show a smart progress meter
+  * [[https://ultraplot.readthedocs.io/en/latest/|UltraPlot]]: a succinct matplotlib wrapper for making beautiful, publication-quality graphics
   * [[https://uxarray.readthedocs.io/|uxarray]]: provide xarray styled functionality for unstructured grid datasets following [[https://ugrid-conventions.github.io/ugrid-conventions/|UGRID Conventions]]
   * [[https://docs.xarray.dev/en/stable/|xarray]]: Xarray makes working with labelled multi-dimensional arrays in Python simple, efficient, and fun!
@@ Line 446: / Line 485: @@
   * [[https://xgrads.readthedocs.io/|xgrads]]: parse and read binary dataset described by a ''.ctl'' file commonly used by [[http://cola.gmu.edu/grads/|GrADS]] or [[http://www.opengrads.org/|openGrADS]]
   * [[https://xlrd.readthedocs.io/|xlrd]]: a library for reading data and formatting information from Excel files in the historical .xls format
-  * [[https://xoa.readthedocs.io/en/latest/|xoa]]: xarray-based ocean analysis library
+    * See also ''pandas'', ''XlsxWriter'', ''openpyxl''
-    * ''xoa'' is the successor of [[http://www.ifremer.fr/vacumm/|vacumm]] (vacumm does **not** support Python3)
+  * [[https://xoa.readthedocs.io/|xoa]]: xarray-based ocean analysis library
+    * ''xoa'' is the successor of [[https://umr-lops-vacumm.ifremer.fr/|vacumm]] (vacumm does **not** support Python3)
   * [[https://xskillscore.readthedocs.io/|xskillscore]]: metrics for verifying forecasts
   * [[https://docs.profiling.ydata.ai/|ydata-profiling]]: a leading package for data profiling, that automates and standardizes the generation of detailed reports, complete with statistics and visualizations.