This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
other:python:jyp_steps [2021/09/22 14:23] jypeter Added statsmodels, scikit-learn and scikit-image |
other:python:jyp_steps [2023/03/29 13:38] jypeter [Extra numpy information] Corrected links |
||
---|---|---|---|
Line 123: | Line 123: | ||
==== Extra numpy information ==== | ==== Extra numpy information ==== | ||
+ | |||
+ | <WRAP center round tip 60%> | ||
+ | You can also check the [[other:python:misc_by_jyp#numpy_related_stuff|numpy section]] of the //Useful python stuff// page | ||
+ | </WRAP> | ||
+ | |||
* More information about **array indexing**:\\ <wrap em>Always check what you are doing on a simple test case, when you use advanced/fancy indexing!</wrap> | * More information about **array indexing**:\\ <wrap em>Always check what you are doing on a simple test case, when you use advanced/fancy indexing!</wrap> | ||
Line 128: | Line 133: | ||
* {{ :other:python:indirect_indexing_2.py.txt |}}: Take a vertical slice in a 3D zyx array, along a varying y 'path' | * {{ :other:python:indirect_indexing_2.py.txt |}}: Take a vertical slice in a 3D zyx array, along a varying y 'path' | ||
* [[https://numpy.org/doc/stable/user/basics.indexing.html|Array indexing basics (user guide)]] (//index arrays//, //boolean index arrays//, //np.newaxis//, //Ellipsis//, //variable numbers of indices//, ...) | * [[https://numpy.org/doc/stable/user/basics.indexing.html|Array indexing basics (user guide)]] (//index arrays//, //boolean index arrays//, //np.newaxis//, //Ellipsis//, //variable numbers of indices//, ...) | ||
- | * [[https://numpy.org/doc/stable/reference/arrays.indexing.html|Array indexing (reference manual)]] | + | * [[https://numpy.org/doc/stable/reference/arrays.indexing.html|Indexing routines (reference manual)]] |
* [[https://numpy.org/doc/stable/user/quickstart.html#advanced-indexing-and-index-tricks|Advanced indexing and index tricks]] and [[https://numpy.org/doc/stable/user/quickstart.html#the-ix-function|the ix_() function]] | * [[https://numpy.org/doc/stable/user/quickstart.html#advanced-indexing-and-index-tricks|Advanced indexing and index tricks]] and [[https://numpy.org/doc/stable/user/quickstart.html#the-ix-function|the ix_() function]] | ||
- | * [[https://numpy.org/doc/stable/reference/routines.indexing.html#routines-indexing|Indexing routines]] | ||
* More information about arrays: | * More information about arrays: | ||
* [[https://numpy.org/doc/stable/reference/routines.array-creation.html|Array creation routines]] | * [[https://numpy.org/doc/stable/reference/routines.array-creation.html|Array creation routines]] | ||
* [[https://numpy.org/doc/stable/reference/routines.array-manipulation.html|Array manipulation routines]] | * [[https://numpy.org/doc/stable/reference/routines.array-manipulation.html|Array manipulation routines]] | ||
+ | * [[https://numpy.org/doc/stable/reference/routines.sort.html|Sorting, searching, and counting routines]] | ||
* [[https://numpy.org/doc/stable/reference/maskedarray.html|Masked arrays]] | * [[https://numpy.org/doc/stable/reference/maskedarray.html|Masked arrays]] | ||
* [[https://numpy.org/doc/stable/reference/routines.ma.html|Masked array operations]] | * [[https://numpy.org/doc/stable/reference/routines.ma.html|Masked array operations]] | ||
* [[https://numpy.org/doc/stable/user/misc.html#ieee-754-floating-point-special-values|Dealing with special numerical values]] (//Nan//, //inf//) | * [[https://numpy.org/doc/stable/user/misc.html#ieee-754-floating-point-special-values|Dealing with special numerical values]] (//Nan//, //inf//) | ||
* If you know that your data has missing values, it is cleaner and safer to handle them with [[https://numpy.org/doc/stable/reference/maskedarray.html|masked arrays]]! | * If you know that your data has missing values, it is cleaner and safer to handle them with [[https://numpy.org/doc/stable/reference/maskedarray.html|masked arrays]]! | ||
+ | * If you know that some of your data //may// have masked values, play safe by explicitly using ''np.ma.some_function()'' rather than just ''np.some_function()'' | ||
+ | * More details in the [[https://github.com/numpy/numpy/issues/18675|Why/when does np.something remove the mask of a np.ma array ?]] discussion | ||
* [[https://numpy.org/doc/stable/user/misc.html#how-numpy-handles-numerical-exceptions|Handling numerical exceptions]] | * [[https://numpy.org/doc/stable/user/misc.html#how-numpy-handles-numerical-exceptions|Handling numerical exceptions]] | ||
* [[https://numpy.org/doc/stable/reference/routines.err.html|Floating point error handling]] | * [[https://numpy.org/doc/stable/reference/routines.err.html|Floating point error handling]] | ||
- | ===== NetCDF files: using cdms2, xarray and netCDF4 ===== | + | ===== Using NetCDF files with Python ===== |
- | There is a good chance that your input array data will come from a file in the [[other:newppl:starting#netcdf_and_file_formats|NetCDF format]]. | + | <note tip>People using CMIPn and model data on the IPSL servers can easily search and process NetCDF files using: |
+ | * the [[https://climaf.readthedocs.io/|Climate Model Assessment Framework (CliMAF)]] environment | ||
+ | * and the [[https://github.com/jservonnat/C-ESM-EP/wiki|CliMAF Earth System Evaluation Platform (C-ESM-EP)]] | ||
+ | </note> | ||
- | Depending on which [[other:python:starting#some_python_distributions|python distribution]] you are using, you can use the //cdms2//, //xarray// or //netCDF4// modules to read the data. | + | * There is a good chance that your input array data will be stored in a [[other:newppl:starting#netcdf_and_related_conventions|NetCDF]] file. |
+ | |||
+ | * There may be different ways of dealing with NetCDF files, depending on which [[other:python:starting#some_python_distributions|python distribution]] you have access to | ||
==== cdms2 ==== | ==== cdms2 ==== | ||
Line 160: | Line 172: | ||
Summary: [[http://xarray.pydata.org/en/stable/|xarray]] is an open source project and Python package that makes working with labelled multi-dimensional arrays simple, efficient, and fun! [...] It is particularly tailored to working with netCDF files | Summary: [[http://xarray.pydata.org/en/stable/|xarray]] is an open source project and Python package that makes working with labelled multi-dimensional arrays simple, efficient, and fun! [...] It is particularly tailored to working with netCDF files | ||
+ | |||
+ | === Some xarray related resources === | ||
+ | |||
+ | Note: more packages (than listed below) may be listed in the [[other:uvcdat:cdat_conda:cdat_8_2_1#extra_packages_list|Extra packages list]] | ||
+ | |||
+ | * [[https://xcdat.readthedocs.io/|xcdat]]: xarray extended with Climate Data Analysis Tools | ||
+ | |||
+ | * [[https://xoa.readthedocs.io/en/latest/|xoa]]: xarray-based ocean analysis library | ||
+ | |||
+ | * [[https://uxarray.readthedocs.io/|uxarray]]: provide xarray styled functionality for unstructured grid datasets following [[https://ugrid-conventions.github.io/ugrid-conventions/|UGRID Conventions]] | ||
+ | |||