other:python:jyp_steps
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
|
other:python:jyp_steps [2023/12/15 10:25] jypeter [YData Profiling] Added D-Tale |
other:python:jyp_steps [2023/12/15 14:45] jypeter [Data file formats] Added link to the NetCDF section |
||
|---|---|---|---|
| Line 317: | Line 317: | ||
| ===== Data analysis ===== | ===== Data analysis ===== | ||
| + | |||
| + | ==== EDA (Exploratory Data Analysis) ? ==== | ||
| + | |||
| + | <note tip> | ||
| + | The //EDA concept// seems to apply to **time series** (and tabular data), which is not exactly the case of full climate model output data</note> | ||
| + | |||
| + | * [[https://www.geeksforgeeks.org/what-is-exploratory-data-analysis/|What is Exploratory Data Analysis ?]] | ||
| + | * //The method of studying and exploring record sets to apprehend their predominant traits, discover patterns, locate outliers, and identify relationships between variables. EDA is normally carried out as a preliminary step before undertaking extra formal statistical analyses or modeling.// | ||
| + | |||
| + | * [[https://medium.com/codex/automate-the-exploratory-data-analysis-eda-to-understand-the-data-faster-not-better-2ed6ff230eed|Automate the exploratory data analysis (EDA) to understand the data faster and easier]]: a nice comparison of some Python libraries listed below ([[#ydata_profiling|YData Profiling]], [[#d-tale|D-Tale]], [[#sweetviz|sweetviz]], [[#autoviz|AutoViz]]) | ||
| + | |||
| + | * [[https://www.geeksforgeeks.org/exploratory-data-analysis-in-python/|EDA in Python]] | ||
| + | |||
| ==== Easy to use datasets ==== | ==== Easy to use datasets ==== | ||
| Line 337: | Line 350: | ||
| * [[https://github.com/xCDAT/xcdat/issues/277|xCDAT test data GH discussion]] | * [[https://github.com/xCDAT/xcdat/issues/277|xCDAT test data GH discussion]] | ||
| + | |||
| + | |||
| ==== Pandas ==== | ==== Pandas ==== | ||
| Line 359: | Line 374: | ||
| Note: check the example in the [[https://lectures.scientific-python.org/packages/statistics/index.html|Statistics in Python]] tutorial | Note: check the example in the [[https://lectures.scientific-python.org/packages/statistics/index.html|Statistics in Python]] tutorial | ||
| + | |||
| + | |||
| + | ==== scikit-learn ==== | ||
| + | |||
| + | [[http://scikit-learn.org/|scikit-learn]] is a Python library for machine learning, and is one of the most widely used tools for supervised and unsupervised machine learning. Scikit–learn provides an easy-to-use, consistent interface to a large collection of machine learning models, as well as tools for model evaluation and data preparation | ||
| + | |||
| + | Note: check the example in [[https://lectures.scientific-python.org/packages/scikit-learn/index.html|scikit-learn: machine learning in Python]] | ||
| + | |||
| + | |||
| + | ==== scikit-image ==== | ||
| + | |||
| + | [[https://scikit-image.org/|scikit-image]] is a collection of algorithms for image processing in Python | ||
| + | |||
| + | Note: check the example in [[https://lectures.scientific-python.org/packages/scikit-image/index.html|scikit-image: image processing]] | ||
| Line 369: | Line 398: | ||
| [[https://github.com/man-group/dtale|D-Tale]] brings you an easy way to view & analyze Pandas data structures. It integrates seamlessly with ipython notebooks & python/ipython terminals. | [[https://github.com/man-group/dtale|D-Tale]] brings you an easy way to view & analyze Pandas data structures. It integrates seamlessly with ipython notebooks & python/ipython terminals. | ||
| - | ==== scikit-learn ==== | ||
| - | [[http://scikit-learn.org/|scikit-learn]] is a Python library for machine learning, and is one of the most widely used tools for supervised and unsupervised machine learning. Scikit–learn provides an easy-to-use, consistent interface to a large collection of machine learning models, as well as tools for model evaluation and data preparation | ||
| - | Note: check the example in [[https://lectures.scientific-python.org/packages/scikit-learn/index.html|scikit-learn: machine learning in Python]] | + | ==== Sweetviz ==== |
| - | ==== scikit-image ==== | + | |
| - | [[https://scikit-image.org/|scikit-image]] is a collection of algorithms for image processing in Python | + | [[https://github.com/fbdesignpro/sweetviz|Sweetviz]] is pandas based Python library that generates beautiful, high-density visualizations to kickstart EDA (Exploratory Data Analysis) with just two lines of code. |
| - | Note: check the example in [[https://lectures.scientific-python.org/packages/scikit-image/index.html|scikit-image: image processing]] | + | |
| + | ==== AutoViz ==== | ||
| + | |||
| + | [[https://github.com/AutoViML/AutoViz|AutoViz]]: the One-Line Automatic Data Visualization Library. Automatically Visualize any dataset, any size with a single line of code | ||
| ===== Data file formats ===== | ===== Data file formats ===== | ||
| - | We list here some resources about non-NetCDF data formats that can be useful | + | * We list below some resources about **non-NetCDF data formats** that can be useful |
| + | |||
| + | * Check the [[#using_netcdf_files_with_python|Using NetCDF files with Python]] section otherwise | ||
| ==== The shelve package ==== | ==== The shelve package ==== | ||
other/python/jyp_steps.txt · Last modified: 2024/08/27 12:51 by jypeter
