This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
other:python:jyp_steps [2023/12/15 13:54] jypeter Added Sweetviz and AutoViz |
other:python:jyp_steps [2023/12/15 14:41] jypeter [EDA (Exploratory Data Analysis) ?] Added internal links to some libraries' sections |
||
---|---|---|---|
Line 317: | Line 317: | ||
===== Data analysis ===== | ===== Data analysis ===== | ||
+ | |||
+ | ==== EDA (Exploratory Data Analysis) ? ==== | ||
+ | |||
+ | <note tip> | ||
+ | The //EDA concept// seems to apply to **time series** (and tabular data), which is not exactly the case of full climate model output data</note> | ||
+ | |||
+ | * [[https://www.geeksforgeeks.org/what-is-exploratory-data-analysis/|What is Exploratory Data Analysis ?]] | ||
+ | * //The method of studying and exploring record sets to apprehend their predominant traits, discover patterns, locate outliers, and identify relationships between variables. EDA is normally carried out as a preliminary step before undertaking extra formal statistical analyses or modeling.// | ||
+ | |||
+ | * [[https://medium.com/codex/automate-the-exploratory-data-analysis-eda-to-understand-the-data-faster-not-better-2ed6ff230eed|Automate the exploratory data analysis (EDA) to understand the data faster and easier]]: a nice comparison of some Python libraries listed below ([[#ydata_profiling|YData Profiling]], [[#d-tale|D-Tale]], [[#sweetviz|sweetviz]], [[#autoviz|AutoViz]]) | ||
+ | |||
+ | * [[https://www.geeksforgeeks.org/exploratory-data-analysis-in-python/|EDA in Python]] | ||
+ | |||
==== Easy to use datasets ==== | ==== Easy to use datasets ==== | ||
Line 337: | Line 350: | ||
* [[https://github.com/xCDAT/xcdat/issues/277|xCDAT test data GH discussion]] | * [[https://github.com/xCDAT/xcdat/issues/277|xCDAT test data GH discussion]] | ||
+ | |||
+ | |||
==== Pandas ==== | ==== Pandas ==== | ||
Line 359: | Line 374: | ||
Note: check the example in the [[https://lectures.scientific-python.org/packages/statistics/index.html|Statistics in Python]] tutorial | Note: check the example in the [[https://lectures.scientific-python.org/packages/statistics/index.html|Statistics in Python]] tutorial | ||
+ | |||
+ | |||
+ | ==== scikit-learn ==== | ||
+ | |||
+ | [[http://scikit-learn.org/|scikit-learn]] is a Python library for machine learning, and is one of the most widely used tools for supervised and unsupervised machine learning. Scikit–learn provides an easy-to-use, consistent interface to a large collection of machine learning models, as well as tools for model evaluation and data preparation | ||
+ | |||
+ | Note: check the example in [[https://lectures.scientific-python.org/packages/scikit-learn/index.html|scikit-learn: machine learning in Python]] | ||
+ | |||
+ | |||
+ | ==== scikit-image ==== | ||
+ | |||
+ | [[https://scikit-image.org/|scikit-image]] is a collection of algorithms for image processing in Python | ||
+ | |||
+ | Note: check the example in [[https://lectures.scientific-python.org/packages/scikit-image/index.html|scikit-image: image processing]] | ||
Line 379: | Line 408: | ||
[[https://github.com/AutoViML/AutoViz|AutoViz]]: the One-Line Automatic Data Visualization Library. Automatically Visualize any dataset, any size with a single line of code | [[https://github.com/AutoViML/AutoViz|AutoViz]]: the One-Line Automatic Data Visualization Library. Automatically Visualize any dataset, any size with a single line of code | ||
- | ==== scikit-learn ==== | ||
- | |||
- | [[http://scikit-learn.org/|scikit-learn]] is a Python library for machine learning, and is one of the most widely used tools for supervised and unsupervised machine learning. Scikit–learn provides an easy-to-use, consistent interface to a large collection of machine learning models, as well as tools for model evaluation and data preparation | ||
- | |||
- | Note: check the example in [[https://lectures.scientific-python.org/packages/scikit-learn/index.html|scikit-learn: machine learning in Python]] | ||
- | ==== scikit-image ==== | ||
- | |||
- | [[https://scikit-image.org/|scikit-image]] is a collection of algorithms for image processing in Python | ||
- | |||
- | Note: check the example in [[https://lectures.scientific-python.org/packages/scikit-image/index.html|scikit-image: image processing]] | ||