README.md

Nirpy Research

Snippets and data from the blog of Nirpy Research.

Data

This folder contains the original datasets used in some of the posts.

milk.csv

This dataset is used in the following posts:

• Classification of NIR spectra by Principal Components Analysis

milk-powder.csv

Used in the following posts:

• Classification of NIR Spectra by Linear Discriminant Analysis in Python
• PLS Discriminant Analysis for binary classification in Python

peach_spectra+brixvalues.csv

This dataset is used in the following posts:

• Principal Component Regression in Python
• Partial Least Square Regression in Python
• A Variable Selection Method for PLS in Python
• Two Scatter Correction Techniques for NIR Spectroscopy in Python
• Exporting NIR Regression Models Built in Python
• Principal Component Regression in Python Revisited
• Principal Component Selection with a Greedy Algorithm
• Principal Component Selection with Simulated Annealing
• Minimal prediction models for linear regression
• The Akaike Information Criterion for model selection
• Parallel computation of loops for cross-validation analysis

plums.csv

Used in the following posts:

• Detecting outliers using the Mahalanobis distance with PCA in Python
• PCA and kernel PCA explained

Honey2021.csv

Used in the following posts:

• Aquagrams with Python and Matplotlib

Raman_spectra_mixtures.csv

Used in the following posts:

• Multivariate curve resolution: an introduction

ABS_plastic.csv

Used in the following posts:

• Optimal spectra smoothing with Fourier ring correlation

Incombustible Material NIR.csv

Kindly provided by S. D'Hyon. About 60 samples with varying "incombustible content". Mix is coal and calcium carbonate. The data is raw and unmodified using a ChemWiz-ADK NIR spectrometer.

Snippets

Whatever piece of code that can be of general use or will not make it in the last versions of the posts will be (in time) posted here.

• bootstrap.py - Data splitter implementing the Bootstrap cross-validation method. This is not currently available in scikit-learn. This class is consistent with scikit-learn usage in a limited case. For more info on how to use this class, read the tutorial K-fold and Montecarlo cross-validation vs Bootstrap: a primer.
• Scatter Correction - Jupyter notebook associated with the post: Two Scatter Correction Techniques for NIR Spectroscopy in Python.
• Basic PLS Regression - Jupyter notebook associated with the Partial Least Square Regression in Python post.
• PCA vs Kernel PCA - Jupyter notebook with the code described in the PCA and kernel PCA explained post.
• Simulated annealing - Companion Jupyter notebook to the simulated annealing post.