Whitening, sharpening & smoothing

Sharpening or distribution sharpening

This process increases the probabilities of more probable classes and decreases the probabilities of less probable classes while maintaining a valid probability distribution. It is often used in machine learning and statistics to enhance the distinction between different classes or outcomes.

Smoothing

The opposite of distribution sharpening is “smoothing” or “distribution smoothing.” This process reduces the differences between probabilities of different classes, making the distribution less extreme or more uniform. It is often used to address issues like overfitting in machine learning or to account for uncertainties in statistical models.

Whitening - also known as sphering or decorrelation

Whitening, also known as sphering or decorrelation, is a preprocessing technique used in signal processing, machine learning, and statistical analysis. It aims to transform the input data such that it has an identity covariance matrix, meaning the transformed features are uncorrelated and have unit variance.

The primary purpose of whitening is to remove any correlations present in the data and to normalize the scale of different features to aid in further analysis, modeling, or processing. By doing this, it helps improve the performance and convergence of learning algorithms.

Whitening can be achieved in several ways, including principal component analysis (PCA), ZCA (zero-phase component analysis), and other linear transformations. The process generally involves calculating the covariance matrix of the input data, computing its eigenvectors and eigenvalues, and applying a transformation that results in uncorrelated, unit-variance features.