DiscreteGaussianProcess is conceptually unsound

[marcelluethi]

The DiscreteGaussianProcess is a finite view obtained by sampling a finite number of points from a GaussianProcess. This is only sound if the sampling is done uniformly, and with sufficiently many points such that the characteristics of the GP can be represented. However, the current implementation does not make sure that this is the case.

Problems that appear include:

• The basis functions are not necessarily orthogonal
• The probabilities (e.g. logpdf, pdf) of the distribution is dependent on the sampling

[marcelluethi]

A potential solution is now available thanks to the PivotedCholesky approximation. It should be possible to apply the PivotedCholesky decomposition on the kernel of the Gaussian process, in order to achieve a proper orthogonalization of the basis.