Linear Discriminant Analysis

[realamirhe]

First of all thanks for the great reference, you've been created and it performs well in its current format.

But, Is it acceptable to use covariance matrices instead of scatter matrices in LDA?
shouldn't it use scatter matrices?

ML-From-Scratch/mlfromscratch/supervised_learning/linear_discriminant_analysis.py

Lines 24 to 25 in a2806c6

	cov1 = calculate_covariance_matrix(X1)
	cov2 = calculate_covariance_matrix(X2)

As we know the relation between these two matrices is
scatter(X) = X.T.dot(X)
covariance(X) = X.T.dot(X) / N
for a given X or X = X - mean(X) and N = |X|

reference

[realamirhe]

in LDA, we can refactor the predict method by NumPy built-ins to a more readable and performant version

ML-From-Scratch/mlfromscratch/supervised_learning/linear_discriminant_analysis.py

Lines 37 to 43 in a2806c6

	def predict(self, X):
	y_pred = []
	for sample in X:
	h = sample.dot(self.w)
	y = 1 * (h < 0)
	y_pred.append(y)
	return y_pred

is equal to this

def predict(self, X):
    return np.array([1 * (x.dot(self._w) < 0) for x in X], dtype=np.int)

which can be implemented like this

def predict(self, X):
    return np.where(X.dot(self._w) < 0, 1, 0)