Statistics - Standard Least Squares Fit (Guassian linear model)

> (Statistics|Probability|Machine Learning|Data Mining|Data and Knowledge Discovery|Pattern Recognition|Data Science|Data Analysis)

1 - About

In a least squares, the coefficients are found in order to make RSS as small as possible.

When p is be much bigger than n (the number of samples), we can't use full least squares, because the solution's not even defined.

Legendre published the method of least squares in 1805.


3 - Scale-invariant

Standard least squares is scale-invariant The scaling of the variable doesn't matter because if a feature is multiplied by a constant, the coefficient can be divided by the same constant in order to get the same target.

Whether a length is measured in feet or inches is not going to matter because the coefficient can just account for the change in units.

4 - Ordinary

Ordinary Least Square (OLS) is a least square procedure performs on the the raw predictors.

5 - Documentation / Reference