About

Popular family of methods called local regression that helps fitting non-linear functions just focusing locally on the data.

LOESS and LOWESS (locally weighted scatterplot smoothing) are two strongly related non-parametric regression methods that combine multiple regression models in a k-nearest-neighbor-based meta-model.

“LOESS” is a later generalization of LOWESS; although it is not a true initialism, it may be understood as standing for “LOcal regrESSion”.

locally weighted polynomial regression

loess gives a better appearance, but is O(n^2) in memory, so does not work for larger datasets.

Procedure

A linear function is fitted only on a local set of point delimited by a region. The polynomial is fitted using weighted least squares. The weights are given by the heights of the kernel (the weighting function) giving:

  • more weight to points near the target point (x) whose response is being estimated
  • and less weight to points further away.

We obtain then a fitted polynomial model but retains only the point of the model at the target point (x). The target point then moves away on the x axis and the procedure repeats and that traces out the orange curve.

Local Regession Loess

where:

  • The orange curve is the fitted function.
  • The points in the region are the orange one.
  • The kernel is a Gaussian distribution.

See also this gif

Advantage

Boundaries

Loess gives a much better extrapolation at the boundaries.

Documentation / Reference