Re: fit, NLLS and Levenberg–Marquardt

[Daniel J S. <dan...@ie...>]

On 05/21/2011 03:15 AM, pl...@pi... wrote:
> Hi,
>
> 'help fit' reports that the fit command uses Levenberg–Marquardt algo to
> do the fit.
>
> I think this raises an important question that very often over-looked by
> many users of least-squares techniques even at maths PhD level.
>
> Such techniques often only optimised the y error rather than the
> perpendicular error from the line. This is implicitly assuming y
> uncertainty>>  x uncertainty.  While this condition is often satisfied
> in a controlled experiment there are many situations where this is not
> applicable and gets totally overlooked.
>
> A common case is scatter plots which are frequently used to seek a
> relations between two quantities , each with significant errors /
> uncertainties.
>
> In this situation the fitted line is "wrong". In fact it's the
> application that is wrong , hence the wrong result. This may or may not
> be apparent to the eye.
>
> I have seen this happen so many times (including once in a PhD thesis
> report!) that I think it needs a serious health warning in the doc.
>
> "Warning: using least-squares inappropriately can seriously damage your
> reputation". ;)
>
> Firstly , could you confirm the basis on which this algo is applied in
> gnuplot? Does it only optimise vertical y residuals?

Often it is an assumption that the independent variables are exact 
measurements.  Not true, typically, but if the variance is small and 
homoscedastic, the two can probably be lumped together.  I.e., we are 
searching for a relationship:

Y = f(X + eps1) + eps2
   ~= f(X) + C eps1 + residual + eps2
   ~= f(X) + (C eps1 + eps2)

where hopefully the residual due to nonlinearity of the relationship is 
small compared to other randomness.  It's up to the user's judgment and 
knowledge of the application to determine that.

Anyway, your point is true of most software packages: details are so 
often lacking.  That's why it would be nice to have a set of white 
papers to go along with the software so that people know exactly what 
the algorithm is, both for the benefit of the user and other developers. 
  Most of the time it is "here's a hunk of code, use it at your own risk".

Dan