Re: fit, NLLS and Levenberg–Marquardt

[<pl...@pi...>]

On 05/24/11 02:08, Ethan Merritt wrote:
> On Monday, May 23, 2011 01:12:23 am pl...@pi... wrote:
>>> if there is an alternative approach for the case when x
>>> uncertainty can't be ignored, keep the discussion going and maybe we can
>>> add such a feature to the list of items we'd like to add in the future.
>>>
>>> Dan
>>
>> Yes , I would love to propose that. Some kind of total least squares may
>> be an option I don't know enough about that to make a concrete proposal.
>
> It is standard in my field to use instead a maximum likelihood residual
> that allows for separate error distributions on x and y.  The maximum
> likelihood treatment reduces to a weighted least squares treatment
> if and only if the distribution of errors on both x and y are
> (1) Gaussian and (2) of equal magnitude.
>
> Gnuplot allows for input of precalculated non-uniform weights on y,
> which partially addresses (2).  But there is no provision for non-Gaussian
> errors on y, and no provision for any error model at all on x.
>
> If the errors in your data do not follow the simple Gaussian model,
> then almost certainly you would be better off using maximum likelihood
> rather than least-squares.  But in order to do so you need first a model
> for the error distributions.  That may be obvious for any particular
> experiment or source of data, but it's difficult to impossible for a
> general-purpose program to figure it out for you. You're the one who
> knows the source of the data, so it's up to you to provide an appropriate
> explicit error model along with the data.
>
> The point is that switching to a better minimization residual is more
> than just a matter of changing the internal code.  It would require a
> more complex description of your data that includes error models for
> both the independent and dependent variables.
>
> NB:  When I say "x", I really mean the full set of independent variables
> [x1,x2,x3,...].   "y" is the single dependent variable estimated by
> f(x1,x2,x3,...)
>
> 	Ethan
>

Thanks for all that detail Ethan.

It seems likely that this may be the job for external processing by some 
stats capable package rather than a job for a plotting tool.

That brings us back to the original point of this thread : the idea of 
putting a specific warning about the limitations of least squares 
techniques used by gnuplot into the help text for "fit".

I won't bore everyone with anecdotes but I never cease to be amazed by 
the widespread ignorance of this issue , even at PhD level.

For the cost of a couple of lines it would be good if at least gnuplot 
user were made aware of it.

best regards.Peter.