Re: [Gnuplot-info] Help request - linear regression giving screwy results

[Eric R. <es...@th...>]

Thomas Sefzick <t.s...@fz...>:
> to make the fit algorithm happy, apply a factor of e.g. 1.e-7 to 'm'
> 
> f(x) = 1.e-7 * m *x + b
> 
> and the fit algorithm will converge after a few iterations because
> the effect of a step in 'm' or 'b' on the deviation of the regression
> function from the data is now nearly equal.
> 
> not necessary, but also helpful: shift the origin of the regression
> function into the x-range defined by the data points (=subtract
> 10 years from 'x'), this makes the fit algorithm faster.

OK, not a floating-point misconvergence but something more subtle.  
Let's see if I can recast this into a FAQ emtry.  Criticize, please.

-----------------------------------------------------------------------------
Q: I'm using time xdata and my curve-fit produces absurd results. How do I 
fix this?

A: Change the coefficients on your fit function so that incrementing any
of them produces a difference in the value of y that is about the same as
incrementing any other.

Say, for example, that you are trying a linear fit on some time-based 
data that looks like this:

2009-12-00 5.2
2011-03-00 34.7

Because dates in gnuplot are calculated as 'seconds since beginning
of the gnuplot epoch' which is '2000-01-01 00:00:00', your data
actually looks to the curve-fitter like this:

312940800.0 5.2
352252800.0 34.7

It's the large difference in magnitude between x and y values that causes 
the problem.  The fit algorithm will make small steps in the 1.e-8 range
when varying 'm', but large steps in the 1.e-1 range when varying
'b'. This difference in step size makes it (nearly) impossible for the
fit algorithm to converge properly, so it will end up with a regression
function which crosses your data points but with a wrong slope.

To fix this, apply a scale factor to the m coefficient that makes values of
mx comparable in magnitude to values of y.  In this case fitting to 
f(x) = 1.e-7 * m *x + b will work much better.

Shifting the origin of the regression function to inside the x-range defined 
by the data points is not necessary for convergence, but it will make the fit 
algorithm run faster.  In this case, try subtracting 10 years, like so:

offset=10*365*24*60*60
f(x)=1.e-7*m*(x-offset)+b

The fix (and the optimization) generalizes to nonlinear fit functions.
The curve-fitting algorithm is going to walk through tuples of
coefficients looking for a tuples with small residuals.  If there are
large enough variations in the magnitude of changes in y as different
coefficience are tweaked, that search may converge on a wrong tuple.
To fix this, tweak the function so the search has roughly equal grain
in all directions.
-----------------------------------------------------------------------------

-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>