#1511 Error bars on fit parameters are are not 1-sigma gaussian errors

[Ed Daw]

In most fitting packages, the +/- errors quoted on the fit parameters are the 1-sigma errors. However, in gnuplot version 4.6.6, the errors derived from diagonal elements of the covariance matrix are scaled by a factor of the square root of chisquared per degree of freedom before being displayed. I think this in an attempt to take account of 'poor' fits in the error bars on fit parameters, but this 'method' is not at all standard, and I think rather dubious. The offending lines of code are in fit.c, lines 792-794. This rather arbitrary kludge, which I've never seen before and am pretty sure is wrong, isn't documented, and will deceive users who wish to derive errors on quantities derived from the fit parameters. Chisquared/DOF is subject to rather large trial-to-trial fluctuations and using it to scale the error bars will lead large trial-to-trial variations in the estimated error bars on fit parameters. Furthermore, this same scaling isn't applied to the off-diagonal elements of the covariance matrix, so that the off-diagonal elements of the Pearson correlation given with the fit are inconsistent with the quoted +/- 'errors'. This makes it difficult to infer the error bars on quantities combining the best fit values of different fit parameters.

I'm pretty sure that the best fix is simply to delete lines 792-794 of fit.c. Or, at least, provide a command line flag to turn off this scaling, maybe gnuplot -nofitsigmascaling or something like that. To my mind, though, the DEFAULT behaviour should be NOT to scale the covariance matrix errors by a factor of chisquared/DOF.

I checked this conclusion using comparisons with results derived from a linear fit algorithm derived from the methods in Bevington, 'data analysis for the physical sciences' - I will be happy to send details of my tests to anyone interested in fixing this bug. I've used gnuplot a lot and I certainly appreciate it's existence and functionality - this is my way of paying the developer group back for a few years of service that you have rendered me.

Do tell me, though, if you think I am wrong about this.

Cheers, Ed Daw
Senior Lecturer in Physics, The University of Sheffield

[Ethan Merritt]

Please always mention the version of gnuplot your bug report refers to. In this case I assume it is version 4.6.something since otherwise the line numbers you cite make no sense.

This behavior is now (current cvs source, version 5-rc2) under the control of the command:

  set fit {no}errorscale

Here is the ChangeLog entry:

2012-05-20  Alexander Täschner  <taschna@users.sourceforge.net>

        * src/fit.c|h (regress)
        * src/save.c (save_set_all)
        * src/set.c (set_fit)
        * src/show.c (show_fit)
        * src/unset.c (unset_fit)
        * docs/gnuplot.doc: New option "set fit [no]errorscaling" which
        allows to toggle scaling of parameter errors. The default remains
        to scale errors.  Patch #3375473, Feature Request #2956524.

The fit command has changed in several ways for version 5. If you have the time to check out the release candidate for version 5, any comments are welcome. Note that the command line syntax for fitting with separate error estimates on the data values has been further tweaked since -rc2 as a result of earlier feedback. Gory details in the ChangeLog from cvs.