gnuplot-beta

fit parameter error bug

[<ha...@on...>]

Apparently a formal bug report was not filed for the issues addressed by
Thomas Mattison in patch 1445064, "Gnuplot fitting improvements" so the
bugs he found were not considered after his patch was determined to be 
not ready to drop into the 4.2 release.

One issue relative to the 4.2 release is that the estimates of 
parameter errors produced by fit are incorrectly identified.

The fit discussion in gnuplot.doc (fit error statistical overview) and 
the fit output labels clearly identify the quantities as "asymptotic 
standard errors" (ASE) which are precisely defined and are the "errors" 
conventionally reported by other NLLS fitting programs.

However the fit code scales the ASE values, I'll denote the result 
as SASE, and outputs the SASE values as ASE without indicating that
they have been scaled, nor does gnuplot.doc mention the scaling or
give any justification for the selected scaling factor.


Possible fixes are
1) omit the scaling (comment out 2 lines) so that the values are 
as defined and indicate in the documentation the former reporting
of scaled ASE.
2) change the output labels to SASE and document the basis for 
applying the scaling factor. 
3) add a FIT control to let the user select ASE or SASE and
appropriately label the output (as did Mattison) and extend the
discussion as for (2).


My preference would be (1), and leave it to the user determine the 
significance of the ASE for their application, rather than (2) or (3) 
for which one would want documented 
- a definition of what gnuplot is attempting to estimate by scaling 
  the ASE,
- the basis for the selected scaling factor,
- the validity of the scale factor when extending its application 
  beyond the one case in which the High Energy Physics Particle Data 
  Group uses such a scaling factor (it is their convention for 
  reporting an estimate of the error in an average of values from 
  different  experiments - a single parameter, linear least squares 
  problem.) 


I have attached patches to fit.c and to gnuplot.doc for (1).

(I did not include a reference for the PDG practice, described in 
the Introduction to the 'Review of Particle Physics', available on-line
  http://pdg.lbl.gov/2006/reviews/textrpp.pdf
The section on confidence limits in the Statistics chaper
  http://pdg.lbl.gov/2006/reviews/statrpp.pdf
is more relevant for the general case.)

The fit.c patch also changes a label in the fit output 
 'final sum of squares of residuals :'
which Mattison reported some found confusing to
 'final sum of squares of (wtd) residuals :''

(The sum is of weighted residuals, with unit weights in the case
of unweighted analyses.)

Also the gih indexing of fit subtopics is flawed in both 4.1.0 and 4.2rc4.

Subtopics available for fit:
    adjustable_parameters               beginners_guide   control
    error             error_estimates   errors            guide
    multi-branch      parameters        starting_values   tips

where error, error_estimates, and errors are synonyms.  However selecting
'Subtopic of fit: error'  gives the text for the level 4 subtopic 
'practical guidelines' while 'error_estimates' or  'errors' give text
for the level 3 'error estimates'

The level 3 subtopic does not list the level 4 subtopics, 'practical 
guidelines' and  'statistical overview'.

The gnuplot.doc fit patch also deletes the 'errors' subtopic, retains 
'error' and 'error_estimates' as synonyms, and causes the two level 4 
subtopics to be listed when either is selected, but someone may have 
some other preference for subtopic names.

- Lucas Hart
  Oregon State University

Re: fit parameter error bug

[Ethan A M. <merritt@u.washington.edu>]

On Saturday 03 March 2007 00:21, Lucas Hart wrote:
> 
> Apparently a formal bug report was not filed for the issues addressed by
> Thomas Mattison in patch 1445064, "Gnuplot fitting improvements" so the
> bugs he found were not considered after his patch was determined to be 
> not ready to drop into the 4.2 release.

OK. Then please file this on SourceForge as a bug report, rather 
than risk its getting lost again.

As to the help message problems, I wonder if these aren't additional
examples of a problem in the help processing code.  Several other
examples turned up recently, and there are a couple of patchsets
outstanding that re-work some of the help message code paths.

	Ethan

> One issue relative to the 4.2 release is that the estimates of 
> parameter errors produced by fit are incorrectly identified.
> 
> The fit discussion in gnuplot.doc (fit error statistical overview) and 
> the fit output labels clearly identify the quantities as "asymptotic 
> standard errors" (ASE) which are precisely defined and are the "errors" 
> conventionally reported by other NLLS fitting programs.
> 
> However the fit code scales the ASE values, I'll denote the result 
> as SASE, and outputs the SASE values as ASE without indicating that
> they have been scaled, nor does gnuplot.doc mention the scaling or
> give any justification for the selected scaling factor.
> 
> 
> Possible fixes are
> 1) omit the scaling (comment out 2 lines) so that the values are 
> as defined and indicate in the documentation the former reporting
> of scaled ASE.
> 2) change the output labels to SASE and document the basis for 
> applying the scaling factor. 
> 3) add a FIT control to let the user select ASE or SASE and
> appropriately label the output (as did Mattison) and extend the
> discussion as for (2).
> 
> 
> My preference would be (1), and leave it to the user determine the 
> significance of the ASE for their application, rather than (2) or (3) 
> for which one would want documented 
> - a definition of what gnuplot is attempting to estimate by scaling 
>   the ASE,
> - the basis for the selected scaling factor,
> - the validity of the scale factor when extending its application 
>   beyond the one case in which the High Energy Physics Particle Data 
>   Group uses such a scaling factor (it is their convention for 
>   reporting an estimate of the error in an average of values from 
>   different  experiments - a single parameter, linear least squares 
>   problem.) 
> 
> 
> I have attached patches to fit.c and to gnuplot.doc for (1).
> 
> (I did not include a reference for the PDG practice, described in 
> the Introduction to the 'Review of Particle Physics', available on-line
>   http://pdg.lbl.gov/2006/reviews/textrpp.pdf
> The section on confidence limits in the Statistics chaper
>   http://pdg.lbl.gov/2006/reviews/statrpp.pdf
> is more relevant for the general case.)
> 
> The fit.c patch also changes a label in the fit output 
>  'final sum of squares of residuals :'
> which Mattison reported some found confusing to
>  'final sum of squares of (wtd) residuals :''
> 
> (The sum is of weighted residuals, with unit weights in the case
> of unweighted analyses.)
> 
> Also the gih indexing of fit subtopics is flawed in both 4.1.0 and 4.2rc4.
> 
> Subtopics available for fit:
>     adjustable_parameters               beginners_guide   control
>     error             error_estimates   errors            guide
>     multi-branch      parameters        starting_values   tips
> 
> where error, error_estimates, and errors are synonyms.  However selecting
> 'Subtopic of fit: error'  gives the text for the level 4 subtopic 
> 'practical guidelines' while 'error_estimates' or  'errors' give text
> for the level 3 'error estimates'
> 
> The level 3 subtopic does not list the level 4 subtopics, 'practical 
> guidelines' and  'statistical overview'.
> 
> The gnuplot.doc fit patch also deletes the 'errors' subtopic, retains 
> 'error' and 'error_estimates' as synonyms, and causes the two level 4 
> subtopics to be listed when either is selected, but someone may have 
> some other preference for subtopic names.
> 
> - Lucas Hart
>   Oregon State University
> 

-- 
Ethan A Merritt
Biomolecular Structure Center
University of Washington, Seattle 98195-7742