Menu
▾
▴
#1467 Fit using named columns not working
Based on the question is gnuplot fit influenced by terminal? on stackoverflow, I saw that fit has troubles when used with named columns (using "A":"B"). Consider the following example:
A data file test.dat:
A B 1 2 2 3 3 4
and a test script test.gp:
f(x) = a*x**2 + b*x + c fit f(x) 'test.dat' using "A":"B" via a,b,c
If I call this script with gnuplot test.gp (version 4.6.5), I get the error
Undefined value during function evaluation "test.gp", line 2: error during fit
This error also appears in the current cvs version, but says "test.gp", line 2: unknown type in real().
Works for me in 4.6.5:
chauvet [16] gnuplot_4.6.5 G N U P L O T Version 4.6 patchlevel 5 last modified 2014-01-31 Build System: Linux i686 Copyright (C) 1986-1993, 1998, 2004, 2007-2014 Thomas Williams, Colin Kelley and many others gnuplot home: http://www.gnuplot.info faq, bugs, etc: type "help FAQ" immediate help: type "help" (plot window: hit 'h') Terminal type set to 'qt' gnuplot> f(x) = a*x**2 + b*x + c gnuplot> fit f(x) 'fit.dat' using "A":"B" via a,b,c Iteration 0 WSSR : 99 delta(WSSR)/WSSR : 0 delta(WSSR) : 0 limit for stopping : 1e-05 lambda : 3.10913 initial set of free parameter values a = 1 b = 1 c = 1 / Iteration 1 WSSR : 1.46711 delta(WSSR)/WSSR : -66.4797 delta(WSSR) : -97.5329 limit for stopping : 1e-05 lambda : 0.310913 resultant parameter values a = 0.197601 b = 0.740398 c = 0.838381 / Iteration 2 WSSR : 0.0939966 delta(WSSR)/WSSR : -14.6081 delta(WSSR) : -1.37311 limit for stopping : 1e-05 lambda : 0.0310913 resultant parameter values a = -0.154874 b = 1.72749 c = 0.215874 / Iteration 3 WSSR : 0.0500004 delta(WSSR)/WSSR : -0.879916 delta(WSSR) : -0.0439961 limit for stopping : 1e-05 lambda : 0.00310913 resultant parameter values a = -0.249704 b = 2.049 c = 0.0505011 / Iteration 4 WSSR : 0.05 delta(WSSR)/WSSR : -8.34878e-06 delta(WSSR) : -4.17439e-07 limit for stopping : 1e-05 lambda : 0.000310913 resultant parameter values a = -0.25 b = 2.05 c = 0.05 After 4 iterations the fit converged. final sum of squares of residuals : 0.05 rel. change during last iteration : -8.34878e-06 degrees of freedom (FIT_NDF) : 1 rms of residuals (FIT_STDFIT) = sqrt(WSSR/ndf) : 0.223607 variance of residuals (reduced chisquare) = WSSR/ndf : 0.05 Final set of parameters Asymptotic Standard Error ======================= ========================== a = -0.25 +/- 0.1118 (44.72%) b = 2.05 +/- 0.35 (17.07%) c = 0.05 +/- 0.2179 (435.9%) correlation matrix of the fit parameters: a b c a 1.000 b -0.958 1.000 c 0.513 -0.688 1.000"fit" seems to be broken in current cvs, possibly for multiple reasons.
I get this, which is very strange:
chauvet [25] gnuplot G N U P L O T Version 5.0 patchlevel rc1 last modified 2014-08-18 Copyright (C) 1986-1993, 1998, 2004, 2007-2014 Thomas Williams, Colin Kelley and many others gnuplot home: http://www.gnuplot.info mailing list: gnuplot-beta@lists.sourceforge.net faq, bugs, etc: type "help FAQ" immediate help: type "help" (plot window: hit 'h') Terminal type set to 'qt' gnuplot> f(x) = a*x**2 + b*x + c gnuplot> fit f(x) 'fit.dat' using "A":"B" via a,b,c fit: Deprecated syntax. Consider using the 'noerror' option, see `help fit`. unknown type in real() gnuplot> fit f(x) 'fit.dat' using "A":"B" via a,b,c fit: Deprecated syntax. Consider using the 'noerror' option, see `help fit`. iter chisq delta/lim lambda a b c 0 1.7900000000e+02 0.00e+00 4.14e+00 1.000000e+00 1.000000e+00 1.000000e+00 1 1.2053169496e+00 -1.48e+07 4.14e-01 1.841282e-01 7.303407e-01 9.145820e-01 2 1.6893526288e-03 -7.12e+07 4.14e-02 3.510949e-02 8.709961e-01 1.081227e+00 3 2.5976370942e-06 -6.49e+07 4.14e-03 1.855649e-03 9.922286e-01 1.006848e+00 4 1.3895598403e-12 -1.87e+11 4.14e-04 1.351612e-06 9.999943e-01 1.000005e+00 5 7.7965843453e-23 -1.78e+15 4.14e-05 1.011377e-11 1.000000e+00 1.000000e+00 6 6.0628299301e-25 -1.28e+07 4.14e-06 -5.995341e-13 1.000000e+00 1.000000e+00 * 7.5806712814e-25 2.00e+04 4.14e-05 1.019208e-12 1.000000e+00 1.000000e+00 * 7.5806712814e-25 2.00e+04 4.14e-04 1.019207e-12 1.000000e+00 1.000000e+00 * 7.5870610547e-25 2.01e+04 4.14e-03 1.019130e-12 1.000000e+00 1.000000e+00 * 7.4719425828e-25 1.89e+04 4.14e-02 1.011488e-12 1.000000e+00 1.000000e+00 7 1.8939544537e-25 -2.20e+05 4.14e-03 5.092287e-13 1.000000e+00 1.000000e+00 * 4.0834753670e-25 5.36e+04 4.14e-02 -6.549797e-13 1.000000e+00 1.000000e+00 8 1.5277316949e-25 -2.40e+04 4.14e-03 -3.626229e-13 1.000000e+00 1.000000e+00 9 4.5986843685e-26 -2.32e+05 4.14e-04 -6.645745e-14 1.000000e+00 1.000000e+00 10 3.1846314744e-27 -1.34e+06 4.14e-05 -6.645745e-14 1.000000e+00 1.000000e+00 * 2.8885660602e-25 9.89e+04 4.14e-04 -2.684881e-14 1.000000e+00 1.000000e+00 * 2.8885660602e-25 9.89e+04 4.14e-03 -2.684888e-14 1.000000e+00 1.000000e+00 * 2.8885660602e-25 9.89e+04 4.14e-02 -2.685599e-14 1.000000e+00 1.000000e+00 * 2.7792141615e-25 9.89e+04 4.14e-01 -2.755134e-14 1.000000e+00 1.000000e+00 * 5.3184410584e-26 9.40e+04 4.14e+00 -4.931128e-14 1.000000e+00 1.000000e+00 * 4.3505678923e-27 2.68e+04 4.14e+01 -6.357443e-14 1.000000e+00 1.000000e+00 * 3.2280188242e-27 1.34e+03 4.14e+02 -6.641108e-14 1.000000e+00 1.000000e+00 11 3.1846314744e-27 0.00e+00 4.14e+01 -6.645698e-14 1.000000e+00 1.000000e+00 iter chisq delta/lim lambda a b c After 11 iterations the fit converged. final sum of squares of residuals : 3.18463e-27 rel. change during last iteration : 0 degrees of freedom (FIT_NDF) : 1 rms of residuals (FIT_STDFIT) = sqrt(WSSR/ndf) : 5.64325e-14 variance of residuals (reduced chisquare) = WSSR/ndf : 3.18463e-27 Final set of parameters Asymptotic Standard Error ======================= ========================== a = -6.6457e-14 +/- 3.961e-14 (59.6%) b = 1 +/- 6.912e-14 (6.912e-12%) c = 1 +/- 1.872e-13 (1.872e-11%) correlation matrix of the fit parameters: a b c a 1.000 b 0.870 1.000 c -0.900 -0.985 1.000i.e. it fails the first time and then works the second time.
Yes, I also observed that the
fitworks when called for the second time in interactive mode. It also works for me if I usef(x) = a*x + bas fitting function. All in all quite strange.The problem with columnheaders is now fixed in CVS for 4.6, 5.0.rc2, and for the main branch (which as of today is relabeled as 5.1).
So I will close this bug, but there are clearly other problems with the CVS version of "fit" (see Bug #1466). It seems that certain commands cause it to mis-count the number of independent variables.
Last edit: Ethan Merritt 2014-08-21
That fit is badly conditioned for at least two reasons:
1) There are so few data points that it's not actually a fitting problem: it's a solve-this-system-of-equations problem. 'fit' is not really meant for that task, and it will have problems with it.
2) the actual solution in this case would be a=0, b=1, c=1. But parameters being zero is not tolerated well by 'fit' (the numerical derivative breaks down).
That is at least part of the reason why changing the function to be fitted makes a difference.
The real problem, however, appears to be that at least in the 4.6pl5 version, 'fit' wasn't properly adapted to handle column-name "using" specs: it got the count of data points wrong. NDF is actually zero here (three parameters from three datapoints), but 'fit' states it was 1. It's not a major stretch of the imagination that there's random garbage data participating in the fit, because of that.
Yes, the data was probably not choosen very well, maybe too minimal. But also using more data points, e.g. a
test.datwithleads to the same error.
Current CVS works well for me, also using the not-so-minimal original data set.
Thanks