You can subscribe to this list here.
2001 
_{Jan}

_{Feb}
(1) 
_{Mar}

_{Apr}

_{May}

_{Jun}

_{Jul}
(1) 
_{Aug}

_{Sep}

_{Oct}

_{Nov}

_{Dec}


2002 
_{Jan}
(1) 
_{Feb}

_{Mar}

_{Apr}

_{May}

_{Jun}

_{Jul}
(1) 
_{Aug}
(1) 
_{Sep}

_{Oct}

_{Nov}
(1) 
_{Dec}

2003 
_{Jan}

_{Feb}

_{Mar}

_{Apr}

_{May}

_{Jun}

_{Jul}
(1) 
_{Aug}
(1) 
_{Sep}

_{Oct}
(83) 
_{Nov}
(57) 
_{Dec}
(111) 
2004 
_{Jan}
(38) 
_{Feb}
(121) 
_{Mar}
(107) 
_{Apr}
(241) 
_{May}
(102) 
_{Jun}
(190) 
_{Jul}
(239) 
_{Aug}
(158) 
_{Sep}
(184) 
_{Oct}
(193) 
_{Nov}
(47) 
_{Dec}
(68) 
2005 
_{Jan}
(190) 
_{Feb}
(105) 
_{Mar}
(99) 
_{Apr}
(65) 
_{May}
(92) 
_{Jun}
(250) 
_{Jul}
(197) 
_{Aug}
(128) 
_{Sep}
(101) 
_{Oct}
(183) 
_{Nov}
(186) 
_{Dec}
(42) 
2006 
_{Jan}
(102) 
_{Feb}
(122) 
_{Mar}
(154) 
_{Apr}
(196) 
_{May}
(181) 
_{Jun}
(281) 
_{Jul}
(310) 
_{Aug}
(198) 
_{Sep}
(145) 
_{Oct}
(188) 
_{Nov}
(134) 
_{Dec}
(90) 
2007 
_{Jan}
(134) 
_{Feb}
(181) 
_{Mar}
(157) 
_{Apr}
(57) 
_{May}
(81) 
_{Jun}
(204) 
_{Jul}
(60) 
_{Aug}
(37) 
_{Sep}
(17) 
_{Oct}
(90) 
_{Nov}
(122) 
_{Dec}
(72) 
2008 
_{Jan}
(130) 
_{Feb}
(108) 
_{Mar}
(160) 
_{Apr}
(38) 
_{May}
(83) 
_{Jun}
(42) 
_{Jul}
(75) 
_{Aug}
(16) 
_{Sep}
(71) 
_{Oct}
(57) 
_{Nov}
(59) 
_{Dec}
(152) 
2009 
_{Jan}
(73) 
_{Feb}
(213) 
_{Mar}
(67) 
_{Apr}
(40) 
_{May}
(46) 
_{Jun}
(82) 
_{Jul}
(73) 
_{Aug}
(57) 
_{Sep}
(108) 
_{Oct}
(36) 
_{Nov}
(153) 
_{Dec}
(77) 
2010 
_{Jan}
(42) 
_{Feb}
(171) 
_{Mar}
(150) 
_{Apr}
(6) 
_{May}
(22) 
_{Jun}
(34) 
_{Jul}
(31) 
_{Aug}
(38) 
_{Sep}
(32) 
_{Oct}
(59) 
_{Nov}
(13) 
_{Dec}
(62) 
2011 
_{Jan}
(114) 
_{Feb}
(139) 
_{Mar}
(126) 
_{Apr}
(51) 
_{May}
(53) 
_{Jun}
(29) 
_{Jul}
(41) 
_{Aug}
(29) 
_{Sep}
(35) 
_{Oct}
(87) 
_{Nov}
(42) 
_{Dec}
(20) 
2012 
_{Jan}
(111) 
_{Feb}
(66) 
_{Mar}
(35) 
_{Apr}
(59) 
_{May}
(71) 
_{Jun}
(32) 
_{Jul}
(11) 
_{Aug}
(48) 
_{Sep}
(60) 
_{Oct}
(87) 
_{Nov}
(16) 
_{Dec}
(38) 
2013 
_{Jan}
(5) 
_{Feb}
(19) 
_{Mar}
(41) 
_{Apr}
(47) 
_{May}
(14) 
_{Jun}
(32) 
_{Jul}
(18) 
_{Aug}
(68) 
_{Sep}
(9) 
_{Oct}
(42) 
_{Nov}
(12) 
_{Dec}
(10) 
2014 
_{Jan}
(14) 
_{Feb}
(139) 
_{Mar}
(137) 
_{Apr}
(66) 
_{May}
(72) 
_{Jun}
(142) 
_{Jul}
(70) 
_{Aug}
(31) 
_{Sep}
(39) 
_{Oct}
(98) 
_{Nov}
(133) 
_{Dec}
(44) 
2015 
_{Jan}
(70) 
_{Feb}
(27) 
_{Mar}
(36) 
_{Apr}
(11) 
_{May}
(15) 
_{Jun}
(70) 
_{Jul}
(30) 
_{Aug}
(63) 
_{Sep}

_{Oct}

_{Nov}

_{Dec}

S  M  T  W  T  F  S 





1
(3) 
2
(2) 
3

4

5
(1) 
6
(3) 
7
(1) 
8
(4) 
9
(2) 
10
(1) 
11

12

13

14

15

16

17

18
(1) 
19
(1) 
20

21

22

23

24

25

26

27

28

29
(6) 
30
(13) 
31
(4) 
From: Thomas Mattison <mattison@ph...>  20051201 22:56:56

Hi To the spectators of this discussion, it's in the context of me volunteering to do actual work, not complaining that someone else should! On 1Dec05, at 4:38 AM, HansBernhard Broeker wrote: > Thomas Mattison wrote: > >> The major annoyance is that the errors from fits are not done in what >> I consider to be the correct way. There exists a definition for the >> error of a fit that is independent of the goodness of the fit, >> although of course it depends on the function, the distribution of >> the data, and the error bars on the data. Gnuplot fits do not report >> this result. > > I know. I made it that way, at least partly on purpose. > > The basic problem is that once the chisq/ndf is far away from one, > discussing correctness of parameter errors is a waste of energy. In > such a case whole fit is plain and simply wrong, so no parameter error > value really has a right of calling itself correct. > > chisq/ndf far away from one means that either the data errors are > unrealistic, or the model is wrong (over/underfitting), or both. > In this situation, it's a quite completely arbitrary choice whether to > believe the input errors, or the residuals. gnuplot chooses to > believe the residuals. > For cases where the model being is fit is good, the data is good, and the errors are appropriate, there will still be fluctuations in the value of chisquare from data set to data set. It is wrong to say that the "lucky" data sets really determine the parameters better than the "unlucky" data sets. The parameter errors should be independent of the fluctuations in the chisquare. Every other fitting program I have run across reports the canonical error, which is independent of the chisquare. I agree that if the chisq/ndf is far greater than 1, one should interpret the canonical error with a grain of salt. That is why I advocate reporting both the canonical error and the rescaled error. >> If the function fits the data perfectly at every point, the error >> returned by gnuplot is zero. This is clearly nonsense. > > No  the fit itself is clearly nonsense. I have to insist that > outputting nonsense as the result is fully justified in such a case. I disagree. If we fit a straight line to two data points, the fit will be perfect, with a chisquare of zero. If the errors on the two data points are known, then it is simple to calculate the errors on the slope and intercept by standard error propagation. The canonical error from a standard fitting program agrees exactly with the errorpropagation result. The errors are welldefined, meaningful, and finite. But the gnuplot convention says that the errors on the slope and intercept are either zero because the chisquare is zero, or undefined (because in the case of two data points and two parameters, there are zero degrees of freedom). >> I would add a feature: create a parameterlog file that would contain >> gnuplotreadable fit summary information: chisquare, parmA, >> normalerrorA, rescalederrorA, parmB, regularerrorB, >> rescalederrorB, ... Each fit would append to the end of the file, >> similar to the present fit log file. I would precede each line with >> a copy of the fit command that produced the fit (with a # in front so >> gnuplot would consider it to be a comment). Probably I would also >> have a commented line giving the time of the fit. It would also be >> possible to create a comment line containing headers to show which >> column means what for the fit, using the user's names for the fit >> variables. > > I think it would make a lot more sense to add a couple lines to > 'fit.log' instead of creating what would be an almost complete copy of > all of its content. The reason I would like a separate file is that the fit.log file is intended to be read by humans, and is not particularly readable by gnuplot. In my original post I gave a realworld example from teaching with gnuplot where students do multiple similar fits to different data sets, then plot the fit parameters (not the fit data!) with gnuplot. They even fit the fit parameters (with errors) using gnuplot. It's tedious and errorprone to read the fit.log files, and type the parameters and errors into another file, before gnuplot can read them. If gnuplot fits put their results in gnuplotreadable form into a file, it would be simple and reliable. Adding lines to fit.log that would be gnuplotreadable would be a partial solution, but a human would still have to delete the nonreadable lines. Perhaps a compromise of putting a comment character at the start of all the other lines of fit.log would be the best solution. >> There is a more major feature addition that would be nice. >> Frequently, data has errors not only on the yvariable, but also on >> the xvariable. Gnuplot (nicely) handles plotting data with both x >> and y errors, but it doesn't know how to fit such data. > > Neither do Mrs. Marquard and Levenberg, or anybody else I've heard > about, for generic nonlinear fitting. The theory behind it is simple enough, and some programs (including mine) can do it, they are just not as widely known as the simpler case of errors only in y. > Rigorously, fitting doesn't even know what x values are. It's just a > typical use case simplification to treat the model as a function > > y[i] = f(x[i], parameters) > > Internally, the algorithm only assumes > > y[i] = f[i](parameters) Yes, as I stated you need to go beyond y[i] = f[i](parameters). It's not a simple extension of standard algorithms, in the way that z[i] = f(x[i], y[i], parameters) is a simple extension of y[i] = f(x[i], parameters). But it can be done, and errors on the x variables are common in practice, so it should be done more often. > >> But the next two lines don't do analogous things >> plot 'file' using 1:2:3:4 with xyerr >> fit f(x,y) 'file' using 1:2:3:4 via a,b,c >> The plot command uses columns 3 and 4 for x and y errors, and the fit >> command uses columns 3 and 4 for z and z errors. >> One solution would be to make the interpretation of the columns in a >> fit command depend on the number of variables in the function. > > It won't work as easy as that  parameters can also be passed as > parameters, i.e. you can > > fit f(x,a,b,c) 'file' u 1:2:3 via a,b,c > > instead of > > fit f(x) 'file' u 1:2:3 via a,b,c > > so the argument count of the function is strictly a red herring. I hadn't thought of that. So that proposal doesn't work. I also suggested having the fit parser respect "with" options to make the distinction. Another possibility would be to make a new command name like "sfit" and use that for fits with both x and y as independent variables. That would probably simplify the fit command parser (at the expense of creating a new, but hopefully simple, command to parse), and allow more parallelism between plot/fit and splot/sfit. Cheers Prof. Thomas Mattison, Dept. of Physics & Astronomy, Univ. of British Columbia Present Address: Stanford Linear Accelerator Center 2575 Sand Hill Road, Menlo Park, CA, 94025 Building 48 (Research Office Building), Mail Station MS35 Office: ROB231 Phone: 6509265342 Fax: 6509268522 
From: HansBernhard Broeker <broeker@ph...>  20051201 12:37:37

Thomas Mattison wrote: > The major annoyance is that the errors from fits are not done in what I > consider to be the correct way. There exists a definition for the error > of a fit that is independent of the goodness of the fit, although of > course it depends on the function, the distribution of the data, and the > error bars on the data. Gnuplot fits do not report this result. I know. I made it that way, at least partly on purpose. The basic problem is that once the chisq/ndf is far away from one, discussing correctness of parameter errors is a waste of energy. In such a case whole fit is plain and simply wrong, so no parameter error value really has a right of calling itself correct. chisq/ndf far away from one means that either the data errors are unrealistic, or the model is wrong (over/underfitting), or both. In this situation, it's a quite completely arbitrary choice whether to believe the input errors, or the residuals. gnuplot chooses to believe the residuals. > If the function fits the data perfectly at every point, the error > returned by gnuplot is zero. This is clearly nonsense. No  the fit itself is clearly nonsense. I have to insist that outputting nonsense as the result is fully justified in such a case. > My proposed solution would be to report both versions of the error: the > conventional error that is independent of the goodness of fit, and the > rescaled error that gnuplot presently reports. I would also explicitly > report the factor by which the data errors have been effectively > rescaled. That factor already is reported. It's sqrt(WSSR/ndf). I don't really think that there's much value in outputting two numbers as seemingly independent results where in reality there's just a multiplication by common factor needed to go from one to the other. > A feature of this fix is that it would change the format of the > parameter and error report from the fits. I don't have as good an idea > as you folks do about whether users would care about format changes. Probably not as much now as they would have before version 4.0, when we introduced 'set fit errorvariables' which lets users get at the parameter errors directly from inside gnuplot, without parsing the fit.log or screen output to extract them. > I would add a feature: create a parameterlog file that would contain > gnuplotreadable fit summary information: chisquare, parmA, > normalerrorA, rescalederrorA, parmB, regularerrorB, > rescalederrorB, ... Each fit would append to the end of the file, > similar to the present fit log file. I would precede each line with a > copy of the fit command that produced the fit (with a # in front so > gnuplot would consider it to be a comment). Probably I would also have > a commented line giving the time of the fit. It would also be possible > to create a comment line containing headers to show which column means > what for the fit, using the user's names for the fit variables. I think it would make a lot more sense to add a couple lines to 'fit.log' instead of creating what would be an almost complete copy of all of its content. > I also have some minor annoyances that I think are worth fixing. I > would change the default FIT_START_LAMBDA to be 0.01 as recommended by > Numerical Recipes rather than the method used now (I tell my students > to do this and it frequently helps). Caution there  the actual algorithm is not the same as that in NR (although it used to be), so not all recommendations issued by that book may apply to gnuplot unmodified. I took the computation for the initial lambda from a textbook on numerical maths (Schwarz "Numerische Mathematik", Teubner Verlag, in German). I.e. the default startup for lambda is not any particular fixed number. It's computed from the problem. > I would make the default value for uninitialized variables in fits to > be 0.01 rather than 1e30 (the numerical derivatives algorithm tends > to break with such a tiny default). It's not 1e30 right now  it's 1.0 > I would fix the numerical derivatives algorithm so it would > not break if the initial value of a parameter is zero. > There is a more major feature addition that would be nice. Frequently, > data has errors not only on the yvariable, but also on the xvariable. > Gnuplot (nicely) handles plotting data with both x and y errors, but it > doesn't know how to fit such data. Neither do Mrs. Marquard and Levenberg, or anybody else I've heard about, for generic nonlinear fitting. > The rigorously correct way involves > fitting for adjustments to all the xvariable values, as well as the > parameters. Rigorously, fitting doesn't even know what x values are. It's just a typical use case simplification to treat the model as a function y[i] = f(x[i], parameters) Internally, the algorithm only assumes y[i] = f[i](parameters) > But the next two lines don't do analogous things > > plot 'file' using 1:2:3:4 with xyerr > fit f(x,y) 'file' using 1:2:3:4 via a,b,c > > The plot command uses columns 3 and 4 for x and y errors, and the fit > command uses columns 3 and 4 for z and z errors. That's because that fit has no useful relation with the 'plot' command you're comparing it to. The type of command to compare it with would be splot 'file' using 1:2:3:4 with errorbars (which unfortunately still doesn't exist). > One solution would be to make the interpretation of the columns in a fit > command depend on the number of variables in the function. It won't work as easy as that  parameters can also be passed as parameters, i.e. you can fit f(x,a,b,c) 'file' u 1:2:3 via a,b,c instead of fit f(x) 'file' u 1:2:3 via a,b,c so the argument count of the function is strictly a red herring. 
From: Lars Hecking <lhecking@us...>  20051201 09:51:16

 Forwarded message from Thomas Mattison <mattison@...>  To: lhecking@..., broeker@..., cgaylord@... From: Thomas Mattison <mattison@...> Subject: volunteering for gnuplot work? Date: Wed, 30 Nov 2005 13:21:24 0800 Hi I'm a physics professor, and I use gnuplot for teaching and my own work. There are some annoyances about the fitting aspects of gnuplot that I would like to volunteer to fix. The major annoyance is that the errors from fits are not done in what I consider to be the correct way. There exists a definition for the error of a fit that is independent of the goodness of the fit, although of course it depends on the function, the distribution of the data, and the error bars on the data. Gnuplot fits do not report this result. Internally, gnuplot does calculate this error, but it then rescales it by the goodness of the fit. If the function fits the data perfectly at every point, the error returned by gnuplot is zero. This is clearly nonsense. Consider the simplest possible case of a single data point with a y value and error, and fitting a singleparameter function whose value is just the parameter. The value of the fit parameter should be the y value, and the error on the fit parameter should be the error on the data point. A gnuplot fit returns the right parameter value, but since the goodness of fit is perfect, gnuplot says the error on the fit parameter is zero. The rescaling that gnuplot does to the fit errors is equivalent to calculating a single scale factor on the data point errors from the residuals of the fit, and using the rescaled data errors in the fit. There is some value to this rescaling in many common circumstances. The standard formula for fit parameter errors requires that valid data errors be input to the fit. If the user does not have them, the gnuplot algorithm is equivalent to computing a uniform value for the data errors from the residuals of the fit. I don't have a better suggestion for what to do in such cases. My proposed solution would be to report both versions of the error: the conventional error that is independent of the goodness of fit, and the rescaled error that gnuplot presently reports. I would also explicitly report the factor by which the data errors have been effectively rescaled. If the user does not include errors in the fit, I would make both answers be the present gnuplot answer. In this case, the above errorrescaling factor is in fact the effective error that gnuplot has calculated for the data points. If the data were plotted with this constant error value, then they should agree with the fitted function within this error about 2/3 of the time. A feature of this fix is that it would change the format of the parameter and error report from the fits. I don't have as good an idea as you folks do about whether users would care about format changes. Perhaps a backwardcompatibility switch could be put in, so the oldformat report could still be available. Another annoyance is the lack of a nice way to record fit error values (it would also be nice to record parameter errors from multiple similar fits to different data sets). I would add a feature: create a parameterlog file that would contain gnuplotreadable fit summary information: chisquare, parmA, normalerrorA, rescalederrorA, parmB, regularerrorB, rescalederrorB, ... Each fit would append to the end of the file, similar to the present fit log file. I would precede each line with a copy of the fit command that produced the fit (with a # in front so gnuplot would consider it to be a comment). Probably I would also have a commented line giving the time of the fit. It would also be possible to create a comment line containing headers to show which column means what for the fit, using the user's names for the fit variables. An example of using this feature is an experiment students do in my lab. They fit the currentvoltage relationship for a diode, which is I(V) = I0 * (exp(V/V0)  1). They do this with the diode dunked in liquid nitrogen, dry ice and alcohol, ice water, and boiling water. The I0 and V0 parameters depend on temperature, and I want the students to plot and fit the parameters vs temperature. The students could take this new parameterlog file, add a temperature column to the file with a text editor, then plot and fit from this file. I also have some minor annoyances that I think are worth fixing. I would change the default FIT_START_LAMBDA to be 0.01 as recommended by Numerical Recipes rather than the method used now (I tell my students to do this and it frequently helps). I would make the default value for uninitialized variables in fits to be 0.01 rather than 1e30 (the numerical derivatives algorithm tends to break with such a tiny default). I would fix the numerical derivatives algorithm so it would not break if the initial value of a parameter is zero. There is a more major feature addition that would be nice. Frequently, data has errors not only on the yvariable, but also on the xvariable. Gnuplot (nicely) handles plotting data with both x and y errors, but it doesn't know how to fit such data. The rigorously correct way involves fitting for adjustments to all the xvariable values, as well as the parameters. The naive way to do this produces impractically big matrices when there is a lot of data, even if the fit function is simple (the computation time goes as datapoints cubed instead of proportional to points). There is a more efficient but still rigorous way to do it (goes as parameters squared times points), and an even more efficient way (just a few times longer than the conventional fit, with not much extra coding required). Very few commonly available programs that do fits deal correctly with errors in x variables as well as y variables. I would be willing to try adding this to gnuplot. Note that there is a userinterface consistency issue. The following lines do analogous things. plot 'file' using 1:2:3 with err fit f(x) 'file' using 1:2:3 via a,b,c But the next two lines don't do analogous things plot 'file' using 1:2:3:4 with xyerr fit f(x,y) 'file' using 1:2:3:4 via a,b,c The plot command uses columns 3 and 4 for x and y errors, and the fit command uses columns 3 and 4 for z and z errors. One solution would be to make the interpretation of the columns in a fit command depend on the number of variables in the function. In this case, fit f(x ) 'file' using 1:2:3:4 via a,b,c fit f(x,y) 'file' using 1:2:3:4 via a,b,c would do different things. The first one would be y vs x with x and y errors, the second would be z vs (x,y) with z errors. Another solution would be to allow fit commands to "with" options like plot commands. If the "fit with" options were absent, gnuplot would do what it does now. If the "fit with" options were present, they would override the current default. I have my own clanguage fitting package for nonlinear fits using numerical derivatives that has all the capability of the gnuplot fitting internals. It would probably be easier for me to add xy error fitting by replacing the fitting guts of gnuplot with my own code than to reverseengineer what's there now well enough to add what would be required (yes I have looked at it). I would be willing to do bugfixes and maintenance to the fitting stuff afterwards. So what do you think of the possibility of me getting involved in this? Cheers Prof. Thomas Mattison, Dept. of Physics & Astronomy, Univ. of British Columbia Present Address: Stanford Linear Accelerator Center 2575 Sand Hill Road, Menlo Park, CA, 94025 Building 48 (Research Office Building), Mail Station MS35 Office: ROB231 Phone: 6509265342 Fax: 6509268522  End forwarded message  