Recent changes to 585: Add option xyerrorbars to set fit to consider x errorshttps://sourceforge.net/p/gnuplot/patches/585/2014-03-04T20:52:54Z#585 Add option xyerrorbars to set fit to consider x errors2014-03-04T20:52:54Z2014-03-04T20:52:54ZBastian Märkischhttps://sourceforge.net/u/markisch/https://sourceforge.netadffc4dfaa1785703553a1c3e963dac1e4d56df0<div class="markdown_content"><ul>
<li><strong>status</strong>: pending-accepted --> closed-accepted</li>
</ul></div>#585 Add option xyerrorbars to set fit to consider x errors2014-02-28T10:56:42Z2014-02-28T10:56:42ZBastian Märkischhttps://sourceforge.net/u/markisch/https://sourceforge.neta9f6c508de0acfe0f8c3c351666c085a5906241e<div class="markdown_content"><ul>
<li><strong>status</strong>: open --> pending-accepted</li>
</ul></div>#585 Add option xyerrorbars to set fit to consider x errors2014-02-28T10:51:48Z2014-02-28T10:51:48ZBastian Märkischhttps://sourceforge.net/u/markisch/https://sourceforge.netc4deac9b74513a346332ab4c846ab4087beae06e<div class="markdown_content"><p>Alexander, thanks for the nice Chi² plot. It would be great to be able to do plots like this after a fit with using a <code>chisq()</code> function in gnuplot. Any ideas how to implement this? Of course we would have to limit it to one or two parameters.</p>
<p>From the plot it seems that it is really the fit algorithm that gets stuck and that it is not a problem of the effective variance calculation per se. An modified version of the patch is now included in CVS. The help text for gnuplot.doc still needs work and is not included yet.</p></div>#585 Add option xyerrorbars to set fit to consider x errors2014-02-22T15:29:42Z2014-02-22T15:29:42ZBastian Märkischhttps://sourceforge.net/u/markisch/https://sourceforge.net4779f3828018eda21908375b64b0417106b581b2<div class="markdown_content"><ul>
<li><strong>labels</strong>: requested feature --> requested feature, fit</li>
</ul></div>#585 Add option xyerrorbars to set fit to consider x errors2014-02-19T09:29:25Z2014-02-19T09:29:25ZAlexander Täschnerhttps://sourceforge.net/u/taschna/https://sourceforge.neta30fa9c66c9bce50d7bd7f62692c914fda2cd07c<div class="markdown_content"><blockquote>
<p>The fact that it does not reach the exact solution is why I started the discussion >again. Without further investigation (e.g. a real Chi² plot of the region), I can >only assume that the algorithm gets stuck in a local minimum.</p>
</blockquote>
<p>Attached is a Chi² plot of the region. The contour lines equal the minimum Chi² <br />
+1, +0.1 and +0.01. The points show the exact solution and Orears one. From the<br />
plot it is clear, that finding the absolut minimum is not really easy.</p></div>#585 Add option xyerrorbars to set fit to consider x errors2014-02-19T07:57:28Z2014-02-19T07:57:28ZBastian Märkischhttps://sourceforge.net/u/markisch/https://sourceforge.net6c9680cf22ed6c90d99addcbff9f63a4646b7b2f<div class="markdown_content"><blockquote>
<p>Demo 1)<br />
How can it be that both the RMS and CHI2 values are better for the xyerr fit, but the p value is worse?</p>
</blockquote>
<p>For the y-errors fit I get p = 3.5e-5, whereas for the xy-error fit it is 0.15, which is much better indeed, i.e. closer to 0.5, and in the 'acceptable' range of lets say <span>[0.06:0.94]</span>.</p>
<blockquote>
<p>Demo 2)<br />
"This is also an example where--due to the large difference in<br />
the order of magnitude of the parameters--prescaling is required<br />
to obtain convergence." I have lost track - is patch #507 still relevant or does it implement the same thing as the current "set fit prescale"?</p>
</blockquote>
<p>Patch #507 is included in CVS since 2013-03-12. Question is, if it shouldn't be included in 4.6.5, too. </p>
<blockquote>
<p>I gather that the fit is not exact even with prescaling, but in the demo the residual error is on the same order of magnitude as the convergence limit. However, if you tighten the convergence limit to "set fit limit 1e-09" then it reaches the same end point and claims an epsilon of e-10. Doesn't this mean there is something wrong with the function evaluation rather than with the convergence?</p>
</blockquote>
<p>The fact that it does not reach the exact solution is why I started the discussion again. Without further investigation (e.g. a real Chi² plot of the region), I can only assume that the algorithm gets stuck in a local minimum. As Alexander pointed out, the calculation of the WSSR seems to be correct. Also, the endpoint is the same as in Orear's paper, which probably tells us that the starting point is just not good enough for both algorithms.</p>
<p>Although the problem looks easy at first glance, it is really very difficult: the correlation between the parameters is very high ( gnuplot says >0.994 - as does a colleagues' Mathematica script ), and -without prescaling- the region of convergence must be very elongated due to the very different scale of the parameters.</p></div>#585 Add option xyerrorbars to set fit to consider x errors2014-02-19T00:25:42Z2014-02-19T00:25:42ZEthan Merritthttps://sourceforge.net/u/sfeam/https://sourceforge.neta10a4eb959ca6d96984e2c5d50263fa7197fa3c8<div class="markdown_content"><p>Thanks. That works nicely other some trivial compiler warnings.</p>
<p>Help me to understand the output of the demo.</p>
<p>Demo 1) <br />
How can it be that both the RMS and CHI2 values are better for the xyerr fit, but the p value is worse?</p>
<p>Demo 2) <br />
"This is also an example where--due to the large difference in<br />
the order of magnitude of the parameters--prescaling is required<br />
to obtain convergence." I have lost track - is patch #507 still relevant or does it implement the same thing as the current "set fit prescale"?<br />
I gather that the fit is not exact even with prescaling, but in the demo the residual error is on the same order of magnitude as the convergence limit. However, if you tighten the convergence limit to "set fit limit 1e-09" then it reaches the same end point and claims an epsilon of e-10. Doesn't this mean there is something wrong with the function evaluation rather than with the convergence?</p></div>#585 Add option xyerrorbars to set fit to consider x errors2014-02-18T23:20:58Z2014-02-18T23:20:58ZBastian Märkischhttps://sourceforge.net/u/markisch/https://sourceforge.netc4b31ef38b82dbc96fcd838b047ab5457a6d4ce1<div class="markdown_content"><p>This patch should apply to current CVS. It includes a demo <code>fitxyerr.dem</code> that uses Orear's data and Pearson's data with York's weights. The documentation still needs to be reworked.</p></div>#585 Add option xyerrorbars to set fit to consider x errors2014-02-17T21:57:02Z2014-02-17T21:57:02ZEthan Merritthttps://sourceforge.net/u/sfeam/https://sourceforge.net96cd1803af8193fd6515916a31643da2fbbcf8e4<div class="markdown_content"><p>Is there a version of this patch that applies against current CVS?</p></div>#585 Add option xyerrorbars to set fit to consider x errors2014-02-17T13:35:00Z2014-02-17T13:35:00ZBastian Märkischhttps://sourceforge.net/u/markisch/https://sourceforge.net3bc7adf01e1cf7cb333042083121363d435eb6b3<div class="markdown_content"><p>Thanks for the quick response. Below are my findings for Orear's data set, which more or less agree with your analysis. I guess the conclusion is that we just get stuck in a local minimum and the code can thus go to CVS right now. We really should do a chi² plot of the region...</p>
<p>Summary of the fit results:</p>
<div class="codehilite"><pre> <span class="n">a1</span> <span class="n">a2</span> <span class="n">a1_err</span> <span class="n">a2_err</span>
<span class="o">------------------------------------------------</span>
<span class="n">initial</span> <span class="n">values</span> <span class="mf">1.0000e-003</span> <span class="mf">5.900e+005</span>
<span class="n">dy</span> <span class="n">only</span> <span class="mf">9.8993e-004</span> <span class="mf">5.893e+005</span> <span class="mf">3.8731e-005</span> <span class="mf">2.289e+004</span>
<span class="p">(</span><span class="o">*</span><span class="p">)</span><span class="n">dy</span> <span class="n">and</span> <span class="n">dx</span> <span class="mf">1.0162e-003</span> <span class="mf">5.937e+005</span> <span class="mf">1.9933e-004</span> <span class="mf">1.116e+005</span>
<span class="n">dy</span> <span class="n">and</span> <span class="n">dx</span><span class="p">,</span> <span class="n">errorscaling</span> <span class="mf">1.0162e-003</span> <span class="mf">5.937e+005</span> <span class="mf">1.7022e-004</span> <span class="mf">9.527e+004</span>
<span class="n">dy</span> <span class="n">and</span> <span class="n">dx</span><span class="p">,</span> <span class="n">no</span> <span class="n">prescale</span> <span class="mf">1.0096e-003</span> <span class="mf">5.900e+005</span> <span class="mf">1.9840e-004</span> <span class="mf">1.110e+005</span>
<span class="o">------------------------------------------------</span>
<span class="n">published</span> <span class="mf">1.0163e-003</span> <span class="mf">5.937e+005</span>
<span class="n">exact</span> <span class="mf">1.0731e-003</span> <span class="mf">6.250e+005</span>
<span class="o">------------------------------------------------</span>
<span class="n">Minuit</span> <span class="mf">1.0727e-003</span> <span class="mf">6.250e+005</span> <span class="mf">2.5254e-004</span> <span class="mf">1.399e+005</span>
</pre></div>
<p>Note that our fit result (*) and the result published in J. Orear, Am. J. Phys. 52, 278 (1984) agree.<br />
This is also an example where--due to the large difference in the order of magnitude of the parameters--prescaling is helpful to obtain convergence!</p></div>