Re: Function stats [was: Patch for new "stats" command]

[Philipp K. J. <ja...@ie...>]

On Sunday 08 November 2009 09:14:16 pm Ethan Merritt wrote:
>  On Sunday 08 November 2009, Philipp K. Janert wrote:
> > We'd like to hear feedback and suggestions. Is this
> > useful? Are we missing anything?
>
> Some first thoughts:

Thanks for checking it out!

>
> The behaviour for functions is not as obvious as for files of
> data points.  For example:

I am not sure. I find this a little unfair. There is
no claim that the stats command does function
minimization. It finds the extrema in the data sets
passed to it. And that it does correctly, I think.
(Even in the example given below.)

Let me state it again: the stats command works
on data sets. Not functions (in the analytic sense).
I don't think it would be reasonable to expect 
anything else.

Regarding "the" min/max : you are right, the documentation
could be clearer. If there are multiple points in a data set,
all of which are of the same (minimal) value, then the stats
command currently makes no guarantee for which of those
points it will report the position in the file. It will just report
the position of one of them.

>
>  gnuplot> set xrange [0:10]
>  gnuplot> stats '+' using 1:(sin($1))
>
>  * FILE:
>    Records:      100
>
>  * COLUMNS:
>    Mean:          5.0000              0.1792
>    Minimum:       0.0000 [  1]       -0.9994 [ 48]
>    Quartile:      2.5253 [ 26]       -0.3837 [ 36]
>    Median:        5.0505 [ 51]        0.3082 [ 29]
>    Quartile:      7.5758 [ 76]        0.8075 [ 85]
>    Maximum:      10.0000 [100]        0.9997 [ 79]
>
>  gnuplot> set samples 1000
>  gnuplot> stats '+' using 1:(sin($1))
>
>  * FILE:
>    Records:      1000
>
>  * COLUMNS:
>    Mean:          5.0000               0.1835
>    Minimum:       0.0000 [   1]       -1.0000 [ 472]
>    Quartile:      2.5025 [ 251]       -0.3941 [ 983]
>    Median:        5.0050 [ 501]        0.3149 [  33]
>    Quartile:      7.5075 [ 751]        0.8113 [ 848]
>    Maximum:      10.0000 [1000]        1.0000 [ 158]
>
> I find several things disconcerting about this output, although
> I know what the underlying causes are.
>
>  - The min/max are artifacts of the sampling.
>    They're not even symmetric even though sin(x) is a symmetric function
>    You can reduce the problem by increasing the number of samples, but I
>    think more drastic alternatives should be considered
>
>    1) The 'stats' command could refuse to operate on functions
>    2) The 'stats' command could temporarily bump up the sampling rate
>       by 100x
>    3) The 'stats' command could do a systematic search in the area of
>       the nominal extrema to determine more accurate values.
>       Even so, if the sampling is too coarse it may miss a true extremum
>       that lies elsewhere.
>
>  - The "mean" of a periodic function would normally be calculated over
>    one period of the function rather than an arbitrary range.
>    Yeah, I know, I gave an explicit xrange.  But still...
>
>  - The quantities in [] are documented as "the" point at which the
>    min/max/whatever occurs.  But there is no expectation for either data
>    or functions that the minimum, for example, is only acheived only at a
>    single point.   I don't think it makes any sense to give these values
>    unless the data or function is monotonic.  And given sampling artifacts,
>    it probably makes no sense to give them for function data at all.