Menu
▾
▴
gnuplot-beta
|
From: <pl...@pi...> - 2014-04-20 08:48:16
|
HI,
I often use conditional using clauses to plot part of a range of data
plot datafile using (($1<=1995.55)?$1:NaN):(2*fcos1($1))
I tried this when writing to a file specified with "set table" and
instead of cutting off the output, it appends garbage data with a "u".
help table tells me this means "undefined"
I see two problems here. I did not ask for undefined output , I
specified NaN.
Secondly , what is the possible use of "undefined" values in a data file?
1995.46 1.00914 i
1995.54 1.10947 i
1.98547e-81 4.74363e+170 u
3.75845e+174 7.52884e-24 u
0 2.122e-314 u
0 0 u
0 0 u
1.9771e+161 1.44402e+214 u
-1.22337e-44 1.1735e-319 u
1.01887e+189 7.49746e+247 u
5.35166e+199 2.10124e+88 u
From my using clause I would expect either
NaN [ correct value of 2*fcos($1) ] o
NaN [ correct value of 2*fcos($1) ] i
or preferable nothing at all , as would reflect the usual terminal
output for NaN points.
Version 4.7 patchlevel 0 last modified 2012-10-16
Build System: Linux i686
Thanks , Peter
|
|
From: <pl...@pi...> - 2014-04-20 09:02:59
|
On 04/20/14 10:48, pl...@pi... wrote: > > HI, > > > I often use conditional using clauses to plot part of a range of data > > plot datafile using (($1<=1995.55)?$1:NaN):(2*fcos1($1)) > > I tried this when writing to a file specified with "set table" and > instead of cutting off the output, it appends garbage data with a "u". > > help table tells me this means "undefined" > > I see two problems here. I did not ask for undefined output , I > specified NaN. > > Secondly , what is the possible use of "undefined" values in a data file? > > 1995.46 1.00914 i > 1995.54 1.10947 i > 1.98547e-81 4.74363e+170 u > 3.75845e+174 7.52884e-24 u > 0 2.122e-314 u > 0 0 u > 0 0 u > 1.9771e+161 1.44402e+214 u > -1.22337e-44 1.1735e-319 u > 1.01887e+189 7.49746e+247 u > 5.35166e+199 2.10124e+88 u > > > From my using clause I would expect either > > NaN [ correct value of 2*fcos($1) ] o > NaN [ correct value of 2*fcos($1) ] i > > or preferable nothing at all , as would reflect the usual terminal > output for NaN points. > > Version 4.7 patchlevel 0 last modified 2012-10-16 > Build System: Linux i686 > > > > Thanks , Peter > I've just found out the if I used "NaN" as a string instead of NaN in the using clause I get the result I expected: the data cuts off at the required date. plot datafile using (($1<=1995.55)?$1:"NaN"):(2*fcos1($1)) It seems there is an inconsistency in the way this is being handled. Clearly I should not need two different versions of the plot command. Peter. |
|
From: Ethan A M. <sf...@us...> - 2014-04-23 18:32:22
|
On Sunday, 20 April, 2014 11:02:48 pl...@pi... wrote: > On 04/20/14 10:48, pl...@pi... wrote: > > > > I often use conditional using clauses to plot part of a range of data > > > > plot datafile using (($1<=1995.55)?$1:NaN):(2*fcos1($1)) [defer discussion of tabular output to a separate reply] > I've just found out the if I used "NaN" as a string instead of NaN in > the using clause I get the result I expected: the data cuts off at the > required date. > > plot datafile using (($1<=1995.55)?$1:"NaN"):(2*fcos1($1)) I think that's because you gave a string where a number was expected. The fact that the string contained "NaN" rather than, say, "foo" is not relevant. > It seems there is an inconsistency in the way this is being handled. I agree. This is a bug. If the program finds a string where a number was expected, the numerical value returned to get_data() should be NaN rather than some random or left-over value from an earlier line. This should get a fix for both 4.6 and 5. Ethan |
|
From: Ethan A M. <sf...@us...> - 2014-04-23 18:49:22
|
On Sunday, 20 April, 2014 10:48:10 pl...@pi... wrote:
> HI,
>
>
> I often use conditional using clauses to plot part of a range of data
>
> plot datafile using (($1<=1995.55)?$1:NaN):(2*fcos1($1))
>
> I tried this when writing to a file specified with "set table" and
> instead of cutting off the output, it appends garbage data with a "u".
>
> help table tells me this means "undefined"
>
> I see two problems here. I did not ask for undefined output , I
> specified NaN.
>
> Secondly , what is the possible use of "undefined" values in a data file?
>
> 1995.46 1.00914 i
> 1995.54 1.10947 i
> 1.98547e-81 4.74363e+170 u
> 3.75845e+174 7.52884e-24 u
> 0 2.122e-314 u
> 0 0 u
> 0 0 u
> 1.9771e+161 1.44402e+214 u
> -1.22337e-44 1.1735e-319 u
> 1.01887e+189 7.49746e+247 u
> 5.35166e+199 2.10124e+88 u
I do not know what the original idea was,
nor do I know if there are existing scripts that depend on the 'u'.
It does seem pointless to write out total junk values as in your example.
If it helps, version 5 has a new mode "with table" that does what you want.
gnuplot> set samples 11
gnuplot> set xrange [0:10]
gnuplot> set table
gnuplot> plot '+' using ((4<$1&&$1<8) ? NaN : $1) : ($1**2) with table
0 0
1 1
2 4
3 9
4 16
nan 25
nan 36
nan 49
8 64
9 81
10 100
gnuplot>
This is not ideal because it requires a separate plot style, which means
you can't do
plot $foo; set table; replot
Should tabular output of the regular plot styles be changes to match this?
What, if anything, depends on the old behavior and would break?
Ethan
|
|
From: <pl...@pi...> - 2014-04-23 18:56:51
|
On 04/23/14 20:30, Ethan A Merritt wrote: > > On Sunday, 20 April, 2014 11:02:48 pl...@pi... wrote: >> On 04/20/14 10:48, pl...@pi... wrote: >>> >>> I often use conditional using clauses to plot part of a range of data >>> >>> plot datafile using (($1<=1995.55)?$1:NaN):(2*fcos1($1)) > > [defer discussion of tabular output to a separate reply] > >> I've just found out the if I used "NaN" as a string instead of NaN in >> the using clause I get the result I expected: the data cuts off at the >> required date. >> >> plot datafile using (($1<=1995.55)?$1:"NaN"):(2*fcos1($1)) > > I think that's because you gave a string where a number was expected. > The fact that the string contained "NaN" rather than, say, "foo" is not > relevant. > >> It seems there is an inconsistency in the way this is being handled. > > I agree. This is a bug. If the program finds a string where a number > was expected, the numerical value returned to get_data() should be > NaN rather than some random or left-over value from an earlier line. > This should get a fix for both 4.6 and 5. > > Ethan > I would have expected the parser the throw this out since it is illegitimate input in this context. I only tried this to see what would happen when the correct syntax produced garbage output. Why didn't the parser reject it ? Peter. |
|
From: Ethan A M. <sf...@us...> - 2014-04-23 19:08:38
|
On Wednesday, 23 April, 2014 20:50:18 pl...@pi... wrote: > On 04/23/14 20:30, Ethan A Merritt wrote: > > > > On Sunday, 20 April, 2014 11:02:48 pl...@pi... wrote: > >> On 04/20/14 10:48, pl...@pi... wrote: > >>> > >>> I often use conditional using clauses to plot part of a range of data > >>> > >>> plot datafile using (($1<=1995.55)?$1:NaN):(2*fcos1($1)) > > > > [defer discussion of tabular output to a separate reply] > > > >> I've just found out the if I used "NaN" as a string instead of NaN in > >> the using clause I get the result I expected: the data cuts off at the > >> required date. > >> > >> plot datafile using (($1<=1995.55)?$1:"NaN"):(2*fcos1($1)) > > > > I think that's because you gave a string where a number was expected. > > The fact that the string contained "NaN" rather than, say, "foo" is not > > relevant. > > > >> It seems there is an inconsistency in the way this is being handled. > > > > I agree. This is a bug. If the program finds a string where a number > > was expected, the numerical value returned to get_data() should be > > NaN rather than some random or left-over value from an earlier line. > > This should get a fix for both 4.6 and 5. > > > > Ethan > > > > I would have expected the parser the throw this out since it is > illegitimate input in this context. > > I only tried this to see what would happen when the correct syntax > produced garbage output. > > Why didn't the parser reject it ? It's fine at the level of parsing. There's nothing intrinsically wrong with reading in a data string rather than a number. The problem only comes later if you try to use that for something that really does require a numerical value. The bug is that there is still an old numerical value hanging around from an earlier read operation that is used instead. That shouldn't happen. Ethan |
|
From: <pl...@pi...> - 2014-04-23 20:31:44
|
On 04/23/14 21:05, Ethan A Merritt wrote:
>
> On Wednesday, 23 April, 2014 20:50:18 pl...@pi... wrote:
>> On 04/23/14 20:30, Ethan A Merritt wrote:
>>>
>>> On Sunday, 20 April, 2014 11:02:48 pl...@pi... wrote:
>>>> On 04/20/14 10:48, pl...@pi... wrote:
>>>>>
>>>>> I often use conditional using clauses to plot part of a range of data
>>>>>
>>>>> plot datafile using (($1<=1995.55)?$1:NaN):(2*fcos1($1))
>>>
>>> [defer discussion of tabular output to a separate reply]
>>>
>>>> I've just found out the if I used "NaN" as a string instead of NaN in
>>>> the using clause I get the result I expected: the data cuts off at the
>>>> required date.
>>>>
>>>> plot datafile using (($1<=1995.55)?$1:"NaN"):(2*fcos1($1))
>>>
>>> I think that's because you gave a string where a number was expected.
>>> The fact that the string contained "NaN" rather than, say, "foo" is not
>>> relevant.
>>>
>>>> It seems there is an inconsistency in the way this is being handled.
>>>
>>> I agree. This is a bug. If the program finds a string where a number
>>> was expected, the numerical value returned to get_data() should be
>>> NaN rather than some random or left-over value from an earlier line.
>>> This should get a fix for both 4.6 and 5.
>>>
>>> Ethan
>>>
>>
>> I would have expected the parser the throw this out since it is
>> illegitimate input in this context.
>>
>> I only tried this to see what would happen when the correct syntax
>> produced garbage output.
>>
>> Why didn't the parser reject it ?
>
> It's fine at the level of parsing. There's nothing intrinsically wrong
> with reading in a data string rather than a number. The problem
> only comes later if you try to use that for something that really does
> require a numerical value. The bug is that there is still an old
> numerical value hanging around from an earlier read operation that
> is used instead. That shouldn't happen.
>
> Ethan
>
>
>
OK, I was forgetting that a string can be valid as a 'using' specifier
for named columns. You are correct ( as often happens ;) ).
However, here's another oddity:
gnuplot> plot "-" u ("foo"):2
input data ('e' ends) > 1 2
input data ('e' ends) > e
Warning: empty x range [1.58805e-314:1.58805e-314], adjusting to
[1.57217e-314:1.60393e-314]
Warning: empty y range [2:2], adjusting to [1.98:2.02]
Why is the string "foo" being evaluated as something close to zero? If I
try addition it does not get interpreted as zero, it gets kicked out:
x=1+"foo"
Non-numeric string found where a numeric expression was expected
Here's another bug:
gnuplot> plot "-" u ("foo"):2
input data ('e' ends) > 1 2
input data ('e' ends) > 3 4
input data ('e' ends) > e
This produces two '+' marks at the extreme left of the wxt window:
outside the plot area !
There is no x-axis labelling nor grid, though I do get y axis and grid
lines.
Unless you see any mistakes here, maybe these should be split to
separate messages too.
/Peter.
|
|
From: Ethan A M. <sf...@us...> - 2014-04-23 20:45:13
|
On Wednesday, 23 April, 2014 21:54:56 pl...@pi... wrote:
> On 04/23/14 21:05, Ethan A Merritt wrote:
> >
> > On Wednesday, 23 April, 2014 20:50:18 pl...@pi... wrote:
> >> On 04/23/14 20:30, Ethan A Merritt wrote:
> >>>
> >>> On Sunday, 20 April, 2014 11:02:48 pl...@pi... wrote:
> >>>> On 04/20/14 10:48, pl...@pi... wrote:
> >>>>>
> >>>>> I often use conditional using clauses to plot part of a range of data
> >>>>>
> >>>>> plot datafile using (($1<=1995.55)?$1:NaN):(2*fcos1($1))
> >>>
> >>> [defer discussion of tabular output to a separate reply]
> >>>
> >>>> I've just found out the if I used "NaN" as a string instead of NaN in
> >>>> the using clause I get the result I expected: the data cuts off at the
> >>>> required date.
> >>>>
> >>>> plot datafile using (($1<=1995.55)?$1:"NaN"):(2*fcos1($1))
> >>>
> >>> I think that's because you gave a string where a number was expected.
> >>> The fact that the string contained "NaN" rather than, say, "foo" is not
> >>> relevant.
> >>>
> >>>> It seems there is an inconsistency in the way this is being handled.
> >>>
> >>> I agree. This is a bug. If the program finds a string where a number
> >>> was expected, the numerical value returned to get_data() should be
> >>> NaN rather than some random or left-over value from an earlier line.
> >>> This should get a fix for both 4.6 and 5.
> >>>
> >>> Ethan
> >>>
> >>
> >> I would have expected the parser the throw this out since it is
> >> illegitimate input in this context.
> >>
> >> I only tried this to see what would happen when the correct syntax
> >> produced garbage output.
> >>
> >> Why didn't the parser reject it ?
> >
> > It's fine at the level of parsing. There's nothing intrinsically wrong
> > with reading in a data string rather than a number. The problem
> > only comes later if you try to use that for something that really does
> > require a numerical value. The bug is that there is still an old
> > numerical value hanging around from an earlier read operation that
> > is used instead. That shouldn't happen.
> >
> > Ethan
> >
> >
> >
>
>
> OK, I was forgetting that a string can be valid as a 'using' specifier
> for named columns.
Well yes, but that's not what's happening here.
You are not asking it to read a value from a column named "NaN".
You are asking it to evaluate a value from an in-line expression rather
than from a column, and that in-line expression happens to be the
string constant "NaN".
That's fine if you are going to do something string-like with the value,
but it doesn't work if you actually needed a numerical value.
The bug is that it doesn't fill in a numerical value _at_ _all_.
It just returns the string and leaves the numerical slot full of
whatever was there before - maybe garbage, maybe never initialize,
maybe a previous data value.
> However, here's another oddity:
>
> gnuplot> plot "-" u ("foo"):2
> input data ('e' ends) > 1 2
> input data ('e' ends) > e
> Warning: empty x range [1.58805e-314:1.58805e-314], adjusting to
> [1.57217e-314:1.60393e-314]
> Warning: empty y range [2:2], adjusting to [1.98:2.02]
>
> Why is the string "foo" being evaluated as something close to zero?
Same bug. The numerical value is meaningless; you get whatever
contents were there before the data was read from the file.
Basically it's a failure to initialize the return value.
If the program successfully parses a number from the input line
then it's fine. This is returned. But if it doesn't successfully
parse a number it returns the uninitialized placeholder.
Very bad, but only triggered by an incorrect plot command.
Fixed now in CVS for both 4.6 and 5.
Ethan
|
|
From: Ethan M. <eam...@gm...> - 2014-04-23 23:19:10
|
> Is it a safe bet that no one has a script that relies on meaningless junk? Not the junk values themselves. But there may be scripts that depend on a 1-to-1 agreement between number of lines in and number of lines out. This is the case, for instance, when dumping a matrix or image array. If you skip a line just because it contains an undefined or NaN entry then the whole array grid alignment fails. > This was presumably an oversight when NaN was added. Is anyone really using this "feature"? The 'u' flag was already in the code when it was imported into CVS back in 1999. So no, it doesn't have anything to do with NaN. That said, I really don't know whether anyone uses it. > What is the most consistent with a terminal plot ? That may be the wrong way to look at it. To the extent that "set table" is used to produce intermediate data for reading back in later, it is more important to ask for consistency with the input format. That's why the "set table" output writes 1 or 2 blank lines between curves and data blocks, and why it writes out both INRANGE and OUTRANGE values. The idea is not to duplicate that plot that would have produced using the current axis range, but to save the data so that it can be plotted later perhaps with different ranges. === Let me back up one step and explain why I think it made sense to add the "plot ... with table" option for version 5. Previously when you said "set table", or before that when you said "set terminal table", a [s]plot command would cause the program to (1) read in whatever data it needed for the plot, filtering through axis range limits, style-specific transformations, etc as it went and then (2) instead of actually plotting the data it would write to the table file instead. This 2-step process is significant because if the program sees a NaN (old style 1/0) in step 1 it does not store any data values internally - just a flag that the point was undefined. So in step 2 there are no values available to write out. I'm not sure but I think originally it would have written all zeros rather than the current bug's random garbage, but either way the only correct item on the output line is the 'u'. The new "with table" mode is different. It kicks in earlier and does the whole table generation process in a single step. So there is a chance to keep more of the original data since it isn't filtered by the current axis ranges, plot style-specific filtering, and so on. While the reason it was proposed in the first place was simply that it can handle more columns than any "real" plot style, I think these other differences are equally important. |
|
From: <pl...@pi...> - 2014-04-24 08:13:59
|
On 04/24/14 01:19, Ethan Merritt wrote: > > What is the most consistent with a terminal plot ? > > That may be the wrong way to look at it. Hi, Thanks for the extra detail. I was indeed missing the main intended use. I tend to use this feature to dump out regression results from fit for external processing. I can use something like awk to filter out 'u' lines but it's just tiresome, not a major problem. > So there is a chance to keep more of the original data since it isn't filtered by the current axis ranges This again raises the question of what happens if a range or a using ($1 < .....) is intended to avoid a singularity or some other region where the ordinate is ill-defined. Couldn't this potentially create a div_zero exception or an overflow, that had been specifically excluded by the user, that would crash the plot command ? Peter |
|
From: <pl...@pi...> - 2014-04-23 23:51:47
|
On 04/23/14 20:47, Ethan A Merritt wrote: > > On Sunday, 20 April, 2014 10:48:10 pl...@pi... wrote: >> HI, >> >> >> I often use conditional using clauses to plot part of a range of data >> >> plot datafile using (($1<=1995.55)?$1:NaN):(2*fcos1($1)) >> >> I tried this when writing to a file specified with "set table" and >> instead of cutting off the output, it appends garbage data with a "u". >> >> help table tells me this means "undefined" >> >> I see two problems here. I did not ask for undefined output , I >> specified NaN. >> >> Secondly , what is the possible use of "undefined" values in a data file? >> >> 1995.46 1.00914 i >> 1995.54 1.10947 i >> 1.98547e-81 4.74363e+170 u >> 3.75845e+174 7.52884e-24 u >> 0 2.122e-314 u >> 0 0 u >> 0 0 u >> 1.9771e+161 1.44402e+214 u >> -1.22337e-44 1.1735e-319 u >> 1.01887e+189 7.49746e+247 u >> 5.35166e+199 2.10124e+88 u > > I do not know what the original idea was, > nor do I know if there are existing scripts that depend on the 'u'. > It does seem pointless to write out total junk values as in your example. > > If it helps, version 5 has a new mode "with table" that does what you want. > > gnuplot> set samples 11 > gnuplot> set xrange [0:10] > gnuplot> set table > gnuplot> plot '+' using ((4<$1&&$1<8) ? NaN : $1) : ($1**2) with table > 0 0 > 1 1 > 2 4 > 3 9 > 4 16 > nan 25 > nan 36 > nan 49 > 8 64 > 9 81 > 10 100 > gnuplot> > > This is not ideal because it requires a separate plot style, which means > you can't do > plot $foo; set table; replot > > Should tabular output of the regular plot styles be changes to match this? > What, if anything, depends on the old behavior and would break? > > Ethan > > Is it a safe bet that no one has a script that relies on meaningless junk? This was presumably an oversight when NaN was added. Is anyone really using this "feature"? It's more an inconsistency than a howler of a bug, so wouldn't v5 be a good opportunity to profit from the relaxation of b/c rules and fix it? What is the most consistent with a terminal plot ? Terminals produce no output for NaN points. Producing no output into the table would seem to be the best equivalent to me. That is the behaviour I was expecting , at least. There may be a case for outputting NaN in column 1 and the expression in col 2 but this could fail , for example , if I was doing the NaN trick to prevent evaluation of the expression in a range where it would fail. f(x)=1/x; plot datafile using (($1<0)?$1:NaN):(2 * f($1)) so perhaps NaN NaN ?? I think no output is the most consistent with no point in a plot unless someone can see something I've missed. /Peter |
|
From: <us...@be...> - 2014-04-24 08:34:14
|
Zitat von pl...@pi...: > On 04/23/14 20:47, Ethan A Merritt wrote: > > What is the most consistent with a terminal plot ? Terminals produce no > output for NaN points. Producing no output into the table would seem to > be the best equivalent to me. That is the behaviour I was expecting , at > least. > > There may be a case for outputting NaN in column 1 and the expression in > col 2 but this could fail , for example , if I was doing the NaN trick > to prevent evaluation of the expression in a range where it would fail. > > f(x)=1/x; > plot datafile using (($1<0)?$1:NaN):(2 * f($1)) > > so perhaps NaN NaN ?? > > I think no output is the most consistent with no point in a plot unless > someone can see something I've missed. I do also agree with you, that no output is more consistent. Because you're talking about the NaN trick, I would like to raise a related discussion: I've been thinking about a new option which allows to treat NaN or 1/0 as missing instead of invalid data points. That would improve the possibilities to filter data in gnuplot and still being able to plot them with lines. I think that won't be much work, but how could such an option be called? Christoph |
|
From: sfeam <sf...@us...> - 2014-05-10 21:08:08
|
On Wednesday, 23 April 2014 09:06:15 PM pl...@pi... wrote: > On 04/23/14 20:47, Ethan A Merritt wrote: > > > > On Sunday, 20 April, 2014 10:48:10 pl...@pi... wrote: > >> HI, > >> > >> > >> I often use conditional using clauses to plot part of a range of data > >> > >> plot datafile using (($1<=1995.55)?$1:NaN):(2*fcos1($1)) > >> > >> I tried this when writing to a file specified with "set table" and > >> instead of cutting off the output, it appends garbage data with a "u". > >> > >> help table tells me this means "undefined" > >> > >> I see two problems here. I did not ask for undefined output , I > >> specified NaN. > >> > >> Secondly , what is the possible use of "undefined" values in a data file? > >> > >> 1995.46 1.00914 i > >> 1995.54 1.10947 i > >> 1.98547e-81 4.74363e+170 u > >> 3.75845e+174 7.52884e-24 u > >> 0 2.122e-314 u > >> 0 0 u > >> 0 0 u > >> 1.9771e+161 1.44402e+214 u > >> -1.22337e-44 1.1735e-319 u > >> 1.01887e+189 7.49746e+247 u > >> 5.35166e+199 2.10124e+88 u > > > > I do not know what the original idea was, > > nor do I know if there are existing scripts that depend on the 'u'. > > It does seem pointless to write out total junk values as in your example. > > > > If it helps, version 5 has a new mode "with table" that does what you want. As of yesterday, the version 5 code has been changed to store the actual input values even if some value on the line is NaN, causing the whole line to be marked "undefined". Since the values are stored, they are available for output in a table. Also they are available to be used in a "refresh" command. I can't think of anything this would break, but it is definitely a change in the version 4 behavior. Ethan |
