Re: Missing/invalid data points

[Daniel J S. <dan...@ie...>]

Ethan Merritt wrote:
> NB:   This discussion is with regard to possible changes
>       or extensions _after_ 4.2
>=20
> On Tuesday 29 August 2006 10:25 am, Hans-Bernhard Br=F6ker wrote:
>=20
>>Ethan Merritt wrote:
>>
>>>The second case keeps the point, marking it as undefined.
>>>That is better, but it would be better yet if the information
>>>stored was
>>>   0.3 NaN 3.33 u
>>>If nothing else, that would allow the tabular output line
>>>to match the original input line.  Beyond that, the extra
>>>info may be of use in auto-scaling the axes,
>>
>>Hardly --- autoscaling should never react to points that aren't
>>actually on the plot.
>=20
>=20
> I am not so sure.  Consider the "using 1:2:3 with pm3d" plots that
> are being discussed.  Because of the oddities in missing/NaN
> handling, the limits of the grid are not determined correctly.=20
> The coordinate information may be necessary for gridding, even
> if some of the points were unmeasured ('missing') or mis-measured
> (NaN or Inf).
>=20
>=20
>>But that's not the actual point.  The core issue is that there's
>>only *one* "undefined" flag per data point.  To use the non-NaN
>>data values safely, datafile.c would have to record which of them are
>>usable, i.e. which caused the DF_UNDEFINED, and which didn't.  And it
>>would have to continue reading / filling after a NaN or missing
>>column.
>>
>>That's a rewrite from scratch of datafile.c and a good portion of all
>>the code using it you've just outlined.
>=20
>=20
> Nevertheless, that is what I am proposing.
> I already said up-thread that this requires changes throughout
> datafile.c
>=20
> However, it is not quite as bad as you make it sound.
> The change can be incremental.=20

I think so too.  The code may look a bit messy, but we've already narrowe=
d down the means for getting data from a file to df_readline() in plot2d.=
c and plot3d.c--in the case of the new binary data file code.  If we beco=
me comfortable enough to accept the binary data file code, which haas bee=
n running for quite a while now, that eliminates "df_3dmatrix()" and make=
s things a little cleaner.

I don't think what Ethan proposed should be too bad.  (Reaching agreement=
 on behavior is a different matter.)

Dan