Re: conflict of separator and missing

[sfeam <sf...@us...>]

On Wednesday, 15 June 2016 08:40:46 AM pl...@pi... wrote:
> On 15/06/16 00:06, Ethan A Merritt wrote:
> >> Only something which IS the 'missing' string or the string with leading
> >> and/or trailing white-space should match, IMO.
> >
> > The conventional indication of missing data in a *.csv file is simply
> > an empty field.  This obviously is not possible in a whitespace-separated
> > file.  Gnuplot's use of "set missing" is outside any standard practice
> > I know of for csv files, so anything we choose is likely to strike
> > someone as wrong.
> >
> > For instance, RFC-4180, the closest thing to a csv standard, states that
> > "any field may be quoted with double quotes".  So in the example above,
> > should we ignore this line?
> >   5, 5, "ignore", 5
> > This one?
> >   5, 5, " ignore ", 5
> >
> >
> > 	Ethan
> >
> >>
> >> Peter.
> >
> >
> 
> 
> Ok, in the absence of any properly defined standard , where software 
> like Excel ( probably the most common source of "CSV" files for a lot of 
> people ) produces comma separated variables without using commas, it is 
> likely to be messy.
> 
>   5, 5, "ignore", 5
> 
> This seems a bit of a contrived case, what software will quote one field 
> in a line but not the others?

Excel for one.  It depends on what "format type" you assign to the column.

> How would gnuplot cope with :
> 
>   "5","5", "ignore", "5"

Gnuplot explicitly checks for both numerical and quoted numerical input
in csv files exactly because of this issue.  But the concept of checking for both
quoted and unquoted "missing" strings never occurred to me until just now.

> Looking at the bug I reported may be a chance to review this whole messy 
> subject but it seems like a diversion from the clear bug case.
> 
> If gnuplot scans for the position of the field separators, it should be 
> stopping BEFORE it gets to the next one when testing for occurrences of 
> the  'missing' string.
> 
> That seems to be a simple bug that does not open a whole can of csv worms.

Yeah, but while revisiting the code it seems like a good idea to not only fix
the specific case in the bug report but also any other corner cases we can
think of.

	Ethan