Re: conflict of separator and missing

[<pl...@pi...>]

On 15/06/16 00:06, Ethan A Merritt wrote:
>> I think you are correct that there is a bug in this part of the code.
>> > > The full-length string from 'set missing' is tested against the
>> > > start of the field contents (after removing leading whitespace);
>> > > then the subsequent character is tested to see if it is whitespace
>> > > rather than a continuation of whatever string is in the field.
>> > > So it works with a tab-separated *.csv file because <tab> counts
>> > > as whitespace, but fails with a comma-separated file because the
>> > > comma is mis-interpreted as part of the field content.
>>

further thoughts:

The cause of this then, seems to be an oversight. The end of the string 
is being tested as though it was the default case of WSpace separators 
and not the specified separator.

It is a little irrelevant what name is give to this sort of file, the 
key point is that the check you describe is not using the current 
datafile separator.

Presumably the same thing would happen if someone had a file using colon 
( or any other non WS char ) as separator and had correctly specified it 
with


set datafile separator ":"

I have not tested this explicitly but there is nothing special about 
using comma sep. so I presume the same bug would manifest.

" the subsequent character is tested to see if it is whitespace"

It seems that this test should be firstly a test for 'separator' and 
then additionally for white-space + separator. As previously stated 
"ignore A" probably should count as a match. Substrings counting as a 
match is not described anywhere and I see not reason for this to be 
taken as a hit.

Thanks for looking into this.



Peter.