Menu ▾ ▴

#2903 Verify end-of-line at end of file does not affect ascii parsing

nextrelease
open-fixed
nobody
None
5
3 days ago
4 days ago
No

Darrelle had a problem where removing an end-of-line at the end of an ascii file fixed a problem she saw. This shouldn't happen, and I need to study what was going on.

Discussion

  • Jeremy Faden

    Jeremy Faden - 4 days ago

    Yes, this is the case. Compare:

    https://space.physics.uiowa.edu/~jbf/autoplot/data/txt/2903/PJ78-Z.nl.txt
    

    and

    https://space.physics.uiowa.edu/~jbf/autoplot/data/txt/2903/PJ78-Z.txt
    

    which doesn't end in fill (or a newline).

    Interestingly, I've verified that when there are additional columns, the tailing newline has no effect. This is why the problem hasn't been noticed. See PJ78-Z-2.txt in the same directory.

     

    Last edit: Jeremy Faden 4 days ago
  • Jeremy Faden

    Jeremy Faden - 3 days ago

    I'm adding a test to automatically skip any empty line, when the delim parser is used. This adds a trim operation to the record parsing, The problem is the code relied on the wrong number of fields found to skip the record, but with there is an empty line there is still one field.

    This will continue to slow performance, but there are other once-per-line activities which could be cleaned up, like creating a matcher object for each record.

     
  • Jeremy Faden

    Jeremy Faden - 3 days ago
    • status: open --> open-fixed