#1243 problem with CVS version of gnuplot trying to read a file with dos line ending character

closed-fixed
nobody
None
5
2015-03-23
2013-05-22
Marco Cammarata
No

When trying to read a text file containing a dos line ending character (^M) [see attached file data.txt] the memory use increases until my computer stops working...
This happens with the current CVS version (4.7 modified on 2013-05-21 and compiled with gcc 4.7.3 under a debian unstable) but it does NOT happen when using the gnuplot version provided by the debian repository (4.6.3 modified on 2013-04-12).

Please let me know if you need any other info to help debugging ...
Thanks a lot,
marco

1 Attachments

Discussion

  • Ethan Merritt
    Ethan Merritt
    2013-05-22

    Your file contains:
    0.183396E+06^M 0.000000E+00^M

    Thus it is not a normal DOS line-ending file, which would instead contain
    0.183396E+06^M^L 0.000000E+00^M^L

    According to Wikipedia, a bare ^M was used as a line terminator by Acorn, Commodore, TRS-80, and various other 8-bit systmes, and by MacOS prior to OSX. Is this where your file came from?

    Anyhow, I agree that the failure mode in the current cvs version is unpleasant. The code was changed recently so as to not skip over embedded ^L characters in internal datablocks ("here data"). The possibility of embedded ^M without a corresponding ^L did not occur to me. Oh well, it's definitely fixable.

    By the way... If you tell gnuplot that the ^M is a field separator
    set datafile separator "^M"
    then the file is read (without error) as a single line of data containing two fields. That's probably not what you want, but might be useful in other circumstances.

     
    • Thanks for your answer.
      Indeed the file was created using the linux command "paste" (GNU coreutils, 8.20) from two files that vim recognizes as "dos" files (so I guess with ^M^L). I don't know how this results in a ^M.

      What surprises me is the way memory consumption... should not stop when EOF is reached ?

      Anyway as temporary fix I used dos2unix to convert the files before using "paste" to "merge" them.

      Thanks,
      marco

       
  • Ethan Merritt
    Ethan Merritt
    2013-05-22

    From paste? In that case it really would work to tell gnuplot
    set datafile separator "^M"

    EOF has nothing to do with it. The problem comes in trying to parse the very first data line. The program assumes that strtod() will consume one field of data and then move on to the next field. But the internal ^M blocks that "move on to the next field" part, so it reads the same number over and over ad infinitum.

    I am a bit surprised that paste does not handle, or at least warn about, using DOS files for input. Other common tools like vi convert the line ending characters on entry.

     
  • Actually every file I am pasting has 5 whitespaace-separated columns so using ^M as separator is not an option.
    I am also surprised about paste.
    If I will have time I will investigate a bit further.
    I just tried the new CVS version that works great.
    Thanks,
    marco

     
  • Ethan Merritt
    Ethan Merritt
    2013-05-22

    • status: open --> closed-fixed