Re: [Plplot-general] cmap0 file format

[Alan W. I. <ir...@be...>]

Hi Thorsten:

On 2011-05-27 12:25+0200 Thorsten Behrens wrote:

> Hi Alan!
>> But I am now wondering if you (and the guy who had similar issues in
>> the previous thread) are using some "smart" tarball unpacker on
>> Windows that is converting the necessary Unix line endings to the
>> incorrect (for our cmap[01]*.pal file reading) Windows line endings?
>>
>> Anyhow, if you have some control of line endings when you unpack the
>> tarball, then please leave them as Unix and see if that solves the
>> problem.
>
> That indeed was the problem. I unpacked the tar-Archive with Winzip.
> Winzip indeed uses a "smart" CR/LF detection and corrects UNIX-style line
> endings to Windows-style line endings. Although I was sure, I had this
> checked before posting to the list, I obviously hadn't. I extracted the
> *.pal files again (with smart detection disabled) and all executables
> using PLplot run without the above mentioned warning.
>
> I also digged into the file reading code. BTW: I now remember, why I
> dislike C ;-)
>
> I'm no expert in C, but the problem to me seems to be somewhere in the
> line-reading code of cmap0_palette_read. After checking the input file for
> existence, the contents of that file are read. First, a line with the
> number of colors to be defined is expected. That line (although also
> ending with "incorrect" CRLF) is read without a problem - otherwise we
> would have seen the error message "Unrecognized cmap0 header".
>
> A loop over the number of colors follows. Within this loop each line is
> analyzed - however, a line is read here using fgets instead of fscanf as
> before (and reading at most 29 characters). The last character of the data
> read is then replaced by NUL which C needs to terminate strings correctly.
> If the line ending is just 0x0A this will work correctly, as it also will,
> if the line has no ending (because 29 characters have been read) -
> however, if the line ending is 0x0D0A the 0x0D will remain in the string
> read.
>
> This string is then checked for either being seven characters long or
> being longer than 9 characters - everything else results in the error we
> saw. This explains, why the column added by me to the cmap0_default.pal
> file seemed to fix the problem.
>
> To me, there seem to be several ways to correct the line-ending problem
> here:
>
> 1) According to
> http://pubs.opengroup.org/onlinepubs/009695399/functions/fgets.html fgets,
> if returning without an error, adds a NUL character already to the string.
> So, to me it seems unnecessary to replace the last character by NUL. OTOH,
> it might not hurt, though.
>
> 2) Wouldn't it be more safe to check, whether the last characters of the
> string are 0x0A, 0x0D or any combination of them and replace each of these
> characters by a Null-byte? That should not hurt in any case and would
> involve just changing line 1300 of file plctrl.c
>
> from
>
> <code>
> color_info[strlen( color_info ) - 1] = '\0'; /* remove return character */
> </code>
>
> to something like (may not be correct C, but the idea should be
> understandable)
>
> <code>
> for (my_local_i = 0 ; my_local_i < 30; my_local_i++)
> {
>   /* remove return character(s) */
>   if (color_info[my_local_i] == 0x0A) ||
>      (color_info[my_local_i] == 0x0D)
>   {
>      color_info[my_local_i] = 0x00;
>   }
> }
> </code>
>
> 3) Usage of fscanf instead of fgets and sscanf. However, this might bring
> up new problems.

I believe we need the fgets/sscanf combination since that allows the
code to reread the line for the case of alternate formats.  So I think
idea (2) is the way to go to fix this issue.

Since it turns out that indeed Windows tarball unpackers can convert
Unix line endings to Windows line endings, this is an accident that
has obviously happened before (the historical posting about this issue
which has been referred to previously in this thread), which has
happened now to Thorsten, and which is waiting to happen to others.
Therefore, it is important to fix the file-reading code so that *.pal
files can have either Unix or Windows line endings.

Does one of our developers with access to Windows want to take
responsibility for fixing this (or evaluating Thorsten's patch if he
goes further with his idea 2?). An obvious test of the updated code
(and something we should do anyway once the issue is fixed) would be
the change the svn properties of the *.pal files to native so they
will be checked out on Windows with the Windows line endings.

Thanks, Thorsten, for being persistent about this issue so we finally
discovered the exact source of this nagging problem.

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state implementation
for stellar interiors (freeeos.sf.net); PLplot scientific plotting software
package (plplot.org); the libLASi project (unifont.org/lasi); the Loads of
Linux Links project (loll.sf.net); and the Linux Brochure Project
(lbproject.sf.net).
__________________________

Linux-powered Science
__________________________