Re: [Plplot-general] cmap0 file format

[Thorsten B. <tho...@do...>]

Hi Alan!

>> ...
>> When running all examples as well as some other test programs of mine, I
>> always get this error on the console that opens when starting a program:
>>
>> ---snip---
>>
>> *** PLPLOT WARNING ***
>> Unrecognized cmap0 format data line.  Line is #000000
>>
>> ---snap---
>>
>> Also, the background color is always white, while all drawing (even if
>> explicitly setting a pen color) is done in red.
>>
>> I'm sure the file is found and read, because when I edit the file
>> cmap0_default.pal, I can achieve better (and seemingly correct) results.
>> I've added an additional "column" after all the color definitions in the
>> cmap0 file. It currently looks like this
>>
>> ---snap---
>> 16
>> #000001 0
>> #ff0000 0
>> #ffff00 0
>> #00ff00 0
>> #7fffd4 0
>> #ffc0cb 0
>> #f5deb3 0
>> #bebebe 0
>> #a52a2a 0
>> #0000ff 0
>> #8a2be2 0
>> #00ffff 0
>> #40e0d0 0
>> #ff00ff 0
>> #fa8072 0
>> #ffffff 0
>> ---snap---
>>
>> Et voilà: There is no error/warning output done to the console and the
>> background is black, while drawings are done in red (or any other color,
>> if explicitly setting apen color).
>>
>> To me it looks as if the format of the cmap0 file has changed, but this
>> change is not (yet) reflected in the cmap0 being delivered.
>
> I agree there should be better documentation of the various cmap0 (and
> cmap1) file formats. (I would be happy to accept such a document from
> you if you dive deeper into this.) However, every style in  
> data/cmap[01]*.pal should be well supported. All of those work well on
> Linux and Mac OS X, and on the Windows platforms we have tested. But
> Windows consists of large numbers of different platforms, and our
> developers don't have access to all of them. So we must rely on our
> Windows users to help us figure out issues.
>
> For example, your Windows experience (and possibly the guy who
> contributed to the thread you mentioned above) seems to be an
> exception for some reason to the good results we have gotten on other
> Windows platforms with the *.pal files.  I would like to get to the
> bottom of this issue with your help.
>
> I have forgotten the details, but IIRC the file reading code does get
> messed up by the extra CR in the usual Windows line endings which is
> why in our svn version of PLplot, all *.pal files _must_ be checked
> out with Unix line endings (only the line feed, not CR) for all
> platforms including Windows. (This happens automatically because of
> the svn:eol-style LF svn property of those files.) And, of course,
> those Unix line endings also get propagated to our distribution
> tarball so all should be well if you leave those Unix line endings as
> is.
>
> But I am now wondering if you (and the guy who had similar issues in
> the previous thread) are using some "smart" tarball unpacker on
> Windows that is converting the necessary Unix line endings to the
> incorrect (for our cmap[01]*.pal file reading) Windows line endings?
>
> Anyhow, if you have some control of line endings when you unpack the
> tarball, then please leave them as Unix and see if that solves the
> problem.

That indeed was the problem. I unpacked the tar-Archive with Winzip.  
Winzip indeed uses a "smart" CR/LF detection and corrects UNIX-style line  
endings to Windows-style line endings. Although I was sure, I had this  
checked before posting to the list, I obviously hadn't. I extracted the  
*.pal files again (with smart detection disabled) and all executables  
using PLplot run without the above mentioned warning.

I also digged into the file reading code. BTW: I now remember, why I  
dislike C ;-)

I'm no expert in C, but the problem to me seems to be somewhere in the  
line-reading code of cmap0_palette_read. After checking the input file for  
existence, the contents of that file are read. First, a line with the  
number of colors to be defined is expected. That line (although also  
ending with "incorrect" CRLF) is read without a problem - otherwise we  
would have seen the error message "Unrecognized cmap0 header".

A loop over the number of colors follows. Within this loop each line is  
analyzed - however, a line is read here using fgets instead of fscanf as  
before (and reading at most 29 characters). The last character of the data  
read is then replaced by NUL which C needs to terminate strings correctly.  
If the line ending is just 0x0A this will work correctly, as it also will,  
if the line has no ending (because 29 characters have been read) -  
however, if the line ending is 0x0D0A the 0x0D will remain in the string  
read.

This string is then checked for either being seven characters long or  
being longer than 9 characters - everything else results in the error we  
saw. This explains, why the column added by me to the cmap0_default.pal  
file seemed to fix the problem.

To me, there seem to be several ways to correct the line-ending problem  
here:

1) According to  
http://pubs.opengroup.org/onlinepubs/009695399/functions/fgets.html fgets,  
if returning without an error, adds a NUL character already to the string.  
So, to me it seems unnecessary to replace the last character by NUL. OTOH,  
it might not hurt, though.

2) Wouldn't it be more safe to check, whether the last characters of the  
string are 0x0A, 0x0D or any combination of them and replace each of these  
characters by a Null-byte? That should not hurt in any case and would  
involve just changing line 1300 of file plctrl.c

from

<code>
color_info[strlen( color_info ) - 1] = '\0'; /* remove return character */
</code>

to something like (may not be correct C, but the idea should be  
understandable)

<code>
for (my_local_i = 0 ; my_local_i < 30; my_local_i++)
{
   /* remove return character(s) */
   if (color_info[my_local_i] == 0x0A) ||
      (color_info[my_local_i] == 0x0D)
   {
      color_info[my_local_i] = 0x00;
   }
}
</code>

3) Usage of fscanf instead of fgets and sscanf. However, this might bring  
up new problems.

-- 
Gruß,
Thorsten