Re: utf-8 description (specify to use utf-8 without BOM)

[Mojca M. <moj...@gm...>]

On Mon, Jan 24, 2011 at 23:57, Ethan A Merritt wrote:
> On Monday, January 24, 2011 02:21:34 pm Mojca Miklavec wrote:
>> On Mon, Jan 24, 2011 at 23:11, Tatsuro MATSUOKA wrote:
>> > Hello
>> >
>> > gnuplot only accepts utf-8 without the BOM (Byte Oder Mark) but not that with the BOM.
>> >
>> > I think it is better to mention it in a proper position  in the manual
>>
>> Or even better: to fix the source code :)
>
> You mean the source code for Notepad?

I understand that that was sarcasm, but still ...

BOM is allowed by the standard. One could argue that Notepad could
offer a few more advanced settings, but it is definitely not
misbehaving, while gnuplot *is* misbehaving according to the standard
if it doesn't accept and ignore the BOM mark.

2011/1/25 Tatsuro MATSUOKA wrote:
>
> When script saved in utf-8 with BOM , bit order marks are attached to the script contests.
> I think that it is not practical to rewrite gnuplot code to accept the script with the utf-8 with BOM.

I don't know enough about gnuplot's source, so I don't know how
difficult it is to change it, but if there is no problem to support
comments (in both data files and scripts), I don't see why ignoring
the first two bytes would not be doable. I consider it "equally hard".

It might be even less practical for users to do dirty tricks to remove
BOM marks from their files. Source code needs to be fixed just once,
while users need to repeat the process over and over again. I never
had any problem with BOM marks, so I don't know how serious problem
that presents in practice.

Mojca

PS: I definitely have to give a compliment about a really nice
surprize to see unicode work almost satisfactory with the latest wxt
terminal in windows (compared to the old one with its own console) ...
It could still be improved (supporting the whole range of unicode as
opposed to just a subset that corresponds to local codepage; and using
unicode automatically/by default), but it is already lightyears ahead
of what it was before that change. This tiny change with BOM seems
nothing compared to the horrible zero-nonascii-support before the new
terminal.