Re: utf-8 description (specify to use utf-8 without BOM)

[Mojca M. <moj...@gm...>]

On Thu, Jan 27, 2011 at 17:10,  <pl...@pi...> wrote:
>
> Since it appears that this BOM is a valid uft-8 white space character
> isn't it conceivable that try to dance around MS non-standard stupidity
> could mess up interpretation of a valid input file or gnuplot script?

My questions are:

- What is the percentage of windows users who have no idea what BOM is
and would want to run the script? (Imagine ... you are not even able
to see it with any given editor apart from hex viewer.) I think that
this is not neglegible.

- What would you need the character for in gnuplot scripting? Can you
give me an example of when you would want to use it? (I really cannot
think of any. Maybe "set xlabel 'abc<zerowidthspace>def'", but what
good does that do, even if the terminal supports the character?)
Gnuplot is not supposed to do high-quality typography with hyphenation
or to implement spell-checker for words ...

Even if there are some obscure examples that do make sense, the
percentage of people that would want to misuse the character in script
is neglegible compared to the poor windows users with no control of
Notepad behaviour. (Seriously: what could be the example?)

- In what way exactly could "please ignore that character" instruction
mess up with "valid input file"? To the contrary. Current implemention
without BOM support that "doesn't dance around the stupidity" might at
best reserve three extra character widths to fit that "zero width"
character between "abc" and "def" in the above example, so that
something that gets printed as "abcdef" would consume 9 character
widths.

(I didn't test what my patch would do with 'abc<zerowidthspace>def',
but no matter whether it does or doesn't do anything, there is no harm
being done if interpreter just ignores the <zerowidthspace>.)

- The only valid reason when this would break something is when
somebody using Latin1 encoding would want to type

    set xlabel 'abc\
     def'

What is the percentage of those users?

Mojca