gnuplot-beta — Developers and beta testers of new gnuplot code

Re: ylabel not rotated for enhanced windows terminal

[Ethan A M. <sf...@us...>]

On Friday, February 04, 2011 01:17:36 am Benjamin Lindner wrote:
> Hello list,
> 
> I ran into another strange behviour with the windows terminal and enhanced text.
> With a 4.5.0 CVS snapshot build, when I execute
> 
> set term windows enhanced
> set ylabel "something in mm^2"
> plot [0:2*pi] sin(x) with linespoints
> 
> the ylabel is placed horizontally at the top left corner, however if I
> do instead

I should note that I cannot reproduce this bug using Tatsuro Matsuoka's CVS
snapshot from 18-Jan-2011 running under wine.   So it may or may not be an
actual code problem as opposed to a configuration problem.

	Ethan

> 
> set tem windows enhanced
> plot [0:2*pi] sin(x) with linespoints
> set ylabel "something in mm^2"
> replot
> 
> it works as expected.
> 

Re: ylabel not rotated for enhanced windows terminal

[sfeam (E. Merritt) <eam...@gm...>]

On Friday, February 04, 2011, Benjamin Lindner wrote:
> Hello list,
> 
> I ran into another strange behviour with the windows terminal and enhanced text.
> With a 4.5.0 CVS snapshot build, when I execute
> 
> set term windows enhanced
> set ylabel "something in mm^2"
> plot [0:2*pi] sin(x) with linespoints
> 
> the ylabel is placed horizontally at the top left corner, however if I
> do instead
> 
> set tem windows enhanced
> plot [0:2*pi] sin(x) with linespoints
> set ylabel "something in mm^2"
> replot
> 
> it works as expected.
> 
> I tracked it down to do_plot() in graphics.c at line 1632 deciding
> that the terminal can't do rotated text.
> do_plot() decides so, because WIN_text_angle (in win.trm) returnes
> FALSE, because graphwin.rotate is FALSE.
> Now graphwin.rotate is set in MakeFonts() in wgraph.c, which
> determines, whether the font in question supports being rotated.
> MakeFonts() in turn is called (among others) by GraphChangeFont()
> which itself is called from drawgraph() with the OP Code W_font.
> 
> Ok, so on startup, when the terminal is initialized, graphwin.rotate
> is set to TRUE (which is correct), but it gets reset to FALSE during
> execution.
> This happens, because GraphChangeFont() is called with an empty font
> name (not a NULL pointer, but the empty string ""). MakeFonts(), when
> presented with a font name which is empty, resets graphwin.rotate to
> FALSE.

It should not do that.
An empty string means "restore the default font".
So unless the default font is for some reason non-rotatable,
WIN_text_angle should always return TRUE.

	Ethan
 
> And at this point I got unsure as to what is the correct way to fix
> this problem.
> 
> Solution 1)
> GraphChangeFont() should test whether the font name is not empty, and
> only then really change the font (i.e. silently ignore empty font
> names)
> 
> Solution 2)
> drawgraph() should only call GraphChangeFont() if the font name is not
> empty (i.e. silently ignore W_font commands with empty font names)
> 
> Solution 3)
> GraphChangeFont() should not be called with an empty font name in the
> first place.
> Such a call is found in WIN_set_font (win.trm) when the fontname is
> NULL. Is this correct?
> And WIN_set_font(NULL) is called from within WIN_options() with the
> remark "font initialization". Is this correct, too?
> 
> Solution 4)
> deal with it in MakeFonts() directly.
> 
> benjamin
> 
> ------------------------------------------------------------------------------
> The modern datacenter depends on network connectivity to access resources
> and provide services. The best practices for maximizing a physical server's
> connectivity to a physical network are well understood - see how these
> rules translate into the virtual world? 
> http://p.sf.net/sfu/oracle-sfdevnlfb
> _______________________________________________
> gnuplot-beta mailing list
> gnu...@li...
> https://lists.sourceforge.net/lists/listinfo/gnuplot-beta
> 

Re: HTML help on Windows

[Bastian M. <bma...@we...>]

Thanks for the offer. I will send you the files in a private mail.

I have tried your proposed directory layout:
Help files (index stub, project file) in docs/win/,
output of files (including intermediates) to config/win/

It turns out that the HTML help compiler "hhc" does not like it very 
much if files are not located in directories below the location of the 
project file (.hhp). In the case of the gnuplot help it spits out a few 
hundred errors, but the result works just fine. Guess we just ignore 
them by piping to > NUL. Otherwise we would have to copy some files
to config/win first.

Am 03.02.2011 21:20, schrieb Hans-Bernhard Bröker:
> On 03.02.2011 20:41, Bastian Märkisch wrote:
>
>> Right now I use Visual C++ to build. Could somebody using MinGW or
>> Cygwin to build on Windows please help me to integrate the new files
>> in these build systems?
>
> I can probably help. I would also like to see OpenWatcom supported
> along with the others.
>
>> Also am uncertain about the directory layout: Where should the
>> project file (wgnuplot.hhp) and the stub index file (wgnuplot.hhk)
>> go? The obvious options being docs/, config/ or src/win/ (as is the
>> case for the current help project file wgnuplot.hpj).
>
> Since it's Windows-specific, it should go into a Windows-specific
> directory. Right now we have only one of those: src/win. But a new
> docs/win directory might make sense, too.
>
> While at it, I think it would help to move the MinGW, Cygwin-MinGW,
> DJGPP and MSVC builds into a directory structure that doesn't clutter
> the main source tree (src, src/win, docs) with build-specific files.
> I've been using that kind of plan on my home box for quite a while now,
> and checked it in for OpenWatcom recently. This layout allows to keep
> current builds by all compilers alive, without having to constantly
> "make realclean" to avoid them getting into each other's way.
>
> The basic idea is to hold the primary makefile in a subdirectory of
> config, and build it such that it finds all the files in src, term,
> src/win and other places from there. In a nutshell, you just move
> Makefile.foo and the accompanying config.foo (if any) from config into a
> new directory config/foo, as Makefile and config.h. After fixing all the
> breakage from that move, you can build right in there, without any need
> for copying makefiles around.
>
>> I modified a copy of docs/plotstyles.gnu to produce png images
>> instead of pdf files. Does anybody have a nice solution how to
>> recombine those two?
>
> Hard to tell without seeing what you modified. If it's just filenames
> and terminal options, that could be encapsulated into macros or
> subroutine scripts.

ylabel not rotated for enhanced windows terminal

[Benjamin L. <bj...@gm...>]

Hello list,

I ran into another strange behviour with the windows terminal and enhanced text.
With a 4.5.0 CVS snapshot build, when I execute

set term windows enhanced
set ylabel "something in mm^2"
plot [0:2*pi] sin(x) with linespoints

the ylabel is placed horizontally at the top left corner, however if I
do instead

set tem windows enhanced
plot [0:2*pi] sin(x) with linespoints
set ylabel "something in mm^2"
replot

it works as expected.

I tracked it down to do_plot() in graphics.c at line 1632 deciding
that the terminal can't do rotated text.
do_plot() decides so, because WIN_text_angle (in win.trm) returnes
FALSE, because graphwin.rotate is FALSE.
Now graphwin.rotate is set in MakeFonts() in wgraph.c, which
determines, whether the font in question supports being rotated.
MakeFonts() in turn is called (among others) by GraphChangeFont()
which itself is called from drawgraph() with the OP Code W_font.

Ok, so on startup, when the terminal is initialized, graphwin.rotate
is set to TRUE (which is correct), but it gets reset to FALSE during
execution.
This happens, because GraphChangeFont() is called with an empty font
name (not a NULL pointer, but the empty string ""). MakeFonts(), when
presented with a font name which is empty, resets graphwin.rotate to
FALSE.

And at this point I got unsure as to what is the correct way to fix
this problem.

Solution 1)
GraphChangeFont() should test whether the font name is not empty, and
only then really change the font (i.e. silently ignore empty font
names)

Solution 2)
drawgraph() should only call GraphChangeFont() if the font name is not
empty (i.e. silently ignore W_font commands with empty font names)

Solution 3)
GraphChangeFont() should not be called with an empty font name in the
first place.
Such a call is found in WIN_set_font (win.trm) when the fontname is
NULL. Is this correct?
And WIN_set_font(NULL) is called from within WIN_options() with the
remark "font initialization". Is this correct, too?

Solution 4)
deal with it in MakeFonts() directly.

benjamin

Re: HTML help on Windows

[Hans-Bernhard B. <HBB...@t-...>]

On 03.02.2011 20:41, Bastian Märkisch wrote:

> Right now I use Visual C++ to build. Could somebody using MinGW or
> Cygwin to build on Windows please help me to integrate the new files
> in these build systems?

I can probably help.  I would also like to see OpenWatcom supported
along with the others.

> Also am uncertain about the directory layout: Where should the
> project file (wgnuplot.hhp) and the stub index file (wgnuplot.hhk)
> go? The obvious options being docs/, config/ or src/win/ (as is the
> case for the current help project file wgnuplot.hpj).

Since it's Windows-specific, it should go into a Windows-specific
directory.  Right now we have only one of those: src/win.  But a new
docs/win directory might make sense, too.

While at it, I think it would help to move the MinGW, Cygwin-MinGW,
DJGPP and MSVC builds into a directory structure that doesn't clutter
the main source tree (src, src/win, docs) with build-specific files.
I've been using that kind of plan on my home box for quite a while now,
and checked it in for OpenWatcom recently. This layout allows to keep
current builds by all compilers alive, without having to constantly
"make realclean" to avoid them getting into each other's way.

The basic idea is to hold the primary makefile in a subdirectory of
config, and build it such that it finds all the files in src, term,
src/win and other places from there. In a nutshell, you just move
Makefile.foo and the accompanying config.foo (if any) from config into a
new directory config/foo, as Makefile and config.h. After fixing all the
breakage from that move, you can build right in there, without any need
for copying makefiles around.

> I modified a copy of docs/plotstyles.gnu to produce png images
> instead of pdf files. Does anybody have a nice solution how to
> recombine those two?

Hard to tell without seeing what you modified.  If it's just filenames
and terminal options, that could be encapsulated into macros or
subroutine scripts.

HTML help on Windows

[Bastian M. <bma...@we...>]

Hello,

Windows Vista and Windows 7 no longer support the old help system used 
by gnuplot (out of the box). For a bug report see for example:

https://sourceforge.net/tracker/?func=detail&aid=2849723&group_id=2055&atid=102055

Creating a new converter from gnuplot.doc to the new (HTML based) format 
proved to be simple enough and the change in API is only minor.

Right now I use Visual C++ to build. Could somebody using MinGW or 
Cygwin to build on Windows please help me to integrate the new files in 
these build systems?

Also am uncertain about the directory layout: Where should the project 
file (wgnuplot.hhp) and the stub index file (wgnuplot.hhk) go? The 
obvious options being docs/, config/ or src/win/ (as is the case for the 
current help project file wgnuplot.hpj).

I modified a copy of docs/plotstyles.gnu to produce png images instead 
of pdf files. Does anybody have a nice solution how to recombine those two?

Bastian

Re: Enhanced rotated text incorrectly positioned for windows terminal

[Ethan A M. <sf...@us...>]

On Thursday, February 03, 2011 05:34:39 am Benjamin Lindner wrote:
> Hello list,
> 
> For the windows terminal, when using rotated enhanced text (e.g. for
> the ylabel) the text positioning is buggy.

Thanks for the analysis, and the patch.
Now we just have to wait for SourceForge to restore CVS access, which
has been disabled since a hacking attempt last week.

	Ethan


> This manifests itself with either sub/superscripts or font changes
> (e.g. something like "pressure in N/mm^2" or "height in {/Symbol m}m")
> - see the testcase below
> I believe I tracked it down to the text extent being caclulated wrong,
> more precisely: the text extent is first calculated and then scaled to
> the terminal's coordinate system. And here the relative scaling is
> done only with respect to the terminal's width, and not (also) with
> respect to the terminal's height. So if the terminal's width is larger
> than it's height (a common scenario), the relative coordinates for
> rotated text are wrong.
> 
> I prepared a patch, which corrects this issue for me.
> 
> As testcase use:
> 
> set term windows enhanced
> plot [0:2*pi] sin(x) with linespoints
> set xlabel "length in {/Symbol m}m"
> set ylabel "height in {/Symbol m}m"
> replot
> 
> benjamin
> 

Enhanced rotated text incorrectly positioned for windows terminal

[Benjamin L. <bj...@gm...>]

Hello list,

For the windows terminal, when using rotated enhanced text (e.g. for
the ylabel) the text positioning is buggy.
This manifests itself with either sub/superscripts or font changes
(e.g. something like "pressure in N/mm^2" or "height in {/Symbol m}m")
- see the testcase below
I believe I tracked it down to the text extent being caclulated wrong,
more precisely: the text extent is first calculated and then scaled to
the terminal's coordinate system. And here the relative scaling is
done only with respect to the terminal's width, and not (also) with
respect to the terminal's height. So if the terminal's width is larger
than it's height (a common scenario), the relative coordinates for
rotated text are wrong.

I prepared a patch, which corrects this issue for me.

As testcase use:

set term windows enhanced
plot [0:2*pi] sin(x) with linespoints
set xlabel "length in {/Symbol m}m"
set ylabel "height in {/Symbol m}m"
replot

benjamin

Issues with windows terminal (size&position)

[Benjamin L. <bj...@gm...>]

Hello list,

I encountered some issues with the windows terminal regarding the size
and position option.

1)
Specifying "set term windows size x,y position x,y" has no effect once
the graph window is shown. I.e. you can't change the size or position
of an already open window

2)
The width and height specified in the size option should define the
terminal's canvas size. Currently it only takes into account the
graph's status line height, but ignores the window's borders (left,
right, top and bottom) and caption area.

I'd like to propose a patch (see attached file) which should fix both
of these issues and also fix a minor one regarding the correct storage
of the terminal options (for set term save/set term push and show
term) - the terminal's title was not correctly stored, and the
size&position options ignored.

Comments?

benjamin

crashing

[Mojca M. <moj...@gm...>]

Dear developers,

I was trying to build libraries for Gnuplot from scratch. It has been
built successfully, but I experience a crash if I want to use
pdfcairo. I have built pango and cairo "--without-x" (if that is
relevant).


gnuplot> set term pdf
Terminal type set to 'pdfcairo'
Options are ' fontscale 0.5 size 5.00in, 3.00in '
gnuplot> set output 'a.pdf'
gnuplot> plot sin(x)

(process:13196): GLib-GObject-CRITICAL **: gtype.c:2706: You forgot to
call g_type_init()

(process:13196): GLib-GObject-CRITICAL **: gtype.c:2706: You forgot to
call g_type_init()

(process:13196): GLib-GObject-CRITICAL **: gtype.c:2761: You forgot to
call g_type_init()

(process:13196): GLib-GObject-WARNING **: cannot retrieve class for
invalid (unclassed) type `<invalid>'
Segmentation fault


I was using the following libraries:
libcairo.2.dylib
libexpat.1.5.2.dylib
libfontconfig.1.dylib
libfreetype.6.dylib
libgd.2.0.0.dylib
libglib-2.0.0.dylib
libgmodule-2.0.0.dylib
libgobject-2.0.0.dylib
libgthread-2.0.0.dylib
libiconv.2.dylib
libintl.8.0.2.dylib
libjpeg.8.dylib
libncurses.5.dylib
libpango-1.0.0.dylib
libpangocairo-1.0.0.dylib
libpangoft2-1.0.0.dylib
libpixman-1.0.dylib
libpng12.0.dylib
libreadline.6.1.dylib
libz.1.2.5.dylib

cairo 1.8.10
glib 2.24.2
gettext 0.17
libiconv 1.13.1
... (if any other library is relevant, I can provide more information)

I will try to figure out what is going on (once I will have some time
and if I figure out how to debug that ...), but if anyone has any
idea, please let me know ...

Mojca

Re: utf-8 description (specify to use utf-8 without BOM)

[Hans-Bernhard B. <HBB...@t-...>]

On 27.01.2011 20:20, Ethan A Merritt wrote:

> I gather that other tools allow you to set a preference for +/- BOM,
> but Notepad gives you no such option.   There is, I am told,
> an equivalent program called Notepad++ that does allow you to set
> a preference.

Calling Notepad++ an equivalent to MS Notepad would be grievously 
unjust.  It is *way* more than that.  I haven't seen many better 
open-source, free programmers' text editors, and none of those as 
seamlessly native to the MS Windows look&feel as Notepad++.

Re: utf-8 description (specify to use utf-8 without BOM)

[Ethan A M. <sf...@us...>]

On Wednesday, January 26, 2011 01:21:52 am ri...@pi... wrote:
> On 01/24/11 23:57, Ethan A Merritt wrote:
> > On Monday, January 24, 2011 02:21:34 pm Mojca Miklavec wrote:
> >> On Mon, Jan 24, 2011 at 23:11, Tatsuro MATSUOKA wrote:
> >>> Hello
> >>>
> >>> gnuplot only accepts utf-8 without the BOM (Byte Oder Mark) but not that with the BOM.
> >>>
> >>> I think it is better to mention it in a proper position  in the manual
> >>
> >> Or even better: to fix the source code :)
> >
> > You mean the source code for Notepad?
> >
> >
> 
> Is this just Notepad or other more general MS sillyness?

I gather that other tools allow you to set a preference for +/- BOM,
but Notepad gives you no such option.   There is, I am told, 
an equivalent program called Notepad++ that does allow you to set
a preference.

	Ethan

Re: utf-8 description (specify to use utf-8 without BOM)

[Mojca M. <moj...@gm...>]

On Thu, Jan 27, 2011 at 17:10,  <pl...@pi...> wrote:
>
> Since it appears that this BOM is a valid uft-8 white space character
> isn't it conceivable that try to dance around MS non-standard stupidity
> could mess up interpretation of a valid input file or gnuplot script?

My questions are:

- What is the percentage of windows users who have no idea what BOM is
and would want to run the script? (Imagine ... you are not even able
to see it with any given editor apart from hex viewer.) I think that
this is not neglegible.

- What would you need the character for in gnuplot scripting? Can you
give me an example of when you would want to use it? (I really cannot
think of any. Maybe "set xlabel 'abc<zerowidthspace>def'", but what
good does that do, even if the terminal supports the character?)
Gnuplot is not supposed to do high-quality typography with hyphenation
or to implement spell-checker for words ...

Even if there are some obscure examples that do make sense, the
percentage of people that would want to misuse the character in script
is neglegible compared to the poor windows users with no control of
Notepad behaviour. (Seriously: what could be the example?)

- In what way exactly could "please ignore that character" instruction
mess up with "valid input file"? To the contrary. Current implemention
without BOM support that "doesn't dance around the stupidity" might at
best reserve three extra character widths to fit that "zero width"
character between "abc" and "def" in the above example, so that
something that gets printed as "abcdef" would consume 9 character
widths.

(I didn't test what my patch would do with 'abc<zerowidthspace>def',
but no matter whether it does or doesn't do anything, there is no harm
being done if interpreter just ignores the <zerowidthspace>.)

- The only valid reason when this would break something is when
somebody using Latin1 encoding would want to type

    set xlabel 'abc\
     def'

What is the percentage of those users?

Mojca

Re: utf-8 description (specify to use utf-8 without BOM)

[Mojca M. <moj...@gm...>]

On Thu, Jan 27, 2011 at 14:46,  <mw...@gm...> wrote:
> Hi,
>
> just my 2 cents:
>
> * as <http://unicode.org/faq/utf_bom.html> points out, utf-8 has no byte order
>  (in contrast to utf-16 and utf-32) and thus does not need a byte order mark.

It definitely doesn't need it. The fact is that files do have it (if
nothing else to signal that it is UTF-8 and not Latin1 encoding for
example), by default when created with some tools.

>  The 3 byte sequens however serve as a hint to the encoding of the file. On
>  the other hand U+FEFF is a valid and normal unicode character ("ZERO WIDTH
>  NO-BREAK SPACE") even if it is at the beginning of a file

However Wikipedia also says:

If the BOM character appears in the middle of a data stream, it
should, according to Unicode, be interpreted as a "zero-width
non-breaking space" (essentially a null character). Its deliberate use
for this purpose is deprecated in Unicode 3.2, however, with the "Word
Joiner" character, U+2060, strongly preferred.

> * The 3 byte sequence should only be skipped if it is at the beginning of a
>  file or string.

But in addition to the statement above ... unless one will have a
super-advanced typographically-aware terminal with enabled hyphenation
... this character is supposed to be ignored anyway.

> * Having an optional 3 byte sequence at the beginning of a file complicates
>  things a lot. I think a script to "fix" damaged utf-8 files is probably the
>  best solution:
>
>    awk '{if(NR==1)sub(/^\xef\xbb\xbf/,"");print}' text.txt
>    # http://www.linuxask.com/questions/how-to-remove-bom-from-utf-8

Unless somebody is working on windows and awk comes preinstalled with
the system ... :) :) :)

> * Nevertheless being tolerant with respect to input is in general a good
>  thing.
>
> * My approach would look like:

Your code works for me as well, with one exception:
    gnuplot < testscript.plt
or
    cat testscript.plt | gnuplot
breaks with your code while it works with the one I sent. Of course
    gnuplot testscript.plt
still works.

My personal preferences are:
- I find it better to ignore BOM in any line to also support cases
with piping (I don't see where it could break anything except in data
file that are read with different routines anyway). Does that sequence
represent anything sensible in any other encoding?
- Either solution is better than no patch at all.
- (I'm not sure if it is better to issue warnings or not. Or at least
... maybe one would want to issue it just once per gnuplot session,
else it probably gets really annoying if one doesn't fix it, so the
fix becomes just a better place to spot the message when compared to
documentation, but one needs to fix it anyway.)

Mojca

Re: utf-8 description (specify to use utf-8 without BOM)

[<pl...@pi...>]

On 01/27/11 14:46, mw...@gm... wrote:
> * Having an optional 3 byte sequence at the beginning of a file complicates
>    things a lot. I think a script to "fix" damaged utf-8 files is probably the
>    best solution:
>
>      awk '{if(NR==1)sub(/^\xef\xbb\xbf/,"");print}' text.txt
>      #http://www.linuxask.com/questions/how-to-remove-bom-from-utf-8
>

Hi,

thanks for the script, that is what I suggested dong a couple of days 
ago but I now find I sent from the wrong account so the list apparently 
dropped it.  (Didn't it used to send a warning for that ??)

Since it appears that this BOM is a valid uft-8 white space character 
isn't it conceivable that try to dance around MS non-standard stupidity 
could mess up interpretation of a valid input file or gnuplot script?


regards

Re: utf-8 description (specify to use utf-8 without BOM)

[<mw...@gm...>]

Hi,

just my 2 cents:

* as <http://unicode.org/faq/utf_bom.html> points out, utf-8 has no byte order
  (in contrast to utf-16 and utf-32) and thus does not need a byte order mark.
  The 3 byte sequens however serve as a hint to the encoding of the file. On
  the other hand U+FEFF is a valid and normal unicode character ("ZERO WIDTH
  NO-BREAK SPACE") even if it is at the beginning of a file. Treating it
  special is just a well educated guess.

* The 3 byte sequence should only be skipped if it is at the beginning of a
  file or string.

* Having an optional 3 byte sequence at the beginning of a file complicates
  things a lot. I think a script to "fix" damaged utf-8 files is probably the
  best solution:

    awk '{if(NR==1)sub(/^\xef\xbb\xbf/,"");print}' text.txt
    # http://www.linuxask.com/questions/how-to-remove-bom-from-utf-8

* Nevertheless being tolerant with respect to input is in general a good
  thing.

* My approach would look like:

diff --git a/src/misc.c b/src/misc.c
index afe3967..ac8ddb4 100644
--- a/src/misc.c
+++ b/src/misc.c
@@ -213,6 +213,8 @@ load_file(FILE *fp, char *name, TBOOLEAN can_do_args)
     int more;
     int stop = FALSE;
 
+    bool start_of_file = true;
+
     lf_push(fp, name, NULL); /* save state for errors and recursion */
     do_load_arg_substitution = can_do_args;
 
@@ -274,6 +276,24 @@ load_file(FILE *fp, char *name, TBOOLEAN can_do_args)
 		}
 	    }
 
+            /* ignore "BOM" ([which is] "only an encoding signature to
+             * distinguish UTF-8 from other encodings - it has nothing to do
+             * with byte order [in the case of UTF-8]"
+             * <http://unicode.org/faq/utf_bom.html>) */
+            if (start_of_file
+                    && strlen(gp_input_line) >= 3
+                    && ((unsigned char)gp_input_line[0] == 0xef)
+                    && ((unsigned char)gp_input_line[1] == 0xbb)
+                    && ((unsigned char)gp_input_line[2] == 0xbf)) {
+
+                int_warn(NO_CARET, "Your file starts with a BOM (byte order mark). UTF-8 has no byte order, please see <http://unicode.org/faq/utf_bom.html>. You also might want to remove it.");
+
+                char *inlptr = gp_input_line + 3;
+                memmove(gp_input_line, inlptr, strlen(inlptr));
+                gp_input_line[strlen(inlptr)] = NUL;
+            }
+            start_of_file = false; /* only check at once */
+
 	    /* process line */
 	    if (strlen(gp_input_line) > 0) {
 		if (can_do_args)

-- 
GMX DSL Doppel-Flat ab 19,99 Euro/mtl.! Jetzt mit 
gratis Handy-Flat! http://portal.gmx.net/de/go/dsl

Re: cannot access cvs

[Juhász P. <pet...@gm...>]

On Thu, 2011-01-27 at 07:51 +0900, Tatsuro MATSUOKA wrote:
> Hello
> 
> I cannot access the cvs of gnuplot dev source.
> 
> $ export CVSROOT=:pserver:ano...@gn...:/cvsroot/gnuplo
> t
> 
> Tatsu@Shiro /d/usr/Tatsu/gnuplotcvs
> $ cvs login
> Logging in to :pserver:ano...@gn...:2401/cvsroot/gnuplot
> CVS password:
> cvs [login aborted]: connect to gnuplot.cvs.sourceforge.net(216.34.181.109):2401 failed: Connection
> refused
> 
> Is there any trouble?
> 
> Regards
> 
> Tatsuro
> 

It seems to be down from here, too:

cvs -d:ext:ju...@gn...:/cvsroot/gnuplot update 
ssh: connect to host gnuplot.cvs.sourceforge.net port 22: Connection
refused
cvs [update aborted]: end of file from server (consult above messages if
any

(By the way, I thought you don't use pserver if you have an account.)

Péter Juhász

Re: cannot access cvs

[Hans-Bernhard B. <HBB...@t-...>]

On 27.01.2011 00:01, Mojca Miklavec wrote:

> I don't know details about sourceforge, I'm also getting
>      rsync: failed to connect to gnuplot.cvs.sourceforge.net:
> Connection refused (111)

According to their site status page

	http://sourceforge.net/apps/wordpress/sourceforge/

they're experiencing some problem with their CVS service, and had to 
take it offline for the time being.

Re: cannot access cvs

[Mojca M. <moj...@gm...>]

On Wed, Jan 26, 2011 at 23:51, Tatsuro MATSUOKA wrote:
> Hello
>
> I cannot access the cvs of gnuplot dev source.

I don't know details about sourceforge, I'm also getting
    rsync: failed to connect to gnuplot.cvs.sourceforge.net:
Connection refused (111)
if I try to access the files via rsync (I didn't try cvs because I
don't really like it), but in case that you only need to access source
code, here you have my clone:
    https://github.com/gnuplot/gnuplot

Mojca

Re: utf-8 description (specify to use utf-8 without BOM)

[Mojca M. <moj...@gm...>]

2011/1/26 Tatsuro MATSUOKA wrote:
> Hello
>
> Mojca made a patch on this matter so that what I wrote is to be ignored.

Did you manage to try it out?

I created a file with BOM and I did some basic tests on mac (except
with data files which need another patch), but I would be grateful if
you could try it out and do some more tests to see if it is working
properly in all the border cases.

(In particular I would say that an additional "if" is desirable to
check that varible "expression" is long enough.)

>> otherwise Mojca himself will try to do it.

(herself, actually)

Best regards,
    Mojca

cannot access cvs

[Tatsuro M. <tma...@ya...>]

Hello

I cannot access the cvs of gnuplot dev source.

$ export CVSROOT=:pserver:ano...@gn...:/cvsroot/gnuplo
t

Tatsu@Shiro /d/usr/Tatsu/gnuplotcvs
$ cvs login
Logging in to :pserver:ano...@gn...:2401/cvsroot/gnuplot
CVS password:
cvs [login aborted]: connect to gnuplot.cvs.sourceforge.net(216.34.181.109):2401 failed: Connection
refused

Is there any trouble?

Regards

Tatsuro

--------------------------------------
Get the new Internet Explorer 8 optimized for Yahoo! JAPAN
http://pr.mail.yahoo.co.jp/ie8/

Re: utf-8 description (specify to use utf-8 without BOM)

[Tatsuro M. <tma...@ya...>]

Hello 

Mojca made a patch on this matter so that what I wrote is to be ignored.

Regards

Tatsuro

--- Tatsuro MATSUOKA  wrote:

> Hello
> 
> Judging from posts in the perhaps no one want to implement functionality to use UTF-8 with the
> BOM
> otherwise Mojca himself will try to it.  
> 
> What is not good for users, fact that the utf-8 with the BOM cannot be used in the current
> gnuplot is
> not open to the public. 
> 
> In the first mail to this thread I have proposed manual modification (gnuplot.doc).
> If it is accepted, it is grateful for me.  If the place I have proposed is not good, please
> suggest
> where is the appropriate place to write it.
> 
> Regards
> 
> Tatsuro
>  
> --- Mojca Miklavec wrote:
> 
> > On Tue, Jan 25, 2011 at 03:12, Allin Cottrell wrote:
> > >
> > > I'm not sure I'd call this a "fix". Wikipedia says of the BOM in
> > > UTF-8:
> > >
> > > "While Unicode standard allows BOM in UTF-8, it does not require
> > > or recommend it.
> > 
> > However ... this has to be read as: gnuplot is not required to
> > *output* files with BOM (and thus doesn't need to be fixed to create
> > BOM marks in output), but it should better support them when *opening*
> > external files. Even if the marks are not required by the standard,
> > they are still there. Even worse ... from what some people here say
> > they are even there by default in some standard Windows tools.
> > 
> > Mojca
> > 
> > (But once again: I don't know the source good enough, so I have no
> > idea how difficult it would be to fix that particular behaviour.)
> > 
> > ------------------------------------------------------------------------------
> > Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)!
> > Finally, a world-class log management solution at an even better price-free!
> > Download using promo code Free_Logger_4_Dev2Dev. Offer expires 
> > February 28th, so secure your free ArcSight Logger TODAY! 
> > http://p.sf.net/sfu/arcsight-sfd2d
> > _______________________________________________
> > gnuplot-beta mailing list
> > gnu...@li...
> > https://lists.sourceforge.net/lists/listinfo/gnuplot-beta
> > 
> 
> 
> --------------------------------------
> Get the new Internet Explorer 8 optimized for Yahoo! JAPAN
> http://pr.mail.yahoo.co.jp/ie8/
> 
> ------------------------------------------------------------------------------
> Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)!
> Finally, a world-class log management solution at an even better price-free!
> Download using promo code Free_Logger_4_Dev2Dev. Offer expires 
> February 28th, so secure your free ArcSight Logger TODAY! 
> http://p.sf.net/sfu/arcsight-sfd2d
> _______________________________________________
> gnuplot-beta mailing list
> gnu...@li...
> https://lists.sourceforge.net/lists/listinfo/gnuplot-beta
> 


--------------------------------------
Get the new Internet Explorer 8 optimized for Yahoo! JAPAN
http://pr.mail.yahoo.co.jp/ie8/

Re: utf-8 description (specify to use utf-8 without BOM)

[Mojca M. <moj...@gm...>]

On Wed, Jan 26, 2011 at 05:16, sfeam (Ethan Merritt) wrote:
> On Tuesday, January 25, 2011, Mojca Miklavec wrote:
>> I don't know enough about gnuplot's source, so I don't know how
>> difficult it is to change it, but if there is no problem to support
>> comments (in both data files and scripts), I don't see why ignoring
>> the first two bytes would not be doable. I consider it "equally hard".
>
> If you want to experiment with that approach,  you can find the
> relevant switch statement at line 201 of scanner.c (scanner):
>
>            switch (expression[current]) {
>            case '#':           /* DFK: add comments to gnuplot */
>                goto endline;   /* ignore the rest of the line */
>            case '^':
>            case '+':
>
> That isn't going to help with data files, however.
> Only with command lines that unexpectedly contain the BOM sequence.

I can catch BOM with the following code:

--- a/src/scanner.c
+++ b/src/scanner.c
@@ -114,8 +114,14 @@ scanner(char **expressionp, size_t *expressionlenp)
            /* leave space for dummy end token */
            extend_token_table();
        }
-       if (isspace((unsigned char) expression[current]))
+       if (isspace((unsigned char) expression[current])) {
            continue;           /* skip the whitespace */
+       } else if (((unsigned char)expression[current] == 0xef) &&
((unsigned char)expression[current+1] == 0xbb) && ((unsigned
char)expression[current+2] == 0xbf)) {
+           current += 2;
+           // optional warning
+           // int_warn(t_num, "Your file starts with a BOM character;
you might want to remove it.");
+           continue;
+       }
        token[t_num].start_index = current;
        token[t_num].length = 1;
        token[t_num].is_token = TRUE;   /* to start with... */

(NOTE 1: to avoid possible segmentation faults or other problems on
files with less than 3 characters one would probably want to test if
expression is long enough first. I didn't test if it really segfaults
or not though, but it is probably polite to check if
expression[current+2] is valid at all ...)

(NOTE 2: I'm not sure if that is a good idea or not; one might want to
set "utf-8" encoding by default in case that BOM is encountered. But
on the other hand doing that might encourage users to always use BOM
to avoid the need to set encoding.)

This would catch any of the following:
- gnuplot filewithbom.plt
- gluplot < filewithbom.plt
- load 'filewithboth.plt'

However it wouldn't catch problematic datafiles (as already
mentioned), but it might be enough to patch df_readascii in datafile.c
to account for those as well. I didn't play with that yet, but I would
like to know what you think about the patch mentioned above.

Mojca

Re: utf-8 description (specify to use utf-8 without BOM)

[Tatsuro M. <tma...@ya...>]

Hello

Judging from posts in the perhaps no one want to implement functionality to use UTF-8 with the BOM
otherwise Mojca himself will try to it.  

What is not good for users, fact that the utf-8 with the BOM cannot be used in the current gnuplot is
not open to the public. 

In the first mail to this thread I have proposed manual modification (gnuplot.doc).
If it is accepted, it is grateful for me.  If the place I have proposed is not good, please suggest
where is the appropriate place to write it.

Regards

Tatsuro
 
--- Mojca Miklavec wrote:

> On Tue, Jan 25, 2011 at 03:12, Allin Cottrell wrote:
> >
> > I'm not sure I'd call this a "fix". Wikipedia says of the BOM in
> > UTF-8:
> >
> > "While Unicode standard allows BOM in UTF-8, it does not require
> > or recommend it.
> 
> However ... this has to be read as: gnuplot is not required to
> *output* files with BOM (and thus doesn't need to be fixed to create
> BOM marks in output), but it should better support them when *opening*
> external files. Even if the marks are not required by the standard,
> they are still there. Even worse ... from what some people here say
> they are even there by default in some standard Windows tools.
> 
> Mojca
> 
> (But once again: I don't know the source good enough, so I have no
> idea how difficult it would be to fix that particular behaviour.)
> 
> ------------------------------------------------------------------------------
> Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)!
> Finally, a world-class log management solution at an even better price-free!
> Download using promo code Free_Logger_4_Dev2Dev. Offer expires 
> February 28th, so secure your free ArcSight Logger TODAY! 
> http://p.sf.net/sfu/arcsight-sfd2d
> _______________________________________________
> gnuplot-beta mailing list
> gnu...@li...
> https://lists.sourceforge.net/lists/listinfo/gnuplot-beta
> 


--------------------------------------
Get the new Internet Explorer 8 optimized for Yahoo! JAPAN
http://pr.mail.yahoo.co.jp/ie8/

Re: utf-8 description (specify to use utf-8 without BOM)

[<ri...@pi...>]

On 01/24/11 23:57, Ethan A Merritt wrote:
> On Monday, January 24, 2011 02:21:34 pm Mojca Miklavec wrote:
>> On Mon, Jan 24, 2011 at 23:11, Tatsuro MATSUOKA wrote:
>>> Hello
>>>
>>> gnuplot only accepts utf-8 without the BOM (Byte Oder Mark) but not that with the BOM.
>>>
>>> I think it is better to mention it in a proper position  in the manual
>>
>> Or even better: to fix the source code :)
>
> You mean the source code for Notepad?
>
>

Is this just Notepad or other more general MS sillyness?

Wouldn't it be simpler to just parse the data with awk or similar?

Peter.