gnuplot-beta

Prerelease version 4.2.1-pre1

[Ethan A M. <merritt@u.washington.edu>]

I have placed a tarball on SourceForge containing a trial version
of an incremental release 4.2.1.  This is a bug-fix release, including
fixes for installation problems reported on several platforms.
It also includes a couple of minor new features that were just a little
too late for the code freeze leading up to 4.2.0

Barring any problem reports, my plan is to bump the PATCHLEVEL up
to 4.2.1 proper and release it with no changes to the code.


   	          GNUPLOT VERSION 4.2.1-pre1
	    =======================================

This is trial release of version 4.2.1, which is itself an incremental
bug-fix release for version 4.2.0.  A synopsis of the changes since 4.2.0
is given below and in the NEWS file.  Full information is given in the
ChangeLog.   Unless people report problems with this pre-release, it will
be re-tagged as 4.2.1 and released with no other changes.

New features, changes and fixes since gnuplot version 4.2.0
===========================================================
* NEW allow extra column in 2D plots containing color information
* NEW set term latex {size XX,YY}
* FIX buffering of very long input lines
* FIX clipping of image data against plot boundary
* FIX polygon clipping bugs
* FIX key sample for plots with variable color
* FIX wxt initialization on non-gnu systems
* FIX escape sequence %% handling in sprintf() format strings
* FIX Apply "set style incr user" to 3D contours and to columnstacked histograms
* FIX Allow string variable as filename for "fit via <filename>"
* CHANGE defer x11 initialization
* CHANGE clean up configuration files for amg, cyg, mgw, dj2
* CHANGE modify SVG output to accommodate non-compliant viewers
* CHANGE allow 'strcol()' as shorthand for 'stringcolumn()'
* CHANGE default to "blacktext" for TeX-based PostScript variants


Demo plots illustrating these and other features are online at

	http://gnuplot.sourceforge.net/demo_4.2/

You can download a source tarball for gnuplot version 4.2.1-pre1 from the 
gnuplot development site on SourceForge.

	http://sourceforge.net/project/showfiles.php?group_id=2055


Installation
------------
Installation instructions are available in the source itself; the short
version for linux/unix-like systems is to unpack the tarball and then
build it:
		cd gnuplot-4.2.1-pre1 ; ./configure ; make
test it:
		make check
install it:
		make install


Known issues
------------
- Plot styles image and rgbimage are marked EXPERIMENTAL; their implementation
  is incomplete and may change in future versions of gnuplot
- Internationalization and locale support is incomplete in this version.
  If you encounter problems with character encodings, numerical formats, or
  other locale issues, try the CVS version on SourceForge.

Support
-------
Please report all bugs and installation problems to the bug tracker
on SourceForge:

	http://sourceforge.net/tracker/?group_id=2055&atid=102055

There is also an active gnuplot discussion forum on usenet group

	comp.graphics.apps.gnuplot

Development
-----------
Gnuplot development is quite active. The development branch (4.3) on 
SourceForge contains preliminary implementations of many new features;
feedback and additional contributions of code are very welcome.

-- 
Ethan A Merritt

Re: Assignment functions (patchset # 1774478)

[Ethan A M. <merritt@u.washington.edu>]

On Wednesday 15 August 2007 00:40, plotter wrote:
> >  Example:
> >       SUM = 0
> >       plot 'foo' using 1:(f($2)):assign(SUM = SUM + $2):assign(N = $0)
> >       print "Sum of y values is ",SUM
> >       print "Mean y value is ", SUM/N
> >
> However I see a couple of disadvantages with this approach of doing this  
> in 'using' clause of plot.
> 
> Firstly plot lines can get very long and cumbersome already, especially by  
> the time one has added linetype title and colour specifiers. Already some  
> of the uses I have for assign run to several lines on their own. I think  
> anything more than your simple example would make plot lines very hard to  
> follow.

The space required to write it out once should be about the same either way.
But yes, I can see that the earlier f(x)=assign("var",expression)
makes it much easier to re-use the definition concisely in multiple plot
commands. 

I will have to think about whether there is a way to tweak the
assign(var = expression)  variant so that you can be equally concise.
 
> Second this is post processing. Much of the usefullness of this technique  
> comes from the way I used it in a funtion that returns a value to using  
> that gets plotted directly. The post processing seems to be a bit of a
> show stopper for the way I was using this.

It would be very easy to move the assignment evaluation to the beginning
of the per-line processing rather than the end.  But then it becomes a
problem what to do if the data line turns out to have missing or invalid
entries. The earlier variant already suffers from this (see below).
I'm open to suggestions here.

However, I'm not sure that the post/pre processing choice makes any
fundamental difference in what you can plot.  The two variants below should
achieve the same result, right?

 old:  plot "foo" using 1:(assign("SUM",SUM+$2)) title 'running sum'

 new:  plot "foo" using 1:(SUM+$2):assign(SUM=SUM+$2) title 'running sum'



One more thing: 
The behaviour of the earlier form is poorly-defined in a case like this

   A(x) = assign("VAR_A", f(VAR_A))
   B(x) = assign("VAR_B", VAR_A + VAR_B)
   plot "foo" using 1:(A($2)):(B($3))

OK, you can say "don't do that".  The point is that I don't think the order
of evaluation is guaranteed.  Furthermore, what happens if column 3 contains
a '?' (missing data) or 'NaN' (illegal value).  Does the value of A get
updated or not?  What about the value of B?  If so, with what value?
Don't we have a problem that A and B can get out of sync? 


I'm cc'ing this to the developer's mailing list.  Maybe useful suggestions
will come in from a new direction.

	Ethan
-- 
Ethan A Merritt

Re: Assignment functions (patchset # 1774478)

[<pl...@pi...>]

On Wed, 15 Aug 2007 18:02:45 +0200, Ethan A Merritt  =

<merritt@u.washington.edu> wrote:

> On Wednesday 15 August 2007 00:40, plotter wrote:
>> >  Example:
>> >       SUM =3D 0
>> >       plot 'foo' using 1:(f($2)):assign(SUM =3D SUM + $2):assign(N =
=3D $0)
>> >       print "Sum of y values is ",SUM
>> >       print "Mean y value is ", SUM/N
>> >
>> However I see a couple of disadvantages with this approach of doing t=
his
>> in 'using' clause of plot.
>>
>> Firstly plot lines can get very long and cumbersome already, especial=
ly  =

>> by
>> the time one has added linetype title and colour specifiers. Already =
 =

>> some
>> of the uses I have for assign run to several lines on their own. I th=
ink
>> anything more than your simple example would make plot lines very har=
d  =

>> to
>> follow.
>
> The space required to write it out once should be about the same eithe=
r  =

> way.
> But yes, I can see that the earlier f(x)=3Dassign("var",expression)
> makes it much easier to re-use the definition concisely in multiple pl=
ot
> commands.
>
> I will have to think about whether there is a way to tweak the
> assign(var =3D expression)  variant so that you can be equally concise=
.

The big difference is the function is outside the plot command so all th=
at  =

is added is f($2) or a meaningful name like run_mean($2) that actually  =

make the plot command more readable and takes all the messy detail  =

elsewhere.


>
>> Second this is post processing. Much of the usefullness of this  =

>> technique
>> comes from the way I used it in a funtion that returns a value to usi=
ng
>> that gets plotted directly. The post processing seems to be a bit of =
a
>> show stopper for the way I was using this.
>
> It would be very easy to move the assignment evaluation to the beginni=
ng
> of the per-line processing rather than the end.  But then it becomes a=

> problem what to do if the data line turns out to have missing or inval=
id
> entries. The earlier variant already suffers from this (see below).
> I'm open to suggestions here.
>
> However, I'm not sure that the post/pre processing choice makes any
> fundamental difference in what you can plot.  The two variants below  =

> should
> achieve the same result, right?
>
>  old:  plot "foo" using 1:(assign("SUM",SUM+$2)) title 'running sum'
>
>  new:  plot "foo" using 1:(SUM+$2):assign(SUM=3DSUM+$2) title 'running=
 sum'
>
>

yes these two are equivalent because you've duplicated the code. If it w=
as  =

less trivial the whole thing would need duplicating and if the code  =

changed some other variables as in my examples this repetition would mes=
s  =

things up badly.

IMHO half the benefit of the old form was being able to operate on the  =

data and pass back a value to be plotted.

>
> One more thing:
> The behaviour of the earlier form is poorly-defined in a case like thi=
s
>
>    A(x) =3D assign("VAR_A", f(VAR_A))
>    B(x) =3D assign("VAR_B", VAR_A + VAR_B)
>    plot "foo" using 1:(A($2)):(B($3))
>
> OK, you can say "don't do that".  The point is that I don't think the =
 =

> order
> of evaluation is guaranteed.  Furthermore, what happens if column 3  =

> contains
> a '?' (missing data) or 'NaN' (illegal value).  Does the value of A ge=
t
> updated or not?  What about the value of B?  If so, with what value?
> Don't we have a problem that A and B can get out of sync?
>

That's true , although I dont think it's unsurmountable as long a functi=
on  =

can return NAN. The behaviour of
plot is defined in these circumstances and this sort of advanced user  =

technique carries a responsability of testing data before assigning and =
 =

conforming to plot behaviour.

I'd just started playing with NaN which is why I marked my running_mean =
 =

example as unfinished. This is something to look at but I dont think it'=
s  =

a problem, at least in the old format where preprocessing allows a means=
  =

to kick out. Returning NaN would be like raising an exception on that da=
ta  =

point. Things can be kept in sync.

Good point though.

You could check up on execution order. If that is not defined that could=
  =

be tricky , it may be necessary to force the order if plot contains  =

functions.

So far I have not tried a situation where this sort of dependancy comes =
in.


>
> I'm cc'ing this to the developer's mailing list.  Maybe useful  =

> suggestions
> will come in from a new direction.
>
> 	Ethan

Sorry, I did not realise I had mailed that last two comments to you only=
.  =

The reply policy of mailman lists is a contant cause of confusion. Since=
 I  =

am on several I often mix up who does what. I'll try to remember I shoul=
d  =

not use "reply" on gnuplot-beta.

thx.

Re: Assignment functions (patchset # 1774478)

[<HBB...@t-...>]

Ethan A Merritt wrote:
> On Wednesday 15 August 2007 00:40, plotter wrote:

> The space required to write it out once should be about the same either way.
> But yes, I can see that the earlier f(x)=assign("var",expression)
> makes it much easier to re-use the definition concisely in multiple plot
> commands. 

Maybe, instead of inventing more new syntax, we should follow 
established C syntax a little further, e.g. implement the comma operator 
and the "each assignment is an expression, too" idea.  E.g. an averaging 
filter could be expressed as:

	using 1:(N = N + 1, sum = sum + $2, sum / N)

or even

	using 1:((sum += $2)/++n)

> It would be very easy to move the assignment evaluation to the beginning
> of the per-line processing rather than the end.  But then it becomes a
> problem what to do if the data line turns out to have missing or invalid
> entries. The earlier variant already suffers from this (see below).
> I'm open to suggestions here.

Honestly, I think this whole idea is going in the wrong direction. 
We're straying awfully far from the "one tool <--> one task" approch of 
Unix tradition here.  gnuplot is a plotting program, not a substitute 
for awk, Perl or Excel.  I'd prefer it if we left generic data 
processing to generic tools designed for the job.

And even if we're going to do this, the using specification is not 
really the right place to do it in.  It should be moved to a separate 
command.

Re: Assignment functions (patchset # 1774478)

[<pl...@pi...>]

On Thu, 16 Aug 2007 14:10:10 +0200, Hans-Bernhard Bröker  
<HBB...@t-...> wrote:

> Ethan A Merritt wrote:
>> On Wednesday 15 August 2007 00:40, plotter wrote:
>
>> The space required to write it out once should be about the same either  
>> way.
>> But yes, I can see that the earlier f(x)=assign("var",expression)
>> makes it much easier to re-use the definition concisely in multiple plot
>> commands.
>
> Maybe, instead of inventing more new syntax, we should follow
> established C syntax a little further, e.g. implement the comma operator
> and the "each assignment is an expression, too" idea.  E.g. an averaging
> filter could be expressed as:
>
> 	using 1:(N = N + 1, sum = sum + $2, sum / N)
>
> or even
>
> 	using 1:((sum += $2)/++n)

That I like, I think the assign() idea was done as a quick and easy way to  
slip this into the existing parser. It served well to prove the principal  
but yields a rather clumbsy syntax.

I would like to see this sort of "each assignment is an expression"  
approach, that was in fact my original request which Ethan rapidly  
provided in what is now called the "old" implementation.

I think it is useful that whatever is done can be done within a function  
call or similar mechanism that allows pulling the detail outside of the  
plot command itself.


>
>> It would be very easy to move the assignment evaluation to the beginning
>> of the per-line processing rather than the end.  But then it becomes a
>> problem what to do if the data line turns out to have missing or invalid
>> entries. The earlier variant already suffers from this (see below).
>> I'm open to suggestions here.
>
> Honestly, I think this whole idea is going in the wrong direction.
> We're straying awfully far from the "one tool <--> one task" approch of
> Unix tradition here.  gnuplot is a plotting program, not a substitute
> for awk, Perl or Excel.  I'd prefer it if we left generic data
> processing to generic tools designed for the job.

I see your point and I dont think gnuplot needs to provide these sort of  
data processing tools itself. The whole thing here is just to provide a  
mechanism not to start providing any data processing functions directly.  
This would be going beyond "one tool <--> one task" .

Adding a means to hook in simple algebraic tasks written by the user is  
more analogous to calling an awk script from gnuplot except that it is  
tightly bound to the plot iteration rather than duplicating that  
externally.

My area under graph example could be done externally but I can do the same  
thing with gnuplot for the price of two local variables and one (longish)  
line added to my gnu file.

I would not wish to do complex data processing this way but for simple  
tasks like a.u.g. , mean/min/max values it makes a lot of sense.


>
> And even if we're going to do this, the using specification is not
> really the right place to do it in.  It should be moved to a separate
> command.
>

Your comment agrees with my wish to see a mechanism outside the plot line  
rather than burgeoning the using spec, though I would like to see this as  
a hook into the existing plot iteration of the data.

If this were done as a separte command it would end up having to duplicate  
much of what plot does. That duplication of code could lead to errors.

I think my preference would be , as you suggest, "each assignment is an  
expression" and comma syntax but inside the existing function call. This  
would avoid the rather awkward assign() .

Thanks for you comments, Peter.

Re: Assignment functions (patchset # 1774478)

[Ethan M. <merritt@u.washington.edu>]

On Thursday 16 August 2007 05:10, Hans-Bernhard Br=F6ker wrote:
>=20
> Maybe, instead of inventing more new syntax, we should follow=20
> established C syntax

That was the original intent. But I was insufficiently clever at the time;
I could not see how to modify gnuplot's parsing routines to handle the
assignment operator.  Now I have stared at the code more intently.

I have updated the patch on SourceForge to implement C-like assignment
syntax.  For example,
gnuplot> show at a =3D b =3D c =3D 1
        pushc "a"
        pushc "b"
        pushc "c"
        pushc 1
        =3D
        =3D
        =3D

This addition to the expression syntax allows the running sum trick
without any special handling of data column processing:

   plot 'foo' using 1:(sum =3D sum + $2)

> implement the comma operator and the "each assignment is an expression, t=
oo"
> idea.  E.g. an averaging filter could be expressed as:
>=20
> 	using 1:(N =3D N + 1, sum =3D sum + $2, sum / N)

Adding the comma operator would be relatively easy, I think.

> And even if we're going to do this, the using specification is not=20
> really the right place to do it in.

I entirely agree.  The patches I posted were equivalent to=20
"thinking out loud".  I didn't like either of the initial two
implementations. But I find it very useful to have a version that runs
and that approximates the desired functionality so that I can
experiment with what the new feature allows or doesn't allow.=20

Anyhow, the version now on SourceForge is the first to truly capture
the original intent.  It adds '=3D' as an assignment operator; all else
follows automatically from that.

It needs to be double-checked for operator precedence, possible
memory leaks, etc.  But it is usable for testing as-is.

=2D-=20
Ethan A Merritt