gnuplot-beta — Developers and beta testers of new gnuplot code

Re: Unintended consequences - things to think about

[Ethan A M. <sf...@us...>]

On Thursday, 13 November, 2014 09:20:52 Philipp K. Janert wrote:
> 
> Here are some things to think about:
> 
> 1) We now have loops. But we don't have an iterable data structure.
> No, wait, we have one - we fake it as white-space separated 
> string. (I admit the ingenuity of the thought, but - how kludgy is THAT?)

Beauty is in the eye of the beholder.
Gnuplot has 3 types of variables: integer, string, and complex

There is one simple iterator for integer variables:
  for [ i = min : max : step]

There is one simple iterator for string variables:
  for [s in "A B C D"]

Arguably there is room for an interator over complex values,
but so far no one has convinced me that it is needed.
 
> So, before too long, the loops will beget
> an array. That seems certain.

[shrug]
I can see reasons one might want to have arrays,
but the existence of loops is not a compelling one.

I'd be more inclined to think about adding support
for lists or unordered sets, with a corresponding 
iterator
  for [element in set]


> 2) Once we have arrays, there will be a
> desire to load files into them. Otherwise,
> why have arrays, right? So, arrays will
> beget a "read_file" feature. 
> 
> 3) Now that we can read file(s) into 
> arrays, it's natural that we want to 
> operate on them. One could do that with
> loops, but (again) it's natural to do 
> things like array1 + array2, right? 
> 
> 4) Whereas items 1-3 lie in the future, the 
> following is already an issue today: we have
> loops and conditionals, so it seems natural
> to write functions. Well - we don't HAVE
> functions, but we can fake them as scripts
> that we load with "call". 

"call" is a prime example of a gnuplot feature that I have ignored
for decades.  We continue to support the "call" mechanism
(there's that backwards compatibility promise)
but I honestly don't see a good use for it.
Version 5 provides a slightly more standard/familiar variant that
treats the call parameters like shell variables, with save+restore
of the values. But I doubt I'll ever want to use the new variant either.

> Problem is that
> these scripts don't define local variables -
> any variable I use in my script will become
> a variable in the gnuplot session. To prevent
> clobbering existing variables, I therefore
> need to prepend a prefix to the name of each
> variable "MY_SCRIPT_i", "MY_SCRIPT_j", and so
> on. Very kludgy. Hence, it is reasonable to
> ask for functions with local variables.

Now it seems you are not so much arguing for discussing
new features as you are implicitly arguing that we should
get rid of a feature that has been there for 25 years.
Isn't this the Python mistake?   To break old scripts as a 
consequence of removing an old feature just because
you thought of a better way to do something similar?

If you don't like "call", don't use it.

If you want to propose a syntax for defining fully-scoped function 
blocks, go right ahead (although maybe you are advocating against
this rather than for this; it's hard to tell).

> My point, in case this still isn't clear:
> 
> Once you have one programming feature (say:
> loops), there is a natural tendency for them 
> to require all other programming features 
> (data structures, functions, local scope).
> 
> Therefore, what starts like a local convenience 
> today will - invariably - grow until it has 
> acquired all the features that are already 
> provided by R, matlab, Octave, Perl, Python, Ruby.
> 
> And THAT I think is the wrong direction. If 
> somebody wants R, why would they use gnuplot? 

Because they find it easier to make nice plots?

Let's flip that question around.  If someone is happy with
gnuplot except for some hypothetical missing feature
provided by R, should we not consider adding that missing
feature to gnuplot?

> I also think that it is a misdirection of effort: 
> do we really need to figure out how to design 
> functions with locally scoped variables? This
> has already been done, several times. Would it
> not be better to leverage other people's work?
> 
> If gnuplot needs to acquire programming features
> to stay competitive, then it should try to borrow
> them from somewhere else, where all these problems
> have already been solved - rather than inventing
> the wheel (and poorly, at that).

So how about instead of dragging in lots of bad examples,
you suggest what nice feature from some other language
or program we should consider adopting.

	Ethan (not actually finding this discussion very useful)

Re: Should gnuplot become a programming environment?

[Ethan A M. <sf...@us...>]

On Thursday, 13 November, 2014 08:15:01 Philipp K. Janert wrote:
> 
> What bothers me about those "programming features"
> is that they WILL start a paradigm shift in how
> gnuplot is used, and in what ways it will be 
> developed. Today, you can "ignore" them. But my
> prediction is that before too long, you won't
> anymore, because that's simply "how it's done".

You can still ignore them, even if many other people use them.

> Programming features have a tendency to beget more
> programming features. Look at almost any language
> out there - they all keep growing (C being the 
> exception).
> 
> The horror story here is Python - every single 
> "clever" feature they introduced turned out to
> have edge cases that needed even more, even
> "cleverer" features to handle them properly.

The horror arises largely because Python development assigns
no worth to backward compatibility.  This is exacerbated by the
poorly-chosen guiding principle that there should be only one
way to do any given task.  Together these [mis]features of the
language act to impose a short life expectancy on working code
and carefully developed scripts.

Fortunately gnuplot development has and will continue to 
proceed under the very different guiding principle that 
backward-compatibility is a primary virtue.  If you've been
paying attention, you'll recall that much of the tuning that
has gone into version 5 release candidates -rc2 and -rc3 has
been to restore backwards compatibility wherever the introduction
of new features damaged it.

> Individual arguments aside, I'd like to raise
> the strategic question: does gnuplot want to
> catch up to matlab, R, numpy - is that the right
> direction? 

matlab  -  I've never used it, so no opinion

numpy  -  in a sense, yes.  The value I see in numpy is that it tries to
make up for Python's own failings as a computational language by
making it easy to call into an external numerical library written in a more
suitable language.  The addition of the "import" capability in gnuplot
version 5 is similar, and if it can be further improved, I'm in favor.

R  -  I don't think "catch up" is the right way to view it.
R and gnuplot have complementary strengths.
I'd be interested in discussion about whether there are missing 
pieces on either side (R or gnuplot) that hinder making optimal
use of both tools together.

	Ethan 

Re: Should gnuplot become a programming environment? - If yes, how?

[Philipp K. J. <ja...@ie...>]

[snip]
> 
> Since Ethan seems to do most of the work around here and it was he
> who started all this by implementing my variable assignment
> suggestion  and IFAIK he has done all the other programming feature
> in gnuplot, maybe he has a fair idea where he's going and has given
> it some top-down planning but  never saw the need to bother making a
> formal design study about it.

I am uncomfortable with this line of argument,
because of its personal nature. This is obviously
not saying anything against Ethan - I think we
all know how crucial he has been to gnuplot
development over the last 10-15 years. 

(Thanks, Ethan! I really mean it - without you, 
gnuplot would be dead by now!)

But I'd prefer to keep the argument centered on
the technical merits of features, rather than on 
who conceived and implemented them.

> 
> He seems to have a fairly good idea where he's going with this.

With that in mind, I'd like engage Ethan (and
everyone else) in this discussion. Do we have
a good idea where we are going? 

Best,

		Ph.

Re: Unintended consequences - things to think about

[Karl R. <ra...@un...>]

Am 13.11.2014 um 18:20 schrieb Philipp K. Janert:
> 
> Here are some things to think about:
> 
> 1) We now have loops. But we don't have an
> iterable data structure. No, wait, we have
> one - we fake it as white-space separated 
> string. (I admit the ingenuity of the thought,
> but - how kludgy is THAT?)
> 
> So, before too long, the loops will beget
> an array. That seems certain.

Arrays would be very, very useful for plotting. Think of a multiplot,
one plot with five datasets plus fitted curves, and an inset showing a
regression on the fitted parameters. The simplest way to do that in
gnuplot today is have an external script that generates your gnuplot
script. ;-)


> 2) Once we have arrays, there will be a
> desire to load files into them. Otherwise,
> why have arrays, right? So, arrays will
> beget a "read_file" feature. 

Yes. It´ll be a single command, not interfering with anything. With
that, I could e.g. leave all the fitting to another program (although
gnuplot does a very good job at it! Try fitting in Origin.) and hand the
parameters over to gnuplot in a straightforward manner.

> 3) Now that we can read file(s) into 
> arrays, it's natural that we want to 
> operate on them. One could do that with
> loops, but (again) it's natural to do 
> things like array1 + array2, right?

Same, but who would want it, if not to plot the result, which is very
reasonable for a plotting program?

> 
> 4) Whereas items 1-3 lie in the future, the 
> following is already an issue today: we have
> loops and conditionals, so it seems natural
> to write functions. Well - we don't HAVE
> functions, but we can fake them as scripts
> that we load with "call". Problem is that
> these scripts don't define local variables -
> any variable I use in my script will become
> a variable in the gnuplot session. To prevent
> clobbering existing variables, I therefore
> need to prepend a prefix to the name of each
> variable "MY_SCRIPT_i", "MY_SCRIPT_j", and so
> on. Very kludgy. Hence, it is reasonable to
> ask for functions with local variables.

I agree this would be one step too far, but you can import library
functions today, so there´s no real need for this. And i guess it would
be quite hard to find a (backward compatible) syntax for this, let alone
implementing it.

> I also think that it is a misdirection of effort: 
> do we really need to figure out how to design 
> functions with locally scoped variables? This
> has already been done, several times. Would it
> not be better to leverage other people's work?
> 
> If gnuplot needs to acquire programming features
> to stay competitive, then it should try to borrow
> them from somewhere else, where all these problems
> have already been solved - rather than inventing
> the wheel (and poorly, at that).

As long as "keep it backward compatible" stays the most important
paradigm in the development, i don´t quite see how the "programming
environment" features could get out of hand too badly. It makes sure the
learning curve will stay flat in the beginning.

Interfacing gnuplot with other programs is however an issue today, imo.
That would be, for example, much easier with an array variable that can
be read in from a file, see above, and also keep away further need for
new complex features.

  Karl

Re: Should gnuplot become a programming environment? - If yes, how?

[<pl...@pi...>]

On 11/13/14 17:58, Philipp K. Janert wrote:
>
> One more question in the whole argument:
>
> Even if we agree that adding programming
> functionality to gnuplot is a good thing,
> it nevertheless leaves open the decision
> HOW this should be done.
>
> Is adding convenience features to the core
> language in an essentially ad-hoc manner
> the best strategy?
>
> Or would it make sense to invest some effort
> to have tighter integration with an existing
> programming environment (rather than reinventing
> loops, functions, data structures within gnuplot)?
>
> Or would it be possible to split functionality
> into a "core" (graphics) part and an "optional"
> part, with a clear separation of concerns?
>
> Or can one have an "extension" framework so
> that people can add features without having
> to hack gnuplot backend C code?
>
> The current strategy seems to be that there
> is none. That may give short term wins, but
> the end result is going to be a mess.
>
> Best,
>
> 		Ph.
>

Of course a bit of planning and a design strategy is better than bolt-on 
evolution.

Since Ethan seems to do most of the work around here and it was he who 
started all this by implementing my variable assignment suggestion  and 
IFAIK he has done all the other programming feature in gnuplot, maybe he 
has a fair idea where he's going and has given it some top-down planning 
but  never saw the need to bother making a formal design study about it.

He seems to have a fairly good idea where he's going with this.

Unless I saw something going badly wrong or there was substantial input 
from others that required some team planning and a formal strategy, I'd 
not want start making demands.

Peter.

Re: Unintended consequences - things to think about

[Philipp K. J. <ja...@ie...>]

Here are some things to think about:

1) We now have loops. But we don't have an
iterable data structure. No, wait, we have
one - we fake it as white-space separated 
string. (I admit the ingenuity of the thought,
but - how kludgy is THAT?)

So, before too long, the loops will beget
an array. That seems certain.

2) Once we have arrays, there will be a
desire to load files into them. Otherwise,
why have arrays, right? So, arrays will
beget a "read_file" feature. 

3) Now that we can read file(s) into 
arrays, it's natural that we want to 
operate on them. One could do that with
loops, but (again) it's natural to do 
things like array1 + array2, right? 

4) Whereas items 1-3 lie in the future, the 
following is already an issue today: we have
loops and conditionals, so it seems natural
to write functions. Well - we don't HAVE
functions, but we can fake them as scripts
that we load with "call". Problem is that
these scripts don't define local variables -
any variable I use in my script will become
a variable in the gnuplot session. To prevent
clobbering existing variables, I therefore
need to prepend a prefix to the name of each
variable "MY_SCRIPT_i", "MY_SCRIPT_j", and so
on. Very kludgy. Hence, it is reasonable to
ask for functions with local variables.

(Side question: is that not how matlab 
implements functions - as separate command
files? I seem to remember something like
that. Do we really need to repeat every
mistake that has been made before?)

Anyway...

My point, in case this still isn't clear:

Once you have one programming feature (say:
loops), there is a natural tendency for them 
to require all other programming features 
(data structures, functions, local scope).

Therefore, what starts like a local convenience 
today will - invariably - grow until it has 
acquired all the features that are already 
provided by R, matlab, Octave, Perl, Python, Ruby.

And THAT I think is the wrong direction. If 
somebody wants R, why would they use gnuplot? 

I also think that it is a misdirection of effort: 
do we really need to figure out how to design 
functions with locally scoped variables? This
has already been done, several times. Would it
not be better to leverage other people's work?

If gnuplot needs to acquire programming features
to stay competitive, then it should try to borrow
them from somewhere else, where all these problems
have already been solved - rather than inventing
the wheel (and poorly, at that).

Best,

		Ph.

Re: Should gnuplot become a programming environment?

[<pl...@pi...>]

On 11/13/14 17:15, Philipp K. Janert wrote:
> The horror story here is Python - every single
> "clever" feature they introduced turned out to
> have edge cases that needed even more, even
> "cleverer" features to handle them properly.

When it first looked at python and found out that indentation level was 
part of the language  syntax I got out quick.

It sounds like I was right and made a wise choice.

Peter.

Re: Should gnuplot become a programming environment? - If yes, how?

[Philipp K. J. <ja...@ie...>]

One more question in the whole argument:

Even if we agree that adding programming
functionality to gnuplot is a good thing,
it nevertheless leaves open the decision
HOW this should be done.

Is adding convenience features to the core
language in an essentially ad-hoc manner
the best strategy? 

Or would it make sense to invest some effort 
to have tighter integration with an existing 
programming environment (rather than reinventing 
loops, functions, data structures within gnuplot)?

Or would it be possible to split functionality 
into a "core" (graphics) part and an "optional" 
part, with a clear separation of concerns?

Or can one have an "extension" framework so
that people can add features without having
to hack gnuplot backend C code?

The current strategy seems to be that there
is none. That may give short term wins, but
the end result is going to be a mess.

Best,

		Ph.

Unintended consequences

[Philipp K. J. <ja...@ie...>]

My prime argument against the trend towards 
"programming features" (and data structures)
in gnuplot is the worry about unintended
consequences down the road.

I wrote up the little "play" (below) to
demonstrate what I mean by that. Of course
it's a satire, and therefore exaggerates
to make a point.

(I sent this before (30-Oct), but apparently 
it got lost somehow.)


Tongue-in-cheek, please! 

Here we go...


"All: I just implemented heredocs. Now we can bundle data and 
commands into self-contained command files. That should help 
so much managing the demo/ folder."

"I love it - it really cuts down on clutter in my directories."

"Dick - here is the data set we talked about. I know that you 
are a gnuplot user also, so I am not sending you the data as a 
text file, but as a self-contained gnuplot command file. Just 
load it into gnuplot and off you go! It is very convenient."

"John - here is the data set I got from Jim. He sent it as a
self-contained gnuplot command file. I know that you are not
a gnuplot user, but I am sure you can figure out a way to 
extract the data regardless. You got a PhD and everything."

"Help, my name is John. Somebody sent me some data as a 
self-contained gnuplot command file. What the hell am I
supposed to do with that?"

"Hello, John, and welcome to the gnuplot-data mailing list. 
Good news - we just implemented this new terminal "raw-data".
All you need to do is start gnuplot, load the self-contained 
command file that you got with "load", then set the output 
device using "set output filename" (where filename is the 
name of the file that you would like to export the data to), 
then set the terminal to "raw-data", then run "replot" and 
you are good.

"Oh, I see, that's really easy. By the way: why does gnuplot
have this odd notion of "data files"? Isn't that redundant
with the data section in the self-contained command files?"

"Yes, John, you are right. In fact, plain data files are a
deprecated feature in gnuplot 6; starting with gnuplot 6.2
we plan to only support self-contained command files. By that
time, we should also have finished our very simple and
convenient toolkit to manage self-contained command files."

"I have begun work on gptk (gnuplot toolkit) to handle our
convenient self-contained command files but it needs more 
work. I imagine something as simple and intuitive as Unix tar."

"I am almost done with gptk-cut (to extract the command part)
and gptk-extract (to cut out the data part), I also have made
good progress on gptk-paste, to add a data part to an existing
command part, but I am still working on gptk-list, which shows
an inventory of all data sets contained in a file."

"Well, the release date for gnuplot 6.2 has slipped again, 
mostly because I am not done with gptk-cut-binary to extract
the command part from self-contained command files that contain
a data section in binary format. It works for binary data sets
in PNG, but it does not yet work for the arbitrary binary file
formats that have been part of gnuplot since version 3.7 or
so and that we all agreed to allow inside of self-contained
command files when we prepared gnuplot 5.9."

"All - I just discoverd this new feature in the release candidate 
for gnuplot 7. I takes a plain text file (whitespace-separated
columns) and plots it. Of course it is not as powerful as
the self-contained command files that we all know and love,
but I find it amazing that I can now simply write data into
a file and plot it. I didn't know gnuplot could do that."

Re: Should gnuplot become a programming environment?

[Philipp K. J. <ja...@ie...>]

On Thu, 13 Nov 2014 10:39:16 +0100
pl...@pi... wrote:

> On 11/12/14 07:11, Daniel J Sebald wrote:
> >> Now, gnuplot has begun to become a programming
> >> >environment. This will make gnuplot more
> >> >complicated: more complicated to learn, but
> >> >eventually also more complicated to develop for.
> 
> If it's too complicated to learn the available programming features, 
> don't learn them.

This sounds so right, but I don't think it is.

That argument rests on the assumptions that you
CAN ignore "those features". That's true for (say)
the new "set pointintervalbox" feature (very useful,
by the way) - if you never knew it was there, your
live won't be any different.

What bothers me about those "programming features"
is that they WILL start a paradigm shift in how
gnuplot is used, and in what ways it will be 
developed. Today, you can "ignore" them. But my
prediction is that before too long, you won't
anymore, because that's simply "how it's done".

Programming features have a tendency to beget more
programming features. Look at almost any language
out there - they all keep growing (C being the 
exception).

The horror story here is Python - every single 
"clever" feature they introduced turned out to
have edge cases that needed even more, even
"cleverer" features to handle them properly.

(C stayed small because it was designed to be small.)

> 
> You are then at the same point as if the features were not available
> and gnuplot becomes easier to learn again. ;)
> 
> If you can't  cope with those features, don't  use them when
> "developing for" gnuplot.

Remember: the life you save may be your own.

It's not each individual "programming feature"
that I am concerned about, but their interactions,
long term. Python (again) is a horror story of
"unintended consequences".

Individual arguments aside, I'd like to raise
the strategic question: does gnuplot want to
catch up to matlab, R, numpy - is that the right
direction? (I don't think so, but that's not
important. What IS important is that the matter
is being discussed and pros and contras are
being considered.)

[snip]

> This is a non argument.

I don't think trying to shut down an exchange of 
ideas is a way to move the world forward. Or, to 
use your logic from above: "If you don't like the 
discussion, don't join it."

Peter - you can do better than that!

> 
> Peter.
> 
> 
> ------------------------------------------------------------------------------
> Comprehensive Server Monitoring with Site24x7.
> Monitor 10 servers for $9/Month.
> Get alerted through email, SMS, voice calls or mobile push
> notifications. Take corrective actions from your mobile device.
> http://pubads.g.doubleclick.net/gampad/clk?id=154624111&iu=/4140/ostg.clktrk
> _______________________________________________
> gnuplot-beta mailing list
> gnu...@li...
> Membership management via:
> https://lists.sourceforge.net/lists/listinfo/gnuplot-beta

Re: Should gnuplot become a programming environment?

[<pl...@pi...>]

On 11/12/14 07:11, Daniel J Sebald wrote:
>> Now, gnuplot has begun to become a programming
>> >environment. This will make gnuplot more
>> >complicated: more complicated to learn, but
>> >eventually also more complicated to develop for.

If it's too complicated to learn the available programming features, 
don't learn them.

You are then at the same point as if the features were not available and 
gnuplot becomes easier to learn again. ;)

If you can't  cope with those features, don't  use them when "developing 
for" gnuplot.

If reversing a car is "complicated" and beyond your capabilities, your 
just drive around the block and stop a further up the road. You don't 
start campaigning for the removal of reverse gear from all cars.

This is a non argument.

Peter.

Nov 2014 Update on version 5 progress: -rc3 available

[Ethan A M. <sf...@us...>]

I have placed a source tarball on SourceForge for the
3rd (final) version 5.0 release candidate.

I do not expect much to change between -rc3  and the final 5.0 release,
but if your valuable testing uncovers a problem there is still time to fix it.

The user-visible changes since -rc2 are
=============================

- revised "fit" command 
 + syntax:
       fit {<ranges>} <expression>
           '<datafile>' {datafile-modifiers}
           {{unitweights} | {y|xy|z}error | errors <var1>{,<var2>,...}}
           via '<parameter file>' | <var1>{,<var2>,...}
  + this is a CHANGE from -rc1 and -rc2
  + most version 4 "fit" syntax is detected and emulated even though
     the command options have changed. Also there is now an explicit
     command "set fit {v4|v5}" to force ambiguous commands to be
     interpreted as they were in version 4.

- completion of time/date format changes begun in -rc1 and -rc2
 + formats used for input and output are now distinct
 + fime format used to read input data or range/position commands
    is specified by 'set timefmt' or  timecolumn(column,"timeformat')
 + timecolumn(column) uses default from 'set timefmt' as in version 4
 + time format used to output axis tic labels is specified by
    set {xyz}tics {timedate|geographic|numeric} format "..."
 + new format modifier "t" distinguishes a time interval from a date

- wxt terminal has an Export-to-File widget on the toolbar

- various bug fixes (see NEWS and ChangeLog)

Known issues (no resolution expected before 5.0)
====================================

- the wxt terminal emits runtime warnings and errors if gnuplot is 
  built using wxWidgets 3.0 rather than 2.8


	thanks for all the testing and feedback on -rc1 and -rc2

			Ethan

Re: Musing about stereoscopic plotting.

[sfeam <sf...@us...>]

On Wednesday, 12 November 2014 12:24:26 AM Daniel J Sebald wrote:
> With regard to what I wrote in the previous post about better 
> organization of 2D/3D data aiding new feature development, one could 
> think of the 2D plot as just a projection of 3D data (with added axis 
> decoration capabilities not suitable for 3D).
> 
> This brings to mind something I've wondered about from time to time, 
> which is whether stereoscopic plotting (i.e., two plots side by side, 
> not "stereographic" plotting) is something that would be useful to maybe 
> a few fields like protein analysis, etc. 

This was common 20 years ago, but has been largely replaced by the use
of true stereo, e.g. Zalman monitors, for interactive viewing.
Side-by-side is still used for publication (usually relegated to on-line
supplementary material these days).

In any case the plots one would want this for are not the sort of thing
that gnuplot is good at.  We use specialized programs that know about
protein structure, crystallographic data formats, and relevant conventions
for display of chemical features.

Try an image search for "protein active site stereo" in Google.

	Ethan

> It would be more than merely 
> doing a multiplot, for example.  Rotating the mouse would rotate the two 
> images in concert to give the illusion of rotating a 3D object.  I've 
> limited experience with stereoscopic "projection", and maybe it is more 
> difficult than I imagine.  (There probably not only needs to be viewing 
> angles, but depth of view, or maybe that's called parallax.)
> Nonetheless, it would be interesting.  But again, gnuplot internals 
> would need to be revamped for something sophisticated like that.
> 
> Dan

Musing about stereoscopic plotting.

[Daniel J S. <dan...@ie...>]

With regard to what I wrote in the previous post about better 
organization of 2D/3D data aiding new feature development, one could 
think of the 2D plot as just a projection of 3D data (with added axis 
decoration capabilities not suitable for 3D).

This brings to mind something I've wondered about from time to time, 
which is whether stereoscopic plotting (i.e., two plots side by side, 
not "stereographic" plotting) is something that would be useful to maybe 
a few fields like protein analysis, etc.  It would be more than merely 
doing a multiplot, for example.  Rotating the mouse would rotate the two 
images in concert to give the illusion of rotating a 3D object.  I've 
limited experience with stereoscopic "projection", and maybe it is more 
difficult than I imagine.  (There probably not only needs to be viewing 
angles, but depth of view, or maybe that's called parallax.) 
Nonetheless, it would be interesting.  But again, gnuplot internals 
would need to be revamped for something sophisticated like that.

Dan

Re: Should gnuplot become a programming environment?

[Daniel J S. <dan...@ie...>]

[Wrote this more than a week ago, then set in aside in draft box.]

On 10/29/2014 06:52 PM, Philipp K. Janert wrote:
>
> (This started as a discussion on heredocs, but
> is really about a bigger-picture-issue. So, I'll
> try to restart this thread again.)
>
> Recently, gnuplot has begun to acquire features
> that are typical of a programming language:
>
> - loops and conditionals
> - structured data types
> - statefulness and persistence.
>
> I don't think this is a good trend, and I'd like
> to raise awareness for what's happening.
>
> Gnuplot used to be a simple plotting tool - that
> was its weakness, but also its strength.
>
> Now, gnuplot has begun to become a programming
> environment. This will make gnuplot more
> complicated: more complicated to learn, but
> eventually also more complicated to develop for.

Point taken about the simplicity of the language, but I think that 
argument is disconnected from the development aspect of gnuplot for a 
couple reasons.

First, gnuplot is still one of the most compact programs I've seen.  It 
compiles in about 45 seconds on my system and making changes and 
recompiling is extremely fast.  Other significant programs can take tens 
of minutes to compile because of overheads for libraries, definitions, 
macros, etc.  In some bigger programs I'm sometimes hesitant to touch a 
header file because it will mean minutes of recompiling.

Second, and the more significant point, maintenance of these small 
features pales in comparison to the overall organization of the code.  A 
little more OOP concepts would aid development far more than the burden 
of the small features.  (The code has actually improved over the years. 
  E.g., barely any global variables left.)  The internal storage of data 
with the 2D version and 3D version is a bit of a headache.  Trying to 
keep track of whether z is being used for dimension or color comes to 
mind.  I wish 2D data was pretty much the same as 3D but with one 
dimension zero.  Overall data flow from input to processing to plotting 
could maybe be cleaner and more flexible.  The gains from code cleanup 
and layout would outweigh these small features, and help for new bigger 
features.

Dan

Re: Clipboard formats and Qt and wxt terminals

[Daniel J S. <dan...@ie...>]

On 11/10/2014 12:24 PM, Daniel J Sebald wrote:

> Might MIME be something used in clipboards these days? Here is the MIME
> for svg:
>
> image/svg+xml

A bit more info:

http://www.ietf.org/rfc/rfc3023.txt

8.19 Image/svg+xml

    Content-type: image/svg+xml

    <?xml version="1.0" ?>

    Scalable Vector Graphics (SVG) documents are XML documents whose
    content describes graphical information, as defined by [SVG].  As a
    format based on XML, SVG documents SHOULD use the '+xml' suffix
    convention in their MIME content-type identifier.  However, no
    content type has yet been registered for SVG and so this media type
    should not be used until such registration has been completed.

Clipboard formats and Qt and wxt terminals

[Daniel J S. <dan...@ie...>]

[Change of subject thread from "Re: Proposal: Saving graphs from Qt and 
wxt terminals"]

On 11/10/2014 10:54 AM, Jun T. wrote:
> 2014/11/09 16:26, Daniel J Sebald<dan...@ie...>  wrote:
>> Jun, do you have some program/application in mind that might accept PDF from the clipboard?
>
> Sorry, I'm not using Linux for creating documents or read/write mails.
>
> On Mac OS X, aquaterm (set term aqua) can copy a plot in PDF, and many apps
> (Pages, Keynote, MS Office, many drawing/graphics apps, etc.) accept it. If I
> paste it to a composing window of Mail.app, it becomes an attachment in PDF.
>
> Probably there are only few gnuplot users on Mac. If so, supporting PDF
> clipboard by wxt/qt on Mac would not be a high priority.
> Aquaterm works fine and, for most of the cases, enough for me.
>
> But I guess copying a plot in a vector format (SVG ?) would also be useful on
> Linux (assuming there are many apps which accept SVG from clipboard).
> Or is this already possible?

Useful?  Probably.  Already possibly?  Don't know.  But from this FAQ on 
something called Inkscape, my guess would be "No" or "Maybe, but not 
fundamentally":

http://wiki.inkscape.org/wiki/index.php/Frequently_asked_questions#Copying_and_pasting_in_Inkscape_creates_pixellated_images_instead_of_copying_the_vector_objects

The way the answer is worded suggests that Mac OS X clipboard is more 
robust than X11 clipboard.  That's likely the main difference here, and 
in the case of Linux it sounds like GTK/KDE may have clipboard support 
which is different--my guess being that KDE attempts to extend the X11 
clipboard capabilities more than GTK does.

I've searched for some definitive X11 documentation on the limitations 
and and future changes/additions for clipboard support, e.g.,

https://www.xpra.org/trac/wiki/Clipboard

and I'm none the wiser for it.

The questions I wonder are, Why the X11 clipboard doesn't seem to have 
emerging, more robust support for formats? and Is there something that 
has emerged that serves as a standard for more formats thereby dampening 
the need for X11 to add better support?

Might MIME be something used in clipboards these days?  Here is the MIME 
for svg:

image/svg+xml

The Qt documentation has a discussion about MIME (whereas wxWidgets 
appear not to):

http://qt-project.org/doc/qt-4.8/qclipboard.html#setMimeData

Quote:

QClipboard features some convenience functions to access common data 
types: setText() allows the exchange of Unicode text and setPixmap() and 
setImage() allows the exchange of QPixmaps and QImages between 
applications. The setMimeData() function is the ultimate in flexibility: 
it allows you to add any QMimeData into the clipboard.

Notice "ultimate in flexibility".  Does that apply only to email 
interchange?  Or does it open the door to many applications on an 
X11-based desktop environment?

Perhaps I'll switch over to Qt terminal and experiment there a bit.

Dan

Re: Proposal: Saving graphs from Qt and wxt terminals

[Jun T. <tak...@kb...>]

2014/11/09 16:26, Daniel J Sebald <dan...@ie...> wrote:
> Jun, do you have some program/application in mind that might accept PDF from the clipboard?

Sorry, I'm not using Linux for creating documents or read/write mails.

On Mac OS X, aquaterm (set term aqua) can copy a plot in PDF, and many apps
(Pages, Keynote, MS Office, many drawing/graphics apps, etc.) accept it. If I
paste it to a composing window of Mail.app, it becomes an attachment in PDF.

Probably there are only few gnuplot users on Mac. If so, supporting PDF
clipboard by wxt/qt on Mac would not be a high priority.
Aquaterm works fine and, for most of the cases, enough for me.

But I guess copying a plot in a vector format (SVG ?) would also be useful on
Linux (assuming there are many apps which accept SVG from clipboard).
Or is this already possible?

Re: Lua/Tikz terminal behavior

[Mojca M. <moj...@gm...>]

On Sun, Nov 9, 2014 at 4:12 AM, Philipp K. Janert wrote:
>
> For what it's worth, here are some differences
> between the TikZ and cairolatex:
>
> 1) TikZ creates only ONE file, whereas cairolatex
>   needs two (one for the actual graph, one for the
>   labels).
> 2) The .tex files created by TikZ may be large,
>   because they must contain all the data (or, for
>   function plots, the fct evaluated at all points).
>   (Apparently, these large tex files can cause memory
>   problems for Latex, but I have not pushed it to that
>   point.)

In the beginning when I was developing the "context" terminal (very
similar to TikZ, but TikZ didn't yet exist at that time) I often
tested with "load all.dem" and ran into "memory limit exceeded" very
often, in particular with all those 100x100 or 200x200 image files
drawn as 40.000 separate rectangles, each rectangle taking up three
not so short lines of text.

In LuaTeX most of those memory limits are gone compared to pdfTeX, so
it's easier and faster to process the graphics, but it doesn't change
the fact that some graphs need tens of thousands of not-so-short lines
which takes a lot of time to read the graphic in. In such cases of
lots of graphical elements using something that creates a separate PDF
is way more efficient.

Cairolatex didn't exist when TikZ was created.

Mojca

Re: Is th number of dashpatterns limited?

[Philipp K. J. <ja...@ie...>]

On Sun, 09 Nov 2014 15:30:48 -0800
sfeam <sf...@us...> wrote:

> On Sunday, 09 November 2014 11:00:37 AM Philipp K. Janert wrote:
> > 
> > Question: is the number of user-defined
> > dashpatterns limited? When using 
> > 	set dashpattern N (...)
> > any definition past N=6 seems to be ignored.
> 
> Not correct. 
> This works fine:
> 
> set dashtype 29 '.--.'
> plot x dt 29
> 

Ok, thanks. I must be doing something wrong -
let me check.

Re: Is th number of dashpatterns limited?

[sfeam <sf...@us...>]

On Sunday, 09 November 2014 11:00:37 AM Philipp K. Janert wrote:
> 
> Question: is the number of user-defined
> dashpatterns limited? When using 
> 	set dashpattern N (...)
> any definition past N=6 seems to be ignored.

Not correct. 
This works fine:

set dashtype 29 '.--.'
plot x dt 29

Is th number of dashpatterns limited?

[Philipp K. J. <ja...@ie...>]

Question: is the number of user-defined
dashpatterns limited? When using 
	set dashpattern N (...)
any definition past N=6 seems to be ignored.

Best,

		Ph.

Re: Proposal: Saving graphs from Qt and wxt terminals

[Daniel J S. <dan...@ie...>]

On 11/09/2014 01:26 AM, Daniel J Sebald wrote:

> So, right now the capability is there, but the utility of it is in
> question.  But this brings to mind that HTML is one of the internal
> clipboard formats.  I think I'm going to try figuring out how to place
> and HTML version of the plot in the clipboard along with the BITMAP (it
> can be multiple types of data).  It seems to me that being able to paste
> into an email composer is one of the nicer features, as it means one
> doesn't have to first save the plot to a file, then to email composer.

Are others able to copy from the WXT terminal and paste into HTML-based 
email composers?  Yes or no, please indicate your desktop environment 
and email composer.

Thanks,

Dan

Re: Proposal: Saving graphs from Qt and wxt terminals

[Daniel J S. <dan...@ie...>]

On 10/31/2014 02:50 PM, Daniel J Sebald wrote:
> On 10/31/2014 02:21 PM, Ethan A Merritt wrote:
>> On Saturday, 01 November, 2014 03:57:01 Jun T. wrote:
[snip]
>>>   When copying the plot to the clipboard, I prefer PDF format rather than
>>
>>>   PNG (either on wxt or qt). Is this possible?
>>
>> I don't know.
>
> I think it would be possible to derive a custom button from the standard
> button and add a right mouse click handler to the event table.  How
> about right mouse click on the clipboard icon opens a dialog or drop
> down menu that configures the image format saved to file?  To make
> things more clear, the format, e.g., "png", "pdf", "svg" could be
> superimposed on the clipboard icon with the format selection.  And maybe
> change the tool tip to read "Left Click: Copy, Right Click: Configure".

Ethan and I looked at the clipboard a bit.  He wondered if it even makes 
sense to put something in the clipboard of a particular format like PDF, 
or PNG.  I'm starting to wonder that as well.

There is a way to create a PNG type of bitmap and then place that in the 
clipboard.  However, I think this is done by converting that PNG bitmap 
to an internal clipboard BITMAP.

I can create custom data objects to place in the clipboard and assign 
them names like "png", but other programs don't seem to highlight the 
"Paste" text in the menu for a data format like that.  (I.e., the 
calling program doesn't know what a "png" data object is.)

Jun, do you have some program/application in mind that might accept PDF 
from the clipboard?

So, right now the capability is there, but the utility of it is in 
question.  But this brings to mind that HTML is one of the internal 
clipboard formats.  I think I'm going to try figuring out how to place 
and HTML version of the plot in the clipboard along with the BITMAP (it 
can be multiple types of data).  It seems to me that being able to paste 
into an email composer is one of the nicer features, as it means one 
doesn't have to first save the plot to a file, then to email composer.

Dan

Re: Lua/Tikz terminal behavior

[Philipp K. J. <ja...@ie...>]

On Sat, 08 Nov 2014 15:20:14 -0800
sfeam <sf...@us...> wrote:

> On Saturday, 08 November 2014 07:12:43 AM Philipp K. Janert wrote:
> > 
> > Thanks all for your comments.
> > I am still looking for the advantages over,
> > say, the epslatex terminal.
> 
> Specifically in comparison to epslatex
> 1) epslatex produces *.eps, not *.pdf, so not as nice if you use
> pdflatex 2) eoslatex does not support transparency or alpha channel
> images.
> 
> A more equal comparison would be to 'set term cairolatex pdf'
> 

Yes, of course. (I am soooo 2008.)

For what it's worth, here are some differences
between the TikZ and cairolatex:

1) TikZ creates only ONE file, whereas cairolatex
  needs two (one for the actual graph, one for the
  labels).
2) The .tex files created by TikZ may be large,
  because they must contain all the data (or, for
  function plots, the fct evaluated at all points).
  (Apparently, these large tex files can cause memory
  problems for Latex, but I have not pushed it to that
  point.)
3) If you are good at Tex AND TikZ, then editing
  the generated file is a real option (whereas the 
  files created by cairolatex do not seem intended
  for manual editing).

Finally, TikZ of course draws all lines (and points,
and ...) itself, whereas cairolatex lets gnuplot 
draw the graphic elements. In principle, this gives
you access to TikZ's greater/different capabilities,
compared to gnuplot (if you know TikZ).

Best,

		Ph.