gnuplot-beta — Developers and beta testers of new gnuplot code

Re: Lengthy discussion about datafile.c...

[Ethan M. <merritt@u.washington.edu>]

On Saturday 14 August 2004 03:50 pm, Daniel J Sebald wrote:
> Ethan Merritt wrote:
> >Didn't we already have this discussion a few months ago?
> >I proposed that the whole notion of tracking input data by how
> >many columns were read in has outlived its usefulness.
> >I think we should get rid of max_cols and all the various
> >tests that depend on it, and instead pass explicit information
> >about the requested input data.  Hans-Bernard disagreed

Oh, and apologies for the typo in Hans-Bernhard's name.

> I know that for ASCII files the number of columns can be determined by
> the file itself and gnuplot readjusts accordingly.

Which brings up another issue.  The description of your binary read
"format" commands looks *really* fragile.  I mean the stuff being parsed
in plot_option_binary_format(). I am seriously worried that
it won't transfer well across 32/64 bit machines, that it won't handle
string data, and worst of all that it requires too much user-knowledge
of file and data types.  Basically I don't like it.

[EAM puts on geezer hat] In the old days of Fortran programming and 
VMS file systems, binary files had actual "records".   In those days
there was an obvious parallel between "columns" in an ascii file
and "records" in a binary file.  But that approach has been drowned
by the unix notion that "everything is a stream of bytes".  
It's *really hard* to figure out what data is in a binary stream, and I
am dubious that it is worth spending thousands of lines of code in
gnuplot trying to do so.  The unix way in such a case would be to
run the input binary data through a tailored filter on its way into
gnuplot.  That way gnuplot only has to know about ascii input, and
you can debug a suitable filter for your application without having
to recode gnuplot.  


Your docs say  
	+ Gnuplot will retrieve a number of binary
	+ variables equal to the largest column specified in the `<using list>`.
	+ For example, `using 1:3` would cause three columns to be read, of which
	+ the second will be ignored. 
So how do you handle the case of 10 logical columns of data in the file,
of which you only want to read the 2nd and 4th?  How do you skip "columns"
5 to 10 of each "line"?

What constitutes the logical equivalent of a "blank line" in your binary
files? Or is there no equivalent to the auto-determination of scan lines?

Do you plan to handle strings?  How? Would you require a full "binary format"
description in this case?  Is there such a thing as a matrix of strings?

The matrix variant is far more straight-forward.  I would think this will be 
by far the most common use anyhow, and it would cover the pixel images
that you obviously have fondness for. Could we maybe have a first cut
version of this patch that only deals with matrix format binary data?

Re: Lengthy discussion about datafile.c...

[Ethan M. <merritt@u.washington.edu>]

On Saturday 14 August 2004 03:50 pm, you wrote:
>
> OK, let me back up here.  I think I see now the more important issue
> here is that the data to be plotted, the imigration.dat file for
> example, won't work because it has more columns than allowed by max_cols
> passed into the df_readline routine.

No, that's completely wrong.  The data is being plotted 1 column at
a time.  Sure there are lots of columns in the data file, but there's 
nothing special about that.

>     **
> "string"   "string"   ....   "string"
> <data>     <data>    ....   <data>

> where the data which is to serve as the tic labels (read as a string
> rather than a number) is contained in the ** element.

No.  The scheme you describe does
not correspond to any of the histogramming modes I implemented.
Just forget about histograms for the purpose of this discussion.

You are going way off on a tangent because you have 
misunderstood how the current code works.  I don't ever
need to read in more than 2 columns of data for histogramming.
And the histogram code doesn't use strings anyhow, except
insofar as you normally want to specify tic labels to go with
your histograms.  But the tic label business and the histograms
are separate bits of code.

[snip]
> No, certainly I could add reading strings from binary data files.  But I
> would propose making it a generic thing.  Say for example, a command
> line syntax (or it doesn't have to be command line, it could be an
> internal variable) whereby one of the columns can be designated as a
> string tic label, e.g., "ticlabel <col>", or whatever.  But it's meaning
> is generic, it is just a string passed back and treated accordingly.  In
> the case of histograms it is a tic label.  Perhaps something different
> for something else.

Ugh.  Daniel.  Please try to understand what the current code is
doing. You're just so far off base I don't know how to reply to
your comments except to say they are irrelevant.

Hint:  In the current code every requested column is returned
twice, once as a number and once as a string.  The caller can 
choose whether it wants the string value or the numeric value.
I don't know how this fits in with your binary data files, but I
assure you it is fully generic.  The histogram code doesn't use
this anyhow; histogramming is not about strings, it's about
columns of numbers.   You are, I am guessing, thinking about
my *other* new plotting mode - 'with labels'.


> Does this get around some problems?  Am I understanding the big issue
> now, that there are more columns now than max_cols?

No. Nothing at all like that.  I want to get rid of max_cols not because
I want lots of columns, but just because it is not used for anything that
really has to do with columns.  It is only used to try to deduce back to
what the plot type is - which I think is nuts. If you need to know the plot
type then just pass the plot type.

> Well, here is the thing.  There is a certain element of this that can't
> be disentangled (if that's a word).  A lot of the parameters for reading
> from a file are set up by df_open() because it is there that the
> keywords from the command line are processed.  So, at the point of
> df_open() it isn't known yet whethere the file is ascii or binary.  That
> could be fixed by first, at the start of df_open, checking all the
> keywords to see if one is "binary", but that's not graceful.

Hunh?  Now I'm the one who is confused.  I really have not been
looking at that part of your patch because I have no use for binary
input.  But I assumed you told the program *somehow* that this
was a binary data file.  How does this work at all if there isn't a
keyword on the command line?

> Let me make this revision, and maybe that will help things fall in place.

OK.  I will wait and have a look at it.  But I seriously hope that you can
basically leave all the existing code in df_readline() untouched.

Re: Lengthy discussion about datafile.c...

[Daniel J S. <dan...@ie...>]

Ethan Merritt wrote:

>On Saturday 14 August 2004 11:56 am, Daniel J Sebald wrote:
>  
>
>>Hope this doesn't sound like a lecture, but I want to discuss how to
>>keep datafile.c clean and prevent the evolution of convoluted code that
>>is starting to occur with that file.
>>    
>>
>
>[snip lengthy rant, some of which is on target, some not]
>

:-)


>Didn't we already have this discussion a few months ago?
>I proposed that the whole notion of tracking input data by how
>many columns were read in has outlived its usefulness.
>I think we should get rid of max_cols and all the various
>tests that depend on it, and instead pass explicit information
>about the requested input data.  Hans-Bernard disagreed.
>

Well, yeah.  Lot's of disagreement; but not much agreement and what the 
paradigm should be and I'm suggesting that there be some agreement to 
avoid too much divergence.  The discussion sort of faded...

>>df_open(int max_using, int plot_mode)
>>    
>>
>
>Like that, yes, except that 
>(1) I think max_using is not necessary or desirable, and 
>(2) I proposed passing a pointer to the whole plot
>structure rather than passing only the plot style.
>

Another paradigm is fine, passing in a pointer to the whole plot is 
fine.  Just not a combination of multiple views.  Some of the image 
stuff may fit to one or the other paradigms, so it might be good to 
adhere to only one in the near future.

I know that for ASCII files the number of columns can be determined by 
the file itself and gnuplot readjusts accordingly.  That code is 
currently in plot2d.c.  That will remain there?  Or will that be moved 
to inside datafile.c as part of df_readline?  df_open?

>>While on this topic of df_readline, I wonder if introducing too much
>>"plot dependent" stuff into df_readline is a good idea.
>>    
>>
>
>And that was Hans-Bernard's counterargument.
>
>  
>
>>For example, 
>>with the histogram tics, this kind of line seems like it shouldn't be in
>>a file-reading routine:
>>
>>             add_tic_user(axis,temp_string,xpos,0);
>>
>>Is there some way to move this functionality outside of df_readline()
>>back into plot2d.c?
>>    
>>
>
>Why?  It is not specific to 2D plots.  But anyway, the answer is no.
>The information being processed, the tic labels, are not specific to 
>the current plot; they are a property of the axis.  The code belongs
>in axis.c, which is where it is currently.  But still you have to call it
>from somewhere, and I maintain the logical place (maybe the only
>possible place) is the point at which you obtain the information.
>That is set.c in the case of axis tic info coming from a "set [xyz]tics"
>command, and datafile.c in the case of tic info read in from a file.
>

OK, let me back up here.  I think I see now the more important issue 
here is that the data to be plotted, the imigration.dat file for 
example, won't work because it has more columns than allowed by max_cols 
passed into the df_readline routine.  That is, the normal gnuplot ascii 
file looks like

"string" **
<data>

"streing"
<data>

"string"
<data>

But the 'imigration.dat' file is

    **
"string"   "string"   ....   "string"
<data>     <data>    ....   <data>

where the data which is to serve as the tic labels (read as a string 
rather than a number) is contained in the ** element.

I don't have all the answers, but I'll make some comments.  In the 
latter case, those bunch of strings at the start of the file could all 
be read at once.  In fact, it is almost similar in strategy to the 
"gnuplot binary" type of file where along the top is the x values and 
the first column afterward is the y_values.  Also, the df_readline() 
routine might be easily arranged to remove the max_cols restriction and 
make the value of j returned dynamic, from 2, 3, 4, etc. all the way up 
to 500 if one wants.  It may just mean dynamic alocation of memory (that 
doesn't need to be reallocated if the number of read values doesn't 
change, thus saving efficiency).

>>I pose this question because I've been trying to make the case that
>>df_readascii() and df_readbinary(), or whatever, should be transparent
>>to the calling routine.  If functionality like above keeps being added
>>to df_readascii (df_readline) then soon the situation arises where
>>certain types of plots can't be done simply because the data comes from
>>a binary data file.
>>    
>>
>
>If that is indeed true then I have reservations about introducing binary
>input at all.  Are you saying that it will not be possible to read strings
>in from a binary file, so that the new "plot with labels" and 
>"using ...:xticlabels(<col>)" will not work?  If so, then the functionality
>has already diverged.  And if the two modes have different capabilities
>then all the more reason to keep them separate in the code as well.
>

No, certainly I could add reading strings from binary data files.  But I 
would propose making it a generic thing.  Say for example, a command 
line syntax (or it doesn't have to be command line, it could be an 
internal variable) whereby one of the columns can be designated as a 
string tic label, e.g., "ticlabel <col>", or whatever.  But it's meaning 
is generic, it is just a string passed back and treated accordingly.  In 
the case of histograms it is a tic label.  Perhaps something different 
for something else.

However, my feeling about df_readline(), df_readascii(), df_readbinary() 
are that these should be core little routines (in scope anyway), a 
kernel if you will, that takes in data and shuffles it off to somewhere 
else to be processed further.

If one mixes dedicated code like plot->histogram, etc. into 
df_readascii(),  then they also need to remember to make that change in 
df_readbinary().  If it is tweak in one location, then it has to be 
touched in another spot, which might go forgotten.

So, maybe a version of df_readline as follows:

int df_readline(double vector[], char **string)

where now the vector can be of any length, and the string is a location 
where df_readline is to put a pointer to a character string that it 
dynamically allocates.  (It can be one, at most, of the columns treated 
as a string rather than a number.)

Does this get around some problems?  Am I understanding the big issue 
now, that there are more columns now than max_cols?  I guess I'm asking 
that if the max_cols restriction were dropped, would the current set up 
allow you to move data into the plot structure as desired?  Is there a 
paradigm shift here for the way data can be arranged in the file for 
histograms?



>>Ethan, what is the minimal amount of information that you would need
>>coming back from df_readline() to implement headers from files?  If
>>df_readline() were equipped with a char pointer for which df_readline
>>could realloc() memory and assign a string, would that do it?
>>    
>>
>
>That's what it does now. Because plot->title is not visible from
>inside df_readline (which actually I would prefer), the title is allocated
>and a pointer to it is stored in a static variable. A helper routine 
>df_set_key_title() is later called from get_data(), which is indeed in 
>plot2d.c.  No global variables are involved.
>

Yeah, that is fine.  I assume that df_set_key_title() is not within 
df_readlin().  My major point in all this is to keep df_readline() clean 
and generic, and in the long run it will promote happiness.

>>That is, I might propose
>>             add_tic_user(axis,temp_string,xpos,0);
>>could be moved to plot2d.c. 
>>    
>>
>
>You are confusing plot titles and axis tic labels.  The two things are
>quite different.  One is a specific property of the current plot,
>the other is not.
>
>I know, you are going to point to a single place where the histogram 
>code stuffs a plot title into an axis tic label.  I'm not terribly happy
>about that either, but let's split that off into a totally separate 
>discussion that only applies to stacked histograms.
>

No biggie.  Got to start somewhere.

>>PS:  I've concluded that moving df_readbinary() to another file would
>>require the sharing of too many "local" variables.
>>    
>>
>
>I don't agree.  Most of those local variables are indeed local.
>They should not *need* to be shared.
>

Well, here is the thing.  There is a certain element of this that can't 
be disentangled (if that's a word).  A lot of the parameters for reading 
from a file are set up by df_open() because it is there that the 
keywords from the command line are processed.  So, at the point of 
df_open() it isn't known yet whethere the file is ascii or binary.  That 
could be fixed by first, at the start of df_open, checking all the 
keywords to see if one is "binary", but that's not graceful.  So, yes 
even a df_open_binary() could be generated where all the keywords are 
again interpretted.  But why repeat all these in a different file if 
they are going to be pretty much the same?  "every" works the same, 
"thru" works the same, etc.

Let me make this revision, and maybe that will help things fall in place.

Dan

Re: Lengthy discussion about datafile.c...

[Ethan M. <merritt@u.washington.edu>]

On Saturday 14 August 2004 11:56 am, Daniel J Sebald wrote:
> Hope this doesn't sound like a lecture, but I want to discuss how to
> keep datafile.c clean and prevent the evolution of convoluted code that
> is starting to occur with that file.

[snip lengthy rant, some of which is on target, some not]

Didn't we already have this discussion a few months ago?
I proposed that the whole notion of tracking input data by how
many columns were read in has outlived its usefulness.
I think we should get rid of max_cols and all the various
tests that depend on it, and instead pass explicit information
about the requested input data.  Hans-Bernard disagreed.

> df_open(int max_using, int plot_mode)

Like that, yes, except that 
(1) I think max_using is not necessary or desirable, and 
(2) I proposed passing a pointer to the whole plot
structure rather than passing only the plot style.

> While on this topic of df_readline, I wonder if introducing too much
> "plot dependent" stuff into df_readline is a good idea.

And that was Hans-Bernard's counterargument.

> For example, 
> with the histogram tics, this kind of line seems like it shouldn't be in
> a file-reading routine:
>
>              add_tic_user(axis,temp_string,xpos,0);
>
> Is there some way to move this functionality outside of df_readline()
> back into plot2d.c?

Why?  It is not specific to 2D plots.  But anyway, the answer is no.
The information being processed, the tic labels, are not specific to 
the current plot; they are a property of the axis.  The code belongs
in axis.c, which is where it is currently.  But still you have to call it
from somewhere, and I maintain the logical place (maybe the only
possible place) is the point at which you obtain the information.
That is set.c in the case of axis tic info coming from a "set [xyz]tics"
command, and datafile.c in the case of tic info read in from a file.

> I pose this question because I've been trying to make the case that
> df_readascii() and df_readbinary(), or whatever, should be transparent
> to the calling routine.  If functionality like above keeps being added
> to df_readascii (df_readline) then soon the situation arises where
> certain types of plots can't be done simply because the data comes from
> a binary data file.

If that is indeed true then I have reservations about introducing binary
input at all.  Are you saying that it will not be possible to read strings
in from a binary file, so that the new "plot with labels" and 
"using ...:xticlabels(<col>)" will not work?  If so, then the functionality
has already diverged.  And if the two modes have different capabilities
then all the more reason to keep them separate in the code as well.

> Ethan, what is the minimal amount of information that you would need
> coming back from df_readline() to implement headers from files?  If
> df_readline() were equipped with a char pointer for which df_readline
> could realloc() memory and assign a string, would that do it?

That's what it does now. Because plot->title is not visible from
inside df_readline (which actually I would prefer), the title is allocated
and a pointer to it is stored in a static variable. A helper routine 
df_set_key_title() is later called from get_data(), which is indeed in 
plot2d.c.  No global variables are involved.

> That is, I might propose
>              add_tic_user(axis,temp_string,xpos,0);
> could be moved to plot2d.c. 

You are confusing plot titles and axis tic labels.  The two things are
quite different.  One is a specific property of the current plot,
the other is not.

I know, you are going to point to a single place where the histogram 
code stuffs a plot title into an axis tic label.  I'm not terribly happy
about that either, but let's split that off into a totally separate 
discussion that only applies to stacked histograms.

> PS:  I've concluded that moving df_readbinary() to another file would
> require the sharing of too many "local" variables.

I don't agree.  Most of those local variables are indeed local.
They should not *need* to be shared.

> So I'm thinking that 
> just putting it to the bottom of datafile.c is best... that's actually
> where it probably should be organized anyway.

All right.  Do that as a first step.  At least it may disentangle the
code paths enough that I can do an xxdiff between the new and
old versions to see what you are actually changing.  Right now
it's such a tangle that I don't have an overall sense of what is
shared and what isn't.

Re: Lengthy discussion about datafile.c...

[Daniel J S. <dan...@ie...>]

Daniel J Sebald wrote:

> In having done quite of bit of modular and object oriented code for a 
> few years, I'd like to discourage the use of global variables across 
> two conceptually different pieces of code.  In particular, in this 
> instance is the global variable "df_datum".


I see now there are several variables made global outside datafile.c:

 * public variables declared in this file.
 *    int df_no_use_specs - number of columns specified with 'using'
 *    int df_no_tic_specs - count of additional ticlabel columns
 *    int df_line_number  - for error reporting
 *    int df_datum        - increases with each data point
 *    TBOOLEAN df_binary  - it's a binary file
 *        [ might change this to return value from df_open() ]
 *    int df_eof          - end of file
 *    int df_timecol[]    - client controls which cols read as time

Well, then its starting to get to be a lot of work to adhere to a 
modular strategy.  Anyway, things like df_line_number could be done 
differently.  From what I see, that variables is used outside of 
datafile.c only as an argument to error messages when things fail, e.g.,

            int_error(this_plot->token,
                  "2 columns only possible with explicit pm3d style 
(line %d)",
                  df_line_number);

Another approach might be to create a funciton df_int_error() which acts 
just like int_error() but in addition tags on information about the file 
line number it is currently at then calls int_error().  That way, error 
messages for the line number can be made consistent and easily alter if 
for some reason more information is to be added.

Dan

Re: Option 'ai' in Postscript terminal

[Ethan M. <merritt@u.washington.edu>]

On Saturday 14 August 2004 12:10 pm, Daniel J Sebald wrote:
> The filter Ethan refers to is the ASCII85Decode filter, which I'm
> guessing would have been in PostScript from the beginning.

No. I am referring to the PostScript command "filter",
which is a Level 2 extension.

Re: Option 'ai' in Postscript terminal

[Daniel J S. <dan...@ie...>]

Harald Harders wrote:

>>However, Daniel's upcoming binary + image patch depends on the
>>Level2 operator "filter", so we will have to keep an eye on that.
>>    
>>
>
>We should speak to him to ensure that also the filter operator is not used
>when using level1.
>

Here are the commands that the image routine uses:

%%%%BeginImage
gsave
translate
scale
[ M 0 0 N 0 0 ]
currentfile /ASCII85Decode filter
false
%%%%BeginPalette
[ /Indexed\n  /DeviceRGB
<bunch of characters for palette>
] setcolorspace
%%%%EndPalette
<<
/ImageType 1
/Width M
/Height N
/BitsPerComponent
/ImageMatrix [ M 0 0 N 0 0 ]
/Decode [ 0 #]
/DataSource currentfile /ASCII85Decode filter
/MultipleDataSources false
/Interpolate false
 >>
%%%%BeginData
colorimage
image
<bunch of characters for image>
%%%%EndData
grestore
%%%%EndImage

The only non level-1 suspect commands might be

/DataSource currentfile
/Interpolate false

The filter Ethan refers to is the ASCII85Decode filter, which I'm 
guessing would have been in PostScript from the beginning.  The reason 
for the /Interpolate false is that for a scientific plotting program 
there shouldn't be any additional image processing done beyond what the 
user may have applied or intended.

Dan

Lengthy discussion about datafile.c...

[Daniel J S. <dan...@ie...>]

  Hope this doesn't sound like a lecture, but I want to discuss how to 
keep datafile.c clean and prevent the evolution of convoluted code that 
is starting to occur with that file.

In having done quite of bit of modular and object oriented code for a 
few years, I'd like to discourage the use of global variables across two 
conceptually different pieces of code.  In particular, in this instance 
is the global variable "df_datum".  Let's begin there.

I see that before the histogram strings code was added, df_datum was 
used outside of datafile.c in one--and only one--case, plot2d.c.  I 
would discourage making df_datum available to the outside world in 
datafile.h.  Here is its use:

    case 0:        /* not blank line, but df_readline couldn't parse it */
        {
        df_close();
        int_error(current_plot->token,
              "Bad data on line %d", df_line_number);
        }

    case 1:
        {            /* only one number */
        /* x is index, assign number to y */
        v[1] = v[0];
        v[0] = df_datum;
        /* nobreak */
        }

    case 2:
        /* x, y */
        /* ylow and yhigh are same as y */

OK.  The idea is simple; if only one variable is read from the file then 
the index into the file should be used as the x value and the read 
variable should be used as y.

But I'd like to point out a few things.  First, consider what is being 
done here, the use of df_readline() starting the loop, i.e.,

     while ((j = df_readline(v, max_cols)) != DF_EOF) {

retrieves from df_readline the variables (v[]) and the number of 
variables read (j) in a modular fashion.  However, plot2d.c retrieves a 
short time later something also generated by df_readline (df_datum) as a 
global variable.  That is not good.  If you were evaluating that 
practise for, say, a software exam, ask yourself what grade you would 
give that.

Alright then, so how to avoid using df_datum like that?  Well, I would 
say that that case 1 situation above really could be easily moved into 
df_readline itself  (** referenced later).  Notice that if at the end of 
df_readline() we knew that this was for 2D data  (in the current case, 
that is a given fact because reading 3D data is its own special routine) 
would could have this exact same code.  For example, at the end of 
df_readline()

  if (df_plot_mode == MODE_PLOT && output == 1) {
        v[1] = v[0];
        v[0] = df_datum;
        output++;
  }

The df_plot_mode variable is something I invented for the image stuff 
that is recorded upon openning the file.  (It isn't the main focus of 
this point.)  You might argue, well, then df_readlin() needs to know 
something about how the data is going to be used.  Yes, but all it needs 
to know is that it is intended for 2D data or 3D data.  In other words, 
we are making available to df_readline some knowledge of what the 
*minimum* allowable number of variables is.  In that scenario then, our 
case statement would conclude that *both* 0 variable returned and 1 
variable returned is invalid, i.e.,

    case 0:        /* not blank line, but df_readline couldn't parse it */
    case 1:
        {
        df_close();
        int_error(current_plot->token,
              "Bad data on line %d", df_line_number);
        }

    case 2:
        /* x, y */
        /* ylow and yhigh are same as y */

In some sense the test for case 0 and case 1 is merely a sanity check. 
 It is when the number of returned variables is 2 or greater that the 
meaning of the data is open for interpretation based upon the plot 
style, etc.

Alright so let me summarize three alternatives with the various concepts:

1.  Allow the use of df_datum as a global variable leaving the code as 
is.  This encourages people to use df_datum at will, and that is 
starting to happen.  It isn't exactly modular and clean practise.

2.  Pass into df_readline() a variable indicating the minimum number of 
columns that is valid.  E.g., df_readline(v[], mincols, maxcols), where 
in plot2d.c mincols would be 2 and in plot3d.c mincols would be 3.  I 
think you understand the point.  This is probably the most appropriate 
"modular" syntax, but I would say that passing in that mincols has 
little advantage if in fact we know it will always be 2 or 3 depending 
upon the plot mode.

3.  What I described above.  It is conceptually the same as 2) but 
instead the information is passed to datafile.c via

df_open(int max_using, int plot_mode)

where in plot2d.c plot_mode would be MODE_PLOT and in plot3d.c plot_mode 
would be MODE_SPLOT.  Like case 2), df_datum no longer needs to be 
globally available.

I would opine that alternative #3 is a good balance.

Now, having said all that, I'd like to point out some code that Ethan 
added to df_readline() somewhere:

            /* FIXME EAM - Trap special case of only a single 'using' 
column. */
            /* But really we need to handle general case of implicit 
column 0 */
            if (output == 1)
                 xpos = (axcol == 0) ? df_datum : v[axcol-1];
            else
                 xpos = v[axcol];

I think that is exactly the concept I've discribed in 3) above.  (**)  
 So you can see how things are starting to get convoluted.


While on this topic of df_readline, I wonder if introducing too much 
"plot dependent" stuff into df_readline is a good idea.  For example, 
with the histogram tics, this kind of line seems like it shouldn't be in 
a file-reading routine:

             add_tic_user(axis,temp_string,xpos,0);

Is there some way to move this functionality outside of df_readline() 
back into plot2d.c?

I pose this question because I've been trying to make the case that 
df_readascii() and df_readbinary(), or whatever, should be transparent 
to the calling routine.  If functionality like above keeps being added 
to df_readascii (df_readline) then soon the situation arises where 
certain types of plots can't be done simply because the data comes from 
a binary data file.

Ethan, what is the minimal amount of information that you would need 
coming back from df_readline() to implement headers from files?  If 
df_readline() were equipped with a char pointer for which df_readline 
could realloc() memory and assign a string, would that do it?  That is, 
I might propose

df_readline(v[], maxcols, string)

where string is a character pointer.  Then

             add_tic_user(axis,temp_string,xpos,0);

could be moved to plot2d.c.  I see absolutely nothing wrong with that 
addition.  I think that is much cleaner than working with so many global 
variables.

In any case, if my comments have motivated anyone to change something, 
please hold off until after the image patch... or if you want me to make 
an attempt at removing df_datum from outside datafile.c as part of the 
image patch, I can do that.

Dan

PS:  I've concluded that moving df_readbinary() to another file would 
require the sharing of too many "local" variables.  So I'm thinking that 
just putting it to the bottom of datafile.c is best... that's actually 
where it probably should be organized anyway.

Re: Option 'ai' in Postscript terminal

[Harald H. <h.h...@tu...>]

On Sat, 14 Aug 2004, Harald Harders wrote:

> > In a quick check of the source for Level 2 extensions listed in the
> > PostScript reference manual I don't see any obvious ones. I am not
> > sure about whether our code assumes Level 2 behaviour with
> > regard to font encodings.
>
> I will have a look to the Postscript Language Reference to find out if we
> really are free of level2 code.

I think the reencoding of fonts also is level 1. According to the Language
Reference, /ISOLatin1Encoding is only predefined in Postscript Level 2.
But the gnuplot postscript files (re)define /ISOLatin1Encoding anyway that
a Postscript Level 2 interpreter is not necessary. As a test I have tried
to view a postscript output file without the definition of
/ISOLatin1Encoding. And it works both with gs and with a Level-2 printer.

Yours
Harald

--=20
Harald Harders                           Langer Kamp 8
Technische Universit=E4t Braunschweig      D-38106 Braunschweig
Institut f=FCr Werkstoffe                  Germany
E-Mail: h.h...@tu...               Tel: +49 (5 31) 3 91-3062
WWW   : http://www.harald-harders.de     Fax: +49 (5 31) 3 91-3058

Re: Option 'ai' in Postscript terminal

[Harald H. <h.h...@tu...>]

> You are right, and I was unduly accepting of the first line of the
> output file, which has been
>    %!PS-Adobe-2.0
> since at least gnuplot version 3.7

This is not the Postscript level but the Version number of the DSC
comment structure. Version 3.0 of the "PostScript Language Document
Structuring Conventions Specification" is dated 25 September 1992, long
before Postscript Level 3 has been started. The Postscript header does not
specify which Postscript level is used.

> In a quick check of the source for Level 2 extensions listed in the
> PostScript reference manual I don't see any obvious ones. I am not
> sure about whether our code assumes Level 2 behaviour with
> regard to font encodings.

I will have a look to the Postscript Language Reference to find out if we
really are free of level2 code.

> However, Daniel's upcoming binary + image patch depends on the
> Level2 operator "filter", so we will have to keep an eye on that.

We should speak to him to ensure that also the filter operator is not used
when using level1.

--=20
Harald Harders                           Langer Kamp 8
Technische Universit=E4t Braunschweig      D-38106 Braunschweig
Institut f=FCr Werkstoffe                  Germany
E-Mail: h.h...@tu...               Tel: +49 (5 31) 3 91-3062
WWW   : http://www.harald-harders.de     Fax: +49 (5 31) 3 91-3058

Re: Option 'ai' in Postscript terminal

[Ethan M. <merritt@u.washington.edu>]

On Friday 13 August 2004 08:55 am, Harald Harders wrote:
> > But the code produced by the driver is *not* Level 1 PostScript,
> > even without the Pattern definitions.   Calling it Level 1 is just not
> > correct.
> 
> Mmmh, which are the Postscript level 2 commands used in Gnuplot?

I apologize for what may have been an unwarranted assumption.
You are right, and I was unduly accepting of the first line of the 
output file, which has been
   %!PS-Adobe-2.0
since at least gnuplot version 3.7

In a quick check of the source for Level 2 extensions listed in the
PostScript reference manual I don't see any obvious ones. I am not
sure about whether our code assumes Level 2 behaviour with 
regard to font encodings.

> Wasn't it 
> a good idea to make a pslevel option which really switches back to
> postscript level 1 syntax? Or does this involve too strong limitations?

It's a good idea.  I thought it would involve actual work, but I 
guess I was too pessimistic.   

However, Daniel's upcoming binary + image patch depends on the
Level2 operator "filter", so we will have to keep an eye on that.

So I am convinced.  I'll change the syntax again, and make the option
"level1".
 

-- 
Ethan A Merritt       merritt@u.washington.edu
Biomolecular Structure Center
Mailstop 357742
University of Washington, Seattle, WA 98195

Re: Option 'ai' in Postscript terminal

[Harald H. <h.h...@tu...>]

On Fri, 13 Aug 2004, Ethan Merritt wrote:

> On Friday 13 August 2004 02:33 am, Harald Harders wrote:
> > - Newer Versions of Adobe Illustrator (at least AI CS 11) do understand
> >   the level 2 code. Thus, using 'ai' is misleading here.
>
> So we are still looking for a good name.  That's OK with me;
> I'm just trying to make everyone happy.
>
> > - There are other programmes that fail using the new code, for example =
gv
> >   (while gs understands it).
>
> ??? gv works for me.  The only issue is the one of anti-aliasing.

I just have noticed this today, too. I have prepared a bug report since
also gstate, setgsate and currentgstate do not work with the x11alpha
device.

> > What about an option 'pslevel' that takes an argument:
> > - 'pslevel 1' restricts to Postscript level 1
> > - 'pslevel 2' allows Postscript level 2 (some day, PS level 3 features =
may
> >   be added)
> > With this method, future problems with level questions will be avoided.
>
> But the code produced by the driver is *not* Level 1 PostScript,
> even without the Pattern definitions.   Calling it Level 1 is just not
> correct.

Mmmh, which are the Postscript level 2 commands used in Gnuplot? Wasn't it
a good idea to make a pslevel option which really switches back to
postscript level 1 syntax? Or does this involve too strong limitations?

> Maybe instead of trying to find a generic label, we must settle for
> being absolutely specific:
> =09set term post {patternlevel2}

If it is not possible to restrict to level 1: What about
  set term post solidfill
versus
  set term post patternfill
, maybe with shortcuts sf and pf?


--=20
Harald Harders                           Langer Kamp 8
Technische Universit=E4t Braunschweig      D-38106 Braunschweig
Institut f=FCr Werkstoffe                  Germany
E-Mail: h.h...@tu...               Tel: +49 (5 31) 3 91-3062
WWW   : http://www.harald-harders.de     Fax: +49 (5 31) 3 91-3058

Re: Option 'ai' in Postscript terminal

[Ethan M. <merritt@u.washington.edu>]

On Friday 13 August 2004 02:33 am, Harald Harders wrote:
> - Newer Versions of Adobe Illustrator (at least AI CS 11) do understand
>   the level 2 code. Thus, using 'ai' is misleading here.

So we are still looking for a good name.  That's OK with me;
I'm just trying to make everyone happy.

> - There are other programmes that fail using the new code, for example gv
>   (while gs understands it).

??? gv works for me.  The only issue is the one of anti-aliasing.

> 
> What about an option 'pslevel' that takes an argument:
> - 'pslevel 1' restricts to Postscript level 1
> - 'pslevel 2' allows Postscript level 2 (some day, PS level 3 features may
>   be added)
> With this method, future problems with level questions will be avoided.

But the code produced by the driver is *not* Level 1 PostScript,
even without the Pattern definitions.   Calling it Level 1 is just not
correct.

Maybe instead of trying to find a generic label, we must settle for
being absolutely specific:
	set term post {patternlevel2}

This describes precisely what the code option does, at the cost of
a long option name. OK, we could allow almost_equals("pat$ternlevel2")









> 
> What do you think?
> 
> Yours
> Harald
> 

-- 
Ethan A Merritt       merritt@u.washington.edu
Biomolecular Structure Center
Mailstop 357742
University of Washington, Seattle, WA 98195

Re: command.c and util.c lost in CVS

[Harald H. <h.h...@tu...>]

>  This might explain it - all works fine for me with ssh access:
>
>  http://sourceforge.net/docman/display_doc.php?docid=3D2352&group_id=3D1
>
>  Project CVS Service:   =09Partial Outage In-Progress  =09    Last update=
d: 2004-08-12 Pacific
>
> (  2004-08-12 12:42:49 - Project CVS Service )   On 2004-08-12 at about
>  10:00 Pacific the pserver based CVS server that hosts projects that have
>  a first letter of c,d,g,a,w,x and u had a hardware malfunction that is
>  currently being worked on. Projects whose letters match those previously
>  mentioned will not be able to have their pserver CVS repositories or Vie=
wCVS
>  interface functional until the issue is resolved. We anticipate this iss=
ue
>  to be resolved by sometime tomorrow.

Thanks. I will wait till it is okay again.

--=20
Harald Harders                           Langer Kamp 8
Technische Universit=E4t Braunschweig      D-38106 Braunschweig
Institut f=FCr Werkstoffe                  Germany
E-Mail: h.h...@tu...               Tel: +49 (5 31) 3 91-3062
WWW   : http://www.harald-harders.de     Fax: +49 (5 31) 3 91-3058

Re: command.c and util.c lost in CVS

[Lars H. <lhe...@us...>]

> $ ls -l gnuplot/src/command.* gnuplot/src/util.*
> -rw-r--r--    1 harders  users        5747 Aug  2 23:57 gnuplot/src/command.h
> -rw-r--r--    1 harders  users        3893 Aug  9 02:51 gnuplot/src/util.h
> $
> 
> I have also tried a total new checkout of everything, but I do not get
> command.c and util.c. Is it my fault or is there a bug in the repository?
 
 This might explain it - all works fine for me with ssh access:

 http://sourceforge.net/docman/display_doc.php?docid=2352&group_id=1

 Project CVS Service:   	Partial Outage In-Progress  	    Last updated: 2004-08-12 Pacific

(  2004-08-12 12:42:49 - Project CVS Service )   On 2004-08-12 at about
 10:00 Pacific the pserver based CVS server that hosts projects that have
 a first letter of c,d,g,a,w,x and u had a hardware malfunction that is
 currently being worked on. Projects whose letters match those previously
 mentioned will not be able to have their pserver CVS repositories or ViewCVS
 interface functional until the issue is resolved. We anticipate this issue
 to be resolved by sometime tomorrow.

Re: command.c and util.c lost in CVS

[Harald H. <h.h...@tu...>]

> > Since my last cvs update, command.c and util.c have gone from the
> > repository. I guess this is not wanted.
>
>  Do you mean s/repository/checked out copy/ ?

Right.

$ cd gnuplot
$ cvs update
=2E..
cvs update: Updating gnuplot/src
cvs update: nothing known about gnuplot/src/command.c
cvs update: nothing known about gnuplot/src/util.c
cvs update: Updating gnuplot/src/NeXT
=2E..
$ ls -l gnuplot/src/command.* gnuplot/src/util.*
-rw-r--r--    1 harders  users        5747 Aug  2 23:57 gnuplot/src/command=
=2Eh
-rw-r--r--    1 harders  users        3893 Aug  9 02:51 gnuplot/src/util.h
$

I have also tried a total new checkout of everything, but I do not get
command.c and util.c. Is it my fault or is there a bug in the repository?

--=20
Harald Harders                           Langer Kamp 8
Technische Universit=E4t Braunschweig      D-38106 Braunschweig
Institut f=FCr Werkstoffe                  Germany
E-Mail: h.h...@tu...               Tel: +49 (5 31) 3 91-3062
WWW   : http://www.harald-harders.de     Fax: +49 (5 31) 3 91-3058

Re: command.c and util.c lost in CVS

[Lars H. <lhe...@us...>]

Harald Harders writes:
> 
> Since my last cvs update, command.c and util.c have gone from the
> repository. I guess this is not wanted.
 
 Do you mean s/repository/checked out copy/ ?

$ cd gnuplot
$ rm src/command.c src/util.c
$ cvs up -Pd
...
cvs update: Updating src
cvs update: warning: src/command.c was lost
U src/command.c
cvs update: warning: src/util.c was lost
U src/util.c
cvs update: Updating src/NeXT
...
$ ll src/command.c src/util.c
-rw-r--r--    1 lhecking users       63611 Aug 13 14:38 src/command.c
-rw-r--r--    1 lhecking users       25079 Aug 13 14:38 src/util.c
$ 

Re: proposal for cvs'ing "with image"

[Daniel J S. <dan...@ie...>]

Hans-Bernhard Br=F6ker wrote:

> Daniel J Sebald wrote:
>
>> Would there be some way to copyright to the group as a whole, i.e.,=20
>> Gnuplot Group, or Gnuplot Project, and then just list the original=20
>> contributor, and successive contributors when mods are made?=20
>
>
> Not really --- for this to be legally viable, I think the Gnuplot=20
> Project would have to exist as a 'legal person', i.e. be a formal=20
> institution, like The Open Group or the Apache Group.  We'ld probably=20
> have to organize a meeting of the current core team members, and Mr.=20
> Williams, and sign some papers.
>
>> There would also be special mention of original copyright by Williams=20
>> & Kelly.  I'm not exactly sure what a copyright means in this case=20
>> either, without some system to enforce or manage it.
>
>
> Copyright means what its name says: the right to make copies, and thus=20
> the right to lay down the rules about making copies (i.e. the license=20
> statement).  For the bulk of the existing code, that right is in the=20
> hands of the original authors, and may remain there essentially forever=
.
> Adding new names to the list of people who hold rights to parts of=20
> gnuplot is not a good idea --- it's become quite unmanageable as it is.


That is what I'm driving at.  There's probably twenty to twenty five=20
different people listed in a Copyright notice.  And then there are some=20
that have a Copyright notice, but no person or entity listed next to the=20
notice.  (How that works, I'm not sure.)  If there is ever a copyright=20
question from an outside source, are they supposed to contact twenty=20
different people?

But you're point is taken.  It makes sense that a chunk of code any=20
particular author writes for the project, should be useable to that=20
person.  I.e., just that chunk of code should be useable for something el=
se.

Dan

command.c and util.c lost in CVS

[Harald H. <h.h...@tu...>]

Since my last cvs update, command.c and util.c have gone from the
repository. I guess this is not wanted.

Yours
Harald

--=20
Harald Harders                           Langer Kamp 8
Technische Universit=E4t Braunschweig      D-38106 Braunschweig
Institut f=FCr Werkstoffe                  Germany
E-Mail: h.h...@tu...               Tel: +49 (5 31) 3 91-3062
WWW   : http://www.harald-harders.de     Fax: +49 (5 31) 3 91-3058

Re: proposal for cvs'ing 'with image'

[Daniel J S. <dan...@ie...>]

Hans-Bernhard Br=F6ker wrote:

> Daniel J Sebald wrote:
>
>> Well, breaking up the routine into modules is alright.  But I don't=20
>> agree that a df_readline() and a df_readbinary() is a good idea. =20
>> That would require a global variable travelling across datafile.c to=20
>> plot2d.c, etc.
>
>
> That cannot possibly be the case, given the fact that your current=20
> code manages without such a global, or any other indication of what's=20
> going on.  So it must obviously be able to figure out whether normal=20
> or binary file reading is being called for.  At the worst, you would=20
> need three routines:
>
> df_readbinary()
> df_readascii()       /* holds the bulk of what is now df_readline() */
> df_readline()       /* very short, just branches into one of the above=20
> two */=20


That is what I'm proposing.

Dan

Option 'ai' in Postscript terminal

[Harald H. <h.h...@tu...>]

In this mailing list some discussion has been about the 'nolevel2' option
in postscript terminal which now has been changed to 'ai' in CVS. I do not
like the new name and prefer the suggestion 'level1' that also has been
made here (or even better the suggestion described below). Some reasons:

- Newer Versions of Adobe Illustrator (at least AI CS 11) do understand
  the level 2 code. Thus, using 'ai' is misleading here.
- There are other programmes that fail using the new code, for example gv
  (while gs understands it).
- The option switches off Level 2 code and switches to Level 1,
  independently where you use the output. Thus naming what it does is
  better.

What about an option 'pslevel' that takes an argument:
- 'pslevel 1' restricts to Postscript level 1
- 'pslevel 2' allows Postscript level 2 (some day, PS level 3 features may
  be added)
With this method, future problems with level questions will be avoided.

What do you think?

Yours
Harald

--=20
Harald Harders                           Langer Kamp 8
Technische Universit=E4t Braunschweig      D-38106 Braunschweig
Institut f=FCr Werkstoffe                  Germany
E-Mail: h.h...@tu...               Tel: +49 (5 31) 3 91-3062
WWW   : http://www.harald-harders.de     Fax: +49 (5 31) 3 91-3058

Re: proposal for cvs'ing 'with image'

[<mi...@ph...>]

>>Copyright (c) Daniel Sebald
>>This code may be used, copied, or modified by anyone for any purpose.
>>
>>Of course if the file includes code taken from other parts
>>of gnuplot then you probably can't do that. Gaah. I hate
>>all this license stuff.  Maybe it has to be
>>
>>Portions of this code
>>Copyright 1986 - 1993, 1998, 2004   Thomas Williams, Colin Kelley and
>>subject to restrictions specified in .
>>Other portions of this code Copyright 2004 Daniel Sebald, and may be
>>used, copied or modified without restriction.
>>
>
> Yes, kind of awkward, this copyright message.  I mean, certainly there
> is so much work by others following the original designers that the new
>  and whole code can't be copyrighted to the original authors, only
> portions thereof.  Would there be some way to copyright to the group as
>  a whole, i.e., Gnuplot Group, or Gnuplot Project, and then just list
> the  original contributor, and successive contributors when mods are
> made?

Considering new files:
You can look at pm3d.h, .c how I have put my copyright there: just my name
and "Copyright: open source as much as possible".Another model is used in gpexecute.h, .c: There is the gnuplot license (no
names), and then list of Authors.
---
PM

Re: proposal for cvs'ing "with image"

[<br...@ph...>]

Daniel J Sebald wrote:

> Would there be some way to copyright to the group as 
> a whole, i.e., Gnuplot Group, or Gnuplot Project, and then just list the 
> original contributor, and successive contributors when mods are made? 

Not really --- for this to be legally viable, I think the Gnuplot 
Project would have to exist as a 'legal person', i.e. be a formal 
institution, like The Open Group or the Apache Group.  We'ld probably 
have to organize a meeting of the current core team members, and Mr. 
Williams, and sign some papers.

> There would also be special mention of original copyright by Williams & 
> Kelly.  I'm not exactly sure what a copyright means in this case either, 
> without some system to enforce or manage it.

Copyright means what its name says: the right to make copies, and thus 
the right to lay down the rules about making copies (i.e. the license 
statement).  For the bulk of the existing code, that right is in the 
hands of the original authors, and may remain there essentially forever.
Adding new names to the list of people who hold rights to parts of 
gnuplot is not a good idea --- it's become quite unmanageable as it is.

Re: proposal for cvs'ing 'with image'

[<br...@ph...>]

Daniel J Sebald wrote:

> Well, breaking up the routine into modules is alright.  But I don't 
> agree that a df_readline() and a df_readbinary() is a good idea.  That 
> would require a global variable travelling across datafile.c to 
> plot2d.c, etc.

That cannot possibly be the case, given the fact that your current code 
manages without such a global, or any other indication of what's going 
on.  So it must obviously be able to figure out whether normal or binary 
file reading is being called for.  At the worst, you would need three 
routines:

df_readbinary()
df_readascii()       /* holds the bulk of what is now df_readline() */
df_readline()       /* very short, just branches into one of the above 
two */

Re: proposal for cvs'ing "with image"

[Daniel J S. <dan...@ie...>]

Ethan A Merritt wrote:

>On Thursday 12 August 2004 06:11 pm, you wrote:
>  
>
>>What should the notice be?  In fact, what should the notice be for this
>>new "binary datafile.c" file?
>>    
>>
>
>Your code, your call.
>I'd put something like
>
>Copyright (c) Daniel Sebald
>This code may be used, copied, or modified by anyone for any purpose.
>
>Of course if the file includes code taken from other parts
>of gnuplot then you probably can't do that. Gaah. I hate
>all this license stuff.  Maybe it has to be
>
>Portions of this code 
>Copyright 1986 - 1993, 1998, 2004   Thomas Williams, Colin Kelley
>and subject to restrictions specified in .
>Other portions of this code Copyright 2004 Daniel Sebald, and may
>be used, copied or modified without restriction.
>

Yes, kind of awkward, this copyright message.  I mean, certainly there 
is so much work by others following the original designers that the new 
and whole code can't be copyrighted to the original authors, only 
portions thereof.  Would there be some way to copyright to the group as 
a whole, i.e., Gnuplot Group, or Gnuplot Project, and then just list the 
original contributor, and successive contributors when mods are made? 
 There would also be special mention of original copyright by Williams & 
Kelly.  I'm not exactly sure what a copyright means in this case either, 
without some system to enforce or manage it.

Dan




-- 

Dan Sebald
email: daniel DOT sebald AT ieee DOT org
URL: http://acer-access DOT com/~dsebald AT acer-access DOT com/