gnuplot-beta — Developers and beta testers of new gnuplot code

Re: [Fwd: Re: 'with pixels' (was Re: Engines are started; ...)]

[Hans-Bernhard B. <br...@ph...>]

On Mon, 12 Jul 2004, Daniel J Sebald wrote:
> Hans-Bernhard Broeker wrote:

> >I agree wholeheartedly that that code needs to be revamped in a major way.  
> >But that has no relevance at all to the job to be done by datafile.c. As
> >long as the 'using' syntax is based on an unstructured list of expressions
> >rather than named columns like this
> >
> >	plot 'data' using (x 1, y 2, yhigh 3, ylow 4) 
[...]

> The above isn't such a bad idea in one sense; that is, it is more 
> meaningful to the user *provided* the syntax makes sense.  But that 
> making sense part is the rub.  

Please don't get all excited by that rough syntax sketch.  It was really
just meant to point at the user interface syntax being the driving force
of how the datafile parsing code works, and especially why get_data() and
get_3ddata() work as they do.  I'm by no means proposing we implement
anything like that example anytime soon.  But as long as 'using' in
its current form exists, I don't see any way of actually getting rid
of the fiddling currently done by get_{3d,}data().  We can only hope
to structure it a bit better.

> Certainly, one can't expect the user to know there are these structure
> variables behind the curtain called "x", "y", "yhigh", "ylow", etc. and
> that it is their responsibility to know what goes where.

No.  The above syntax doesn't require him to know "what goes where" ---
it's the existing code that does that.  The above would just require the
user to know "what named values are there".  I.e. they could just as well
have written

	plot 'data' u (yhigh 3, y 2, ylow 4, x 1) w err

to express the exact same relation between data file and plot.  As usual,
additional flexibility like that comes at a price: the command line gets
longer.

>  If I'm not mistaken, there are plot types that use 
> "yhigh", "ylow", etc. as just some variable which doesn't necessarily 
> make sense to the context at hand. 

Yes.  I'm quite painfully aware of that, for all the grief it caused me
during the work on the 'axis branch'.

> So, right there you have moved the mess, pretty much in tact, from 
> get_data() to somewhere else.

Not really.  The mess we're talking about here (the mapping from data
values accessed a seris of 'using' specifiers to elements of 'struct
coordinate' (x,y,z,ylow,...) is currently a two-part one: part one is that
the number of specifiers changes the assignment from a position in the
using spec sequence to struct elements (e.g. errorbars: 3 columns mean
x:y:dy, 4 columns mean x:y:ylow:yhigh, i.e. the meaning of the third
specifier depends on whether there's a forth one or not).  The *other*
aspect of the mess is that users have to memorize the order of using spec
for all plot styles.  And as we keep adding plot styles and optional
values (acsplines, color mapping and whatnot), that's only getting worse.

[...]
> configuration inside a table.  That is, there is the enumeration for the 
> PLOT_STYLE and one could build tables off of that.  

Or rather, I guess what we should consider is a re-organization of the
entire way plot styles are handled.  The current 'enum with flags in'
approach of defining plot style indices as magic numbers is of rather
limited versatility, and more of a hack than a proper design.  

Currently we have 9 switch() statements over the plot type, and about 90
other comparisons of the plot_style field with particular types, scattered
all across the 4 inner core modules (plot[23]d, graphics, graph3d) and
several others.  I think this would all make a lot more sense if we
collected at least all code governed by those switch()es, but possibly
more, in one contiguous block of code *per plot style*.  I.e. all special
code for handling the 'with yerrorbars' style should be in *one* place.


-- 
Hans-Bernhard Broeker (br...@ph...)
Even if all the snow were burnt, ashes would remain.

Re: Strange behaviour of arrow rto in logarithmic scale (fwd)

[Hans-Bernhard B. <br...@ph...>]

Re-sent of a message that collided with SF.net's mail server problem:

On Sun, 11 Jul 2004, Harald Harders wrote:

> ranges. When using the same coordinates for the start point and the
> relative end point the end point is calculated by summating:
> endpos = startpos + difference. If I, instead, change the coordinate
> system between start point and relative end point, the `difference' is
> taken as factor: endpos = startpos * difference.

I looked into the code, and found that somebody actually code this special
behaviour explicitly (see graphics.c:get_arrow()).  I don't think that's a
good idea.  But it's not exactly a surprise, either.  The subtle
difference between relative and absolute numbers is quite definitely not
handled consistently in current gnuplot.

-- 
Hans-Bernhard Broeker (br...@ph...)
Even if all the snow were burnt, ashes would remain.

Re: 'with pixels' (was Re: Engines are started; ...)

[Hans-Bernhard B. <br...@ph...>]

Re-send of message that collided with a problem at SF.net on Sunday:

On Sat, 10 Jul 2004, Daniel J Sebald wrote:
> Hans-Bernhard Broeker wrote:
> 
> >So, let's see: the general plan (before datastrings) is
> >
> >*) datafiles have colums
> >*) 'using' processes columns into records of at most 7 numbers
> >*) get_data() maps numbers from that record to data points
> >
> >Of these three actions, only the first two take place in datafile.c At
> >some point in the process, datafile handling *will* become dependent on
> >the plot style, but datafile.c is not where that point should be.

>  From what I remember, the issue in the image case had to do with how to 
> route the columns in the instance where the user doesn't give the 
> "using" details.  

The option of omitting the using specification is legacy behaviour of
gnuplot.  How about this proposal, then: require binary file always to
have using specifiers, and be done with that complication.

> The question for me is should the user be required 
> *always* to enter specifically how columns are to be used via "using"? 

I have no problems with such a requirement, as far as newly invented
features are concerned --- we already require it for time/date parsing,
e.g.

Re: 'with pixels' (was Re: Engines are started; ...)

[Daniel J S. <dan...@ie...>]

Hans-Bernhard Broeker wrote:

>So, let's see: the general plan (before datastrings) is
>
>*) datafiles have colums
>*) 'using' processes columns into records of at most 7 numbers
>*) get_data() maps numbers from that record to data points
>
>Of these three actions, only the first two take place in datafile.c At
>some point in the process, datafile handling *will* become dependent on
>the plot style, but datafile.c is not where that point should be.
>
>I think we may have to turn all that on its head: let the caller tell the
>df_... routines in advance how many using specs it needs, and what values
>(axis assignment, time/date flag, datastring flag) each of those should
>represent.
>

 From what I remember, the issue in the image case had to do with how to 
route the columns in the instance where the user doesn't give the 
"using" details.  The question for me is should the user be required 
*always* to enter specifically how columns are to be used via "using"? 
 I mean, there are some logical default methods of treating the data if 
the user doesn't give the "using".  However, that kind of "logical 
default" is plot_type dependent.

Dan

Strange behaviour of arrow rto in logarithmic scale

[Harald H. <h.h...@tu...>]

When thinking of relative coordinates, I have found a strange behaviour of
`set arrow' when using `rto' in conjunction with logarithmic coordinates.

Have a look at the output of the following example:

# --------------------------------------------------------------
set logscale x
set logscale x2
set xrange [1:100]
set yrange [-1:1]
set x2range [1:100]
set y2range [-1:1]

set arrow from first 10,-0.3 rto first 5,0
set arrow from second 10,-0.1 rto second 5,0
set arrow from first 10,0.1 rto second 5,0
set arrow from second 10,0.3 rto first 5,0

set label ' first rto first ' at 10,-0.3 right
set label ' second rto second ' at 10,-0.1 right
set label ' first rto second ' at 10,0.1 right
set label ' second rto first ' at 10,0.3 right

plot 1/0 notitle

pause -1
# --------------------------------------------------------------

I am using two coordinate systems `first' and `second' with the same
ranges. When using the same coordinates for the start point and the
relative end point the end point is calculated by summating:
endpos =3D startpos + difference. If I, instead, change the coordinate
system between start point and relative end point, the `difference' is
taken as factor: endpos =3D startpos * difference.
Thus, for the cases `first to first' and `second to second', the arrows
end at 10+5=3D15 while for the cases `first to second' and `second to
first', they end at 10*5=3D50.

I think, this really should be consistent. And in order to allow also the
other changes, e.g., `screen to first', I think the usage as a factor is
better than the difference. Interpreting the offset as factor whould also
enable text offsets more easily.

What do you think?

Yours
Harald

P.S. I have not posted this as bug to sourceforge since I think here is
some discussion necessary.

--=20
Harald Harders                           Langer Kamp 8
Technische Universit=E4t Braunschweig      D-38106 Braunschweig
Institut f=FCr Werkstoffe                  Germany
E-Mail: h.h...@tu...               Tel: +49 (5 31) 3 91-3062
WWW   : http://www.ifw.tu-bs.de          Fax: +49 (5 31) 3 91-3058

Re: 'with pixels' (was Re: Engines are started; ...)

[Ethan A M. <merritt@u.washington.edu>]

On Saturday 10 July 2004 05:20 pm, you wrote:

This is going to be a long explanation, so please bear with me.

> So, let's see: the general plan (before datastrings) is
>
> *) datafiles have colums
> *) 'using' processes columns into records of at most 7 numbers
> *) get_data() maps numbers from that record to data points

Right.  df_readline() fills in a vector v[] whose elements are in
order of the requested data items.
The datastrings patchset added a parallel vector df_tokens[]
whose elements point instead to a string representing the
particular requested data item.  get_data() can use these
to fill in the 'data points' in the '[s]plot ... with labels' style.

So far, so good.  Then it gets a bit messier because the new code
also handles entities such as an in-line plot title or tic labels, 
and the lines below are part of that.

> >   /* FIXME EAM - Trap special case of only a single 'using' column. */
> >   /* But really we need to handle general case of implicit column 0 */
> >         if (output == 1)
> >             xpos = (axcol == 0) ? df_datum : v[axcol-1];
> >         else
> >             xpos = v[axcol];
>
> Hmmm... then how did that work before EAM_DATASTRINGS?  I can't seem to
> find any similar snippet in 4.0 df_readline().

For completeness sake, let me explain what this code is doing:

At this point in the code we know that the string value we are looking
at is supposed to be an axis tic label.  We also know which axis it is
a tic label for. We want to create a pair ["ticlabel", axis-value]  just
as in  a 'set [xyz]tics { "label" pos, ...}' command.  But which input 
column contains the value for that axis?  That is, if we are reading
a ytic label, how do we know which column contains y?  

Problem #1: Most of the plot styles are consistent in that the first
column is x, the second column is y, and the third column is z.
But the 3D plot modes can violate this order and in particular, as I
mentioned before, it is problematic which column will be used to
generate a color value. Given that there is not currently any way
to tell what the plot style is (or even if it is 3D) all I can do is 
hope that the x:y:z order holds. But currently   
'plot ... using ...cbticlabels(<n>)' doesn't work, because there is
no way to know which column the color info is in without
access to more information about the plot.

Problem #2:  The mapping of columns 1:2:3 to x:y:z coordinate value
is broken by the use of implicit x coordinates.  Consider the command
'plot ... using 2:yticlabel(1)'.   In this case the  first (and only)
column contains a y coordinate rather than an x coordinate.
Since no currently supported plot styles really use only a single
input coordinate, it is easy to detect this simple case, and that is
exactly what the code section extracted above is doing.

The code is marked FIXME because it cannot detect, let alone deal
with, any more complicated cases where the order of axes cannot be
deduced from the number of columns.   In practice this is not much
of a limitation, though eventually someone is bound to trip over
it and report it as a bug.  If it becomes important to handle the
oddball cases as well, then I think it will be necessary to have
access to the current plot and style information.  That is why I
used it as an example of wanting a pointer to the current plot.

-- 
Ethan A Merritt
Department of Biochemistry & Biomolecular Structure Center
University of Washington, Seattle

Re: 'with pixels' (was Re: Engines are started; ...)

[Daniel J S. <dan...@ie...>]

Hans-Bernhard Broeker wrote:

>On Sat, 10 Jul 2004, Daniel J Sebald wrote:
>
>  
>
>>Ethan A Merritt wrote:
>>
>>    
>>
>>>For what it's worth -
>>>After I apply the with-image patch df_readline contains 653 lines of
>>>code, of which only 303 are shared by the two modes.  
>>>
>>>The largest block of shared code is the section (lines 3362-3465)
>>>checking for blank lines, skipped lines, and EOF.  Is this even relevant
>>>to binary mode? 
>>>
>>>      
>>>
>>That's correct.  But my point is that they do pretty much similar 
>>things, i.e., loading the variables to prepare the next stage of the 
>>code that determines where to reroute them.
>>    
>>
>
>Well, achieving the same goal is not the same thing as doing the same job,
>and thus doesn't call for having the two routines combined. I tend to
>agree with Ethan here: if they share so little actual code, they had
>better be in separate routines, and the common code broken out into
>a common subroutine.
>

Right.  They are not intermixed in a convoluted fashion.  If I recall, 
there is an if statement "if (df_general_binary)" that pretty much 
breaks things into two chunks of code that easily could be put into 
separate routines.  The point is I didn't want to upset the apple cart 
too much, by creating all kinds of new subroutines and the like...  I 
still make the point that higher level plot{23}d.c routines should hot 
have to know whether the file is binary or ascii, so there should still 
just be "df_readlin()" from that perspective.  A much cleaner and 
organized "df_readline()" is a different matter.

Dan

Re: 'with pixels' (was Re: Engines are started; ...)

[Hans-Bernhard B. <br...@ph...>]

On Sat, 10 Jul 2004, Daniel J Sebald wrote:

> 
> 
> Ethan A Merritt wrote:
> 
> >For what it's worth -
> >After I apply the with-image patch df_readline contains 653 lines of
> >code, of which only 303 are shared by the two modes.  
> >
> >The largest block of shared code is the section (lines 3362-3465)
> >checking for blank lines, skipped lines, and EOF.  Is this even relevant
> >to binary mode? 
> >
> 
> That's correct.  But my point is that they do pretty much similar 
> things, i.e., loading the variables to prepare the next stage of the 
> code that determines where to reroute them.

Well, achieving the same goal is not the same thing as doing the same job,
and thus doesn't call for having the two routines combined. I tend to
agree with Ethan here: if they share so little actual code, they had
better be in separate routines, and the common code broken out into
a common subroutine.

-- 
Hans-Bernhard Broeker (br...@ph...)
Even if all the snow were burnt, ashes would remain.

Re: 'with pixels' (was Re: Engines are started; ...)

[Hans-Bernhard B. <br...@ph...>]

On Sat, 10 Jul 2004, Ethan A Merritt wrote:

> Hans-Bernhard Broeker <br...@ph...> wrote:
> > Ethan A Merritt wrote:
> > > The code in datafile.c and elsewhere (df_readline is not so much
> > > the culprit here) makes an assumption that it can deduce the style of
> > > plot based on the number of input columns requested.
> >
> > Could you show a concrete example of that?
> 
> The worst offenders are one level up, in plot[23]d.c in the routines
> get_data and get_3ddata.  Each of these contains a huge switch
> statement that controls the interpretation of input data columns
> based on the total number of returned columns.  Within each
> 'case <ncols>:' of the switch statement, the code then tries to
> run through all possible plot modes that could have generated
> that number of input columns. I think this is an example of the tail
> wagging the dog;  the main switch statement should rather be
> over the plot types.

I agree wholeheartedly that that code needs to be revamped in a major way.  
But that has no relevance at all to the job to be done by datafile.c. As
long as the 'using' syntax is based on an unstructured list of expressions
rather than named columns like this

	plot 'data' using (x 1, y 2, yhigh 3, ylow 4) 

, the mess in get_{3d,}data() is going to stay as it is.  It's not pretty,
admitted, but it has nothing to do with the point at hand.  Actually,
the fact that this stuff is *not* done in datafile.c is what I'm talking
about.  It's not in there, and it shouldn't be.

So, let's see: the general plan (before datastrings) is

*) datafiles have colums
*) 'using' processes columns into records of at most 7 numbers
*) get_data() maps numbers from that record to data points

Of these three actions, only the first two take place in datafile.c At
some point in the process, datafile handling *will* become dependent on
the plot style, but datafile.c is not where that point should be.

I think we may have to turn all that on its head: let the caller tell the
df_... routines in advance how many using specs it needs, and what values
(axis assignment, time/date flag, datastring flag) each of those should
represent.

> The problematic code in df_readline() itself is the bit which decides
> where to stuff the input values based on the number of requested input
> columns. It allows for an implicit x coordinate value, but only in the
> case of 'plot ... using <y> with <something that only takes 2 cols>' 
> 
>   /* FIXME EAM - Trap special case of only a single 'using' column. */
>   /* But really we need to handle general case of implicit column 0 */
>         if (output == 1)
>             xpos = (axcol == 0) ? df_datum : v[axcol-1];
>         else
>             xpos = v[axcol];

Hmmm... then how did that work before EAM_DATASTRINGS?  I can't seem to 
find any similar snippet in 4.0 df_readline(). 

-- 
Hans-Bernhard Broeker (br...@ph...)
Even if all the snow were burnt, ashes would remain.

Re: 'with pixels' (was Re: Engines are started; ...)

[Daniel J S. <dan...@ie...>]

Ethan A Merritt wrote:

>Hans-Bernhard Broeker <br...@ph...> wrote:
>  
>
>The problematic code in df_readline() itself is the bit which decides
>where to stuff the input values based on the number of requested input
>columns.
>

[...]

This is correct.  I thought long and hard about the implications of 
this, trying to get something that made sense.  You know, "if this 
column is present but not this, then one should be routed here and the 
other there, etc."  But then you reach a contradiction that comes to 
light when it messes up some other demo.  Of course, in binary files, 
there is no end of line to determine how many columns are present.  It 
has to be specified by the user.  But still, it's the same idea that 
Ethan is expressing.

Dan

Re: 'with pixels' (was Re: Engines are started; ...)

[Daniel J S. <dan...@ie...>]

Ethan A Merritt wrote:

>For what it's worth -
>After I apply the with-image patch df_readline contains 653 lines of
>code, of which only 303 are shared by the two modes.  
>
>The largest block of shared code is the section (lines 3362-3465)
>checking for blank lines, skipped lines, and EOF.  Is this even relevant
>to binary mode? 
>

That's correct.  But my point is that they do pretty much similar 
things, i.e., loading the variables to prepare the next stage of the 
code that determines where to reroute them.

>>And at the bottom
>>of df_readline() is the important code which knows how and where to put
>>data in the v[] variables
>>    
>>
>
>Yes, that is clearly important to both modes.  But perhaps it should be
>split out into a common subroutine shared by two different input routines.
>
>At that point you might even find that 2 input stages + 1 shared output
>stage have fewer lines of code jointly than the current single routine with 
>its tangle of #ifdefs.
>

I've no problem with a common subroutine, as that even makes it more 
obvious the common idea between the two.  Recall, my premise with the 
#ifdefs was to have the code *exactly unaltered* when BINARY_DATA_FILE 
is disabled.

Dan

Re: 'with pixels' (was Re: Engines are started; ...)

[Ethan A M. <merritt@u.washington.edu>]

Hans-Bernhard Broeker <br...@ph...> wrote:
> Ethan A Merritt wrote:
> > The code in datafile.c and elsewhere (df_readline is not so much
> > the culprit here) makes an assumption that it can deduce the style of
> > plot based on the number of input columns requested.
>
> Could you show a concrete example of that?

The worst offenders are one level up, in plot[23]d.c in the routines
get_data and get_3ddata.  Each of these contains a huge switch
statement that controls the interpretation of input data columns
based on the total number of returned columns.  Within each
'case <ncols>:' of the switch statement, the code then tries to
run through all possible plot modes that could have generated
that number of input columns. I think this is an example of the tail
wagging the dog;  the main switch statement should rather be
over the plot types.

The problematic code in df_readline() itself is the bit which decides
where to stuff the input values based on the number of requested input
columns. It allows for an implicit x coordinate value, but only in the
case of 'plot ... using <y> with <something that only takes 2 cols>' 

  /* FIXME EAM - Trap special case of only a single 'using' column. */
  /* But really we need to handle general case of implicit column 0 */
        if (output == 1)
            xpos = (axcol == 0) ? df_datum : v[axcol-1];
        else
            xpos = v[axcol];


I even have a real-life example of why this sort of assumption is limiting 
our options.   I would have liked to allow the new 'splot with labels' style
to accept the same sort of pm3d coloring options that other plotting
styles offer.  But I gave up trying to work around the assumptions in the
current code that if you have a color value at all, then it must be in the
4th input column.  'splot with labels' needs the 4th specified input
column to be the label itself; if there were an additional color value it
would be in an optional 5th input column.  Yeah, I could hack this to
work somehow, but it would be a whole lot easier if the code were
organized based on plot style rather than on number of columns. 


-- 
Ethan A Merritt
Department of Biochemistry & Biomolecular Structure Center
University of Washington, Seattle

Re: 'with pixels'

[Ethan A M. <merritt@u.washington.edu>]

On Saturday 10 July 2004 08:20 am, Dave Denholm wrote:
>
> There is (or used to be) an undocumented command  show at
> to show the internal representation of an expression.

Thanks for that pointer!   
I had not noticed that feature at all.
It may prove to be invaluable in debugging of string variable
code.

-- 
Ethan A Merritt
Department of Biochemistry & Biomolecular Structure Center
University of Washington, Seattle

Re: 'with pixels' (was Re: Engines are started; ...)

[Ethan A M. <merritt@u.washington.edu>]

On Saturday 10 July 2004 01:21 am, Daniel J Sebald wrote:
> Ethan A Merritt wrote:
> >
> >Daniel - don't you think it would be cleaner to provide an entirely
> >separate routine, df_readbinary(), and call it instead of df_readline()
> >when needed?
>
> That is correct, and I sort of started out that way.  However, the more
> I thought about it, it just seemed that the formatted and binary input
> really weren't too much different.  One can still imagine data inside a
> binary file as being in columns, with implicit columns filled in by the
> line number, etc.; just like with the formatted I/O.  

For what it's worth -
After I apply the with-image patch df_readline contains 653 lines of
code, of which only 303 are shared by the two modes.  

The largest block of shared code is the section (lines 3362-3465)
checking for blank lines, skipped lines, and EOF.  Is this even relevant
to binary mode? 

> And at the bottom
> of df_readline() is the important code which knows how and where to put
> data in the v[] variables

Yes, that is clearly important to both modes.  But perhaps it should be
split out into a common subroutine shared by two different input routines.

At that point you might even find that 2 input stages + 1 shared output
stage have fewer lines of code jointly than the current single routine with 
its tangle of #ifdefs.

-- 
Ethan A Merritt
Department of Biochemistry & Biomolecular Structure Center
University of Washington, Seattle

Re: 'with pixels'

[Dave D. <dde...@es...>]

Daniel J Sebald <dan...@ie...> writes:



> the bottom of df_readline() is the important code which knows how and
> where to put data in the v[] variables, and there may even be some
> scripts there, as I don't really understand the "at" stuff and that.
>

The "at" stuff is just the way expressions are stored. All (?) expression
evaluation in gnuplot is done by compiling the expression into an
action table, then stepping through the actions to compute a result.

There is (or used to be) an undocumented command  show at
to show the internal representation of an expression.

gnuplot> f(x)=x+2
gnuplot> show at f(sin(x)-cos(y+z))

        push x
        sin
        push y
        push z
        plus
        cos
        minus
        call f
          pushd1 f dummy
          pushc 2
          plus


action tables are clearly stack-based, and user-defined functions are
(not surprisingly) stored as action tables.

For plot ... using (expression), each expression gets
compiled into an action table, and then the action table is evaluated
for each datafile line to calculate the required values.

In the version I'm looking at, parse.c does the compiling of
expressions, and eval.c does the evaluation (by just running through
the function pointers stored in the at)

Looks like a global variable "undefined" gets set if something goes
wrong during the evaluation.


dd
-- 
Dave Denholm              <dde...@es...>       http://www.esmertec.com

Re: 'with pixels' (was Re: Engines are started; ...)

[Hans-Bernhard B. <br...@ph...>]

Ethan A Merritt wrote:

> The code in datafile.c and elsewhere (df_readline is not so much
> the culprit here) makes an assumption that it can deduce the style of
> plot based on the number of input columns requested.  

Could you show a concrete example of that?

> I maintain that if a piece of code needs to know the
> current plot style it should look in the proper place, the plot structure,
> rather than trying to guess it from incomplete information.

*If* it needs them.  What I'm debating here is whether this need is 
real, rather than just a consequence of a design weakness.

> The second reason arose when I added the "datastrings" code
> that allows reading meta-information related to the plot from the datafile,
> including axis tic labels, plot key titles, and so on.   If you want to 
> argue that these are a bad idea, or propose a different way to 
> accomplish this, please feel free.  

That indeed is a valid reason.  I'm just still not convinced that we
should be passing in the entire plot struct for that, as opposed to
just those parts of it that the routines are actually going to 
manipulate.  We should try to keep those inter-linkages between
different parts of the program as narrow and focused as possible. 
Passing in the entire plot struct wholesale is the opposite of that.

-- 
Hans-Bernhard Broeker (br...@ph...)
Even if all the snow were burnt, ashes would remain.

Re: General bug - No out of bounds checking on 3D coordinates

[Hans-Bernhard B. <br...@ph...>]

Ethan A Merritt wrote:

> Program received signal SIGSEGV, Segmentation fault.
> 0x00000000 in ?? ()
> (gdb) where
> #0  0x00000000 in ?? ()
> #1  0x080ab1d8 in do_arrow (sx=0, sy=0, ex=135447060, ey=135447060, head=0) at 
> term.c:938
> #2  0x0806917b in place_arrows (layer=0) at graphics.c:1114

Well, I *did* mention "set arrow to far away points causes garbage" bug 
report, didn't I?

-- 
Hans-Bernhard Broeker (br...@ph...)
Even if all the snow were burnt, ashes would remain.

Re: 'with pixels' (was Re: Engines are started; ...)

[Daniel J S. <dan...@ie...>]

Ethan A Merritt wrote:

>I have not looked much at Daniel's "with image" code, except for
>the terminal API.  I am a bit surprised that it needs to call df_readline()
>at all, since I would have expected that reading a binary file is a
>distinct operation from reading successive lines of formatted input.  
>
>Daniel - don't you think it would be cleaner to provide an entirely
>separate routine, df_readbinary(), and call it instead of df_readline()
>when needed?  You can never mix the two modes of input (is that right?), 
>so it seems a bad idea to mix the code that handles them.
>

That is correct, and I sort of started out that way.  However, the more 
I thought about it, it just seemed that the formatted and binary input 
really weren't too much different.  One can still imagine data inside a 
binary file as being in columns, with implicit columns filled in by the 
line number, etc.; just like with the formatted I/O.  And at the bottom 
of df_readline() is the important code which knows how and where to put 
data in the v[] variables, and there may even be some scripts there, as 
I don't really understand the "at" stuff and that.

I thought about splitting it off into binary data input, say some 
derivative of "df_3dmatrix()" or whatever it is called.  However, then 
the flexibility like in df_readline() wouldn't be there, unless I copied 
parts of df_readline().  Also, if there were a hypothetical 
"df_readbinary()" then at the higher levels like plot2d.c and plot3d.c 
one would have to know if it is reading a binary data file or a formated 
ascii file.  You then get these kind of limiting conditional statements 
in strange places, and a somewhat big df_3dmatrix() routine.

That is why I proposed earlier that it may be possible to modify what 
I've done inside df_readline() so that it reads gnuplot binary.  I don't 
think it would take too much.  The trickiest part is that along the 
x-dimension can be a *non-uniform* sampling as described by the very 
first line of the gnuplot binary file.  So, that needs to be stored in 
memory.  I've got some ideas, but it probably isn't worth discussing 
now... or risking breaking the current df_3dmatrix() way of doing things.

Looking at df_3dmatrix() in datafile.c and all that supporting code in 
binary.c, I couldn't see where to put in new binary code.  Judging from 
the comments with those routines, it looks like someone had in mind to 
create arbitrary binary files but maybe gave up.  It seems the paradigm 
of storing coordinates in the point->x, y, z, dx, dy, etc. format will 
not change in the near future, if at all.  The free_matrix(), 
extend_matrix() etc. seems like a powerful set of routines, but outside 
the paradigm of point->x, y, z, dx, dy.

Dan

Re: 'with pixels' (was Re: Engines are started; ...)

[Ethan A M. <merritt@u.washington.edu>]

On Friday 09 July 2004 06:07 pm,  Hans-Bernhard Broeker 
<br...@ph... wrote:
> I strictly disagree with that reasoning.  If df_readline needs to know
> what the the plot style is, then IMHO that's a sign that there's something
> seriously wrong with df_readline().

Let me present two arguments for passing a pointer to the current plot.

The code in datafile.c and elsewhere (df_readline is not so much
the culprit here) makes an assumption that it can deduce the style of
plot based on the number of input columns requested.  Maybe that
used to be true back when gnuplot supported fewer plot styles, but it
no longer holds.  I maintain that if a piece of code needs to know the
current plot style it should look in the proper place, the plot structure,
rather than trying to guess it from incomplete information.

The second reason arose when I added the "datastrings" code
that allows reading meta-information related to the plot from the datafile,
including axis tic labels, plot key titles, and so on.   If you want to 
argue that these are a bad idea, or propose a different way to 
accomplish this, please feel free.  But if you accept that df_readline
is now reading information that has to be stored in the plot structure
eventually, then it seems cleaner to me that it receive a pointer to the
structure so that it can be stored there directly upon input.

I have not looked much at Daniel's "with image" code, except for
the terminal API.  I am a bit surprised that it needs to call df_readline()
at all, since I would have expected that reading a binary file is a
distinct operation from reading successive lines of formatted input.  

Daniel - don't you think it would be cleaner to provide an entirely
separate routine, df_readbinary(), and call it instead of df_readline()
when needed?  You can never mix the two modes of input (is that right?), 
so it seems a bad idea to mix the code that handles them.


-- 
Ethan A Merritt
eme...@es...

Re: General bug - No out of bounds checking on 3D coordinates

[Ethan A M. <merritt@u.washington.edu>]

On Sunday 04 July 2004 11:46 am, Ethan A Merritt wrote:
>
> The basic problem is that the 3D mapping routine map3d_xy() in util3d.c
> stuffs its output into (unsigned int) terminal coordinates.
> But it does so without ever checking that the results are positive numbers.
>
> This may also happen with 2D plots, but if so I haven't encountered it yet.

But now I have.
The new emf driver segfaults in arrowstyle.dem for the same reason:

Program received signal SIGSEGV, Segmentation fault.
0x00000000 in ?? ()
(gdb) where
#0  0x00000000 in ?? ()
#1  0x080ab1d8 in do_arrow (sx=0, sy=0, ex=135447060, ey=135447060, head=0) at 
term.c:938
#2  0x0806917b in place_arrows (layer=0) at graphics.c:1114


-- 
Ethan A Merritt
Department of Biochemistry & Biomolecular Structure Center
University of Washington, Seattle

Re: regression: 'with pixels' affects 'all.dem'

[Daniel J S. <dan...@ie...>]

Hans-Bernhard Broeker wrote:

>As a matter of fact, map3d_xy() is a bit of a mess already as it is.
>  
>
>It does two unrelated jobs rolled into one function call.  This
>will need some cleanup eventually.  But maybe not right now...
>

I know.

>>or as I've argued in the discussion initiated by Ethan, just plain ints.
>>    
>>
>
>No.  Using 'int' for this is a placebo.  It may appear to cure the worst
>of the symptoms, but that's more by coincidence and belief in the cure
>than by design.
>

Faith shall heal ye.  :)

Dan

Re: 'with pixels' (was Re: Engines are started; ...)

[Daniel J S. <dan...@ie...>]

Hans-Bernhard Broeker wrote:

>On Fri, 9 Jul 2004, Ethan Merritt wrote:
>
>  
>
>>#ifdef EAM_HISTOGRAMS
>>    /* EAM FIXME - There are places in df_readline where it would be really */
>>    /* nice to know what kind of plot we are making, so I think that        */
>>    /* current_plot should be a parameter to df_readline. 
>>    
>>
>
>I strictly disagree with that reasoning.  If df_readline needs to know
>what the the plot style is, then IMHO that's a sign that there's something 
>seriously wrong with df_readline().
>
>It's separations like this being broken down or "tunnelled", tieing every
>piece of the program to all others, that got gnuplot into the mess it's
>still in.
>
>Bluntly put, adding a global variable is essentially never the right
>solution for any problem other than stack size limitations.  
>
>Instead of designing a correct structure of "who is responsible for what",
>creating a global means that we've given up. In the case at hand,
>df_readline is the core interface between the datafile itself and the next
>stage in data processing: the 'using' specifiers. It should never need to
>know anything about plot styles, which are at least two steps removed from
>its work.
>

Well, global variables is certainly not good.  The issue is that 
df_readline() has a number of "local" variables that must be configured 
by "df_open", e.g., what quantities to put into what variable.  The 
question is, should the user have to specify every detail about that, 
*or* should there be some defaults, i.e., the logical extension to 3 
space, for example.  If you think the latter is good, then there is some 
information about the plotting style required.

Dan

Re: regression: 'with pixels' affects 'all.dem'

[Hans-Bernhard B. <br...@ph...>]

On Fri, 9 Jul 2004, Daniel J Sebald wrote:

[...]
> What do you think?  Certainly, duplicating this code is a no-no.  The 
> core translation could be defined as a macro and included in two 
> separate functions, one where xt and yt are doubles, one where xt and yt 
> are unsigned ints... 

As a matter of fact, map3d_xy() is a bit of a mess already as it is.
It does two unrelated jobs rolled into one function call.  This
will need some cleanup eventually.  But maybe not right now...

> or as I've argued in the discussion initiated by Ethan, just plain ints.

No.  Using 'int' for this is a placebo.  It may appear to cure the worst
of the symptoms, but that's more by coincidence and belief in the cure
than by design.

-- 
Hans-Bernhard Broeker (br...@ph...)
Even if all the snow were burnt, ashes would remain.

Re: 'with pixels' (was Re: Engines are started; ...)

[Hans-Bernhard B. <br...@ph...>]

On Fri, 9 Jul 2004, Ethan Merritt wrote:

> #ifdef EAM_HISTOGRAMS
>     /* EAM FIXME - There are places in df_readline where it would be really */
>     /* nice to know what kind of plot we are making, so I think that        */
>     /* current_plot should be a parameter to df_readline. 

I strictly disagree with that reasoning.  If df_readline needs to know
what the the plot style is, then IMHO that's a sign that there's something 
seriously wrong with df_readline().

It's separations like this being broken down or "tunnelled", tieing every
piece of the program to all others, that got gnuplot into the mess it's
still in.

Bluntly put, adding a global variable is essentially never the right
solution for any problem other than stack size limitations.  

Instead of designing a correct structure of "who is responsible for what",
creating a global means that we've given up. In the case at hand,
df_readline is the core interface between the datafile itself and the next
stage in data processing: the 'using' specifiers. It should never need to
know anything about plot styles, which are at least two steps removed from
its work.

-- 
Hans-Bernhard Broeker (br...@ph...)
Even if all the snow were burnt, ashes would remain.

Re: regression: 'with pixels' affects 'all.dem'

[Daniel J S. <dan...@ie...>]

Hans-Bernhard Broeker wrote:

>As threatened, I've begun regression-testing prospective new
>patches against the existing code.
>
>And already found the first one: the current edition of the "with pixels &
>friends" patch (after fixing a conflict in misc.c...) causes off-by-one
>differences to the current head of CVS.  Affected plots include a
>considerable subset of 'surface.dem' (starting at the first that has x**3
>in it, then every plot up to the second plot of the sinc function), the
>see-through sphere plot in world.dem, and all the pm3d "flush" option's
>demos.
>

I found and verified the change.  I recall now I needed to retain 
precision for the map3d_xy() routine beyond unsigned int for a 
hyperplane model of corner pixel coordinates in the image routine.  So I 
modified the map3d_xy() routine in util3d.c in the following way:

==========

/* Function to map from user 3D space to normalized 'camera' view
 * space, and from there directly to terminal coordinates */
void
map3d_xy(
    double x, double y, double z,
    unsigned int *xt, unsigned int *yt)
{
#ifdef WITH_IMAGE
    double xtd, ytd;
    map3d_xy_double(x, y, z, &xtd, &ytd);
    *xt = (unsigned int) xtd;
    *yt = (unsigned int) ytd;
}

void
map3d_xy_double(
    double x, double y, double z,
    double *xt, double *yt)
{
#endif
    int i, j;
    double v[4], res[4],    /* Homogeneous coords. vectors. */
     w = trans_mat[3][3];

    v[0] = map_x3d(x);        /* Normalize object space to -1..1 */
    v[1] = map_y3d(y);
    v[2] = map_z3d(z);
    v[3] = 1.0;

    for (i = 0; i < 2; i++) {    /* Dont use the third axes (z). */
    res[i] = trans_mat[3][i];    /* Initiate it with the weight factor */
    for (j = 0; j < 3; j++)
        res[i] += v[j] * trans_mat[j][i];
    }

    for (i = 0; i < 3; i++)
    w += v[i] * trans_mat[i][3];
    if (w == 0)
    w = 1e-5;

#ifdef WITH_IMAGE
    *xt = ((res[0] * xscaler / w) + xmiddle);
    *yt = ((res[1] * yscaler / w) + ymiddle);
#else
    *xt = (unsigned int) ((res[0] * xscaler / w) + xmiddle);
    *yt = (unsigned int) ((res[1] * yscaler / w) + ymiddle);
#endif
}

=========

That is, it is a routine, map3d_xy_double() that retains the precision 
and then map3d_xy() uses that routine and makes the cast to an unsigned 
int.  I certainly didn't expect that to make a change.  The difference 
here is likely that the result of

((res[0] * xscaler / w) + xmiddle)

is not cast directly from the register contents, but instead is saved in 
memory as a variable, retrieved and then cast.  Something must happen in 
that transition to result in a difference of 1.

What do you think?  Certainly, duplicating this code is a no-no.  The 
core translation could be defined as a macro and included in two 
separate functions, one where xt and yt are doubles, one where xt and yt 
are unsigned ints... or as I've argued in the discussion initiated by 
Ethan, just plain ints.

Dan

-- 

Dan Sebald
email: daniel DOT sebald AT ieee DOT org
URL: http://acer-access DOT com/~dsebald AT acer-access DOT com/