Re: Connect gnuplot_x11 to exterior application window

[Daniel J S. <dan...@ie...>]

>>   Enable general binary data file reading (EXPERIMENTAL)
>>   Enable X11 polygon info in binary, not ascii (EXPERIMENTAL)
> 
> 
> I have mixed feelings about these.  They do work, but the code is so
> tangled that at some point we may have to rip it apart and re-do it.
> To me that deserves the "experimental" label.  But it is the code
> layout, rather than the actual user interface or capability that 
> might change.  So I don't know that the experimental nature needs to
> be exposed to the user during ./configure. 

I don't think that is the case.  I'll re-iterate what is accomplished with the above code.  (I've said this to the list several times, I think.)

>>   Enable general binary data file reading (EXPERIMENTAL)

The situation before this patch was that 2D plots got data one way, 3D plots effectively got data another way, with the "binary matrix" being unique to 3D plots for things like pm3d.  [This was the result of the two being separated early on and allowed to grow on their own.]

The EXPERIMENTAL portion of this brings everything together.  Data comes into 2D/3D plots the very same way.  There is no longer a special routine for "binary matrix".  The implementation of that is hidden behind the scenes at a low level such that plot3d/graph3d never needs to worry about such things.

Let me try to illustrate the levels of abstraction by considering this key routine from datafile.c:

/*{{{  int df_readline(v, max) */
int
df_readline(double v[], int max)
{
#ifdef BINARY_DATA_FILE
    if (df_read_binary)
        /* General binary, matrix binary or matrix ascii
         * that's been converted to binary.
         */
        return df_readbinary(v, max);
    else
#endif
        return df_readascii(v, max);
}
/*}}} */

After the command has been parsed, the internal data file routines, via static (not global) variable "df_read_binary" knows whether it will be attempting to read a binary file or an ascii file.  "df_readline" is the only thing that plot2d/3d needs to know.  [This approach is in fact attributable to Ethan who worried about the original approach of binary code mixed in with ascii code of the original df_readline.  Admittedly, that was a mess.]

So that is one level of abstraction.

Now, that means at some point *everything* that comes in looks like "double floats" in the array "v".  What that means for big data files is that there is a bit of churning of data.  Petr and I proposed a long time ago that there be left open some way of allowing an array of data to not bloat things with the "points" concept.  That proposal was met with strong opposition, so fine.  I then think that the above is a reasonable approach.

So, abstractions:

------------------
level 1 (low)

df_readline_ascii and df_readline_binary put data into similar format, double floats.
------------------
level 2 (intermediate)

perhaps interpret that data into more meaningful chunks, for example *strings*
------------------
level 3 (application)

plot2d/3d do their things making all kinds of nifty graphs
------------------

Right now, there is no intermediate level of processing.  But let me illustrate one that could be: strings.

Recently introduced, strings are meant to read in strings as though they were some type of data object.  No problem.  However, the code for it was placed in df_readascii using a global variable, placing strings I think (correct me if I'm wrong), into a slightly different "container" than the "points" model that seems to be the basis of much of gnuplot.  (Why was it not good for there to be an array of data for big data sets, but strings should fall outside the "points" model?  Anyway...)

Had the strings instead been implemented at the (currently non-existent) intermediate level, it would work automatically for ascii and binary.  It might not be as bad as one would think.  If at the intermediate level, one knew a string was to be read, you'd just keep calling either "df_read_ascii" or "df_read_binary" until a string was complete and then return flow back up to plot3d/plot2d.  Of course, there is the detail of a char value stored in the "v" array (i.e., treating a char as a double or vice versa), but that can be worked out.

Now, as for the code in df_read_binary, I think that if one studied this code, they'd find it to be very good code.  You might have to tune into a slightly unconventional way of programming, but it does very logical stuff.  Tell me what is bad about it and needs to be redone.  I'm listening and can probably offer an explanation for why things are done the way they are or change things if there is an obvious better method.

But note that what I've offered as a level of abstraction means that if binary can be done better, it is isolated to df_read_binary() and doesn't cause problems throughout various source files.

> 
>>   Enable plot style image (EXPERIMENTAL)
>>   Enable verify every pixel coordinate in image (EXPERIMENTAL)
> 
> 
> I don't even know what that 'verify' one does.

Because we are forced to stay with the "points" method of storing data (see above), every pixel in an image has an (x,y,z) location.  Now, that data could have come from binary or ascii, with or without implied physical position.  So, it could be the case that the user did or did not supply the coordinates.  If the user did supply coordinates, there is the possibility that the the coordinates don't form a valid grid.  We then ask, should or should we not verify the location of every individual pixel?  Or is it good enough just to know the four corners of the image and assume an evenly spaced grid even though the user may have entered data that doesn't fit to that.

I'm happy with not verifying the points.  (Matlab/Octave sometimes does that sort of thing... just looks at the end points and assumes even spacing.  Also, speeds things up a bit.)  I don't know how others feel.

> Is it worth marking this separately from the image processing proper?

Maybe yes, maybe no.  But keeping this separate is good in the sense that I think the pertinent question is what I expressed above.  Does the group think it should be one way or the other?  If strong opinion in one direction emerges, we'll either get rid of the conditional part of the code or get rid verification code.

> 
> 
>>   Enable string variables (EXPERIMENTAL)
>>   Enable command line macros (EXPERIMENTAL)
>>
>>I propose to remove these flags -- these features work for some time 
>>already.

See discussion above.  I like the string variables, I just wish there were some way of incorporating them at a higher level.

I mean, the laudable idea is to remove the binary/ascii dependence as soon as possible.  It's a difficult balance of flexibility and not repeating code.

As it is in CVS now is fine.  I mean, binary is really meant for large data files in my mind.  If strings don't work for binary, that's OK.

But there is a comment in the code somewhere about wanting to introduce a variable to df_readline() which would give the file reading code an idea of what type of data it is dealing with.  This is a paradigm shift from the "all things are double floats" approach.  However, it is workable, but again I'd hope the conditional parts of it doesn't work its way all the way down to the lowest level of abstraction.

Dan