Re: [RFC] Proposal to re-think binary file handling

[Daniel S. <dan...@ie...>]

Petr Mikulik <mikulik <at> physics.muni.cz> writes:

> 
> >    set binary datafile array=Infx5 format="%double%double"
> > 
> > Someone please remind me why we need oddball format specs like
> > 	format="%double%double"
> > rather than 
> > 	format="%lf%lf"
> > I'm sure I must have asked this before, when the code went into CVS,
> > but I have forgotten the answer.  Can we get rid of additional hundreds
> > of lines of obscure code simply by requiring that users provide
> > a valid C format statement for reading their own binary data files?
> 
> Implementation of general binary files comes from 2003. We exchanged a lot 
> of e-mails with Daniel Sebald. Nobody came with such a proposal as you have 
> now.
> 
> The main reason for format="%double%double" and not "%lf%lf" is that textual 
> headers of image or volume files have keywords like char, byte, double, int, 
> float, etc. That's why we had used the same keywords for "format" as well.
> 
> On the other hand, your proposal "%lf%lf" reflects the formating from 
> programmer's point of view. However, in C, there is no special formatting 
> for signed and unsigned char, so we are out of luck for this notation.

Hi Folks,

The bugs found by Shigeharu Takeno (I think there were two separate bugs)
seemed insignificant.  If they are still outstanding, point me to the bug
reports.

'set datafile binary format' was implemented from what I recall, but that
is where one of the bugs may lie.

Generally, I don't think the binary code is the awful mess claimed to be,
but sure the syntax and layout could perhaps use improvement.  I'm certain
I asked time and again for feedback five years ago, but Petr seemed the
only one to take interest.  In fact, I agreed it should be labeled
'EXPERIMENTAL', and I'm open for changes if others are.

Petr has described the reason for the more varied format specifiers; it's
for the programmer trying to communicate with gnuplot.  That is, 'double',
'uint', etc. is more the language of binary programming.  There are
machine independent and machine dependent sizes for various instances.
Anyway, parsing that code isn't too bad.  The '10x5' dimension specifier
is the inelegant one.

It wasn't fun to program the binary array specification, mostly because--
as Ethan pointed out--almost anything one can dream up causes confusion
with already-used tokens.  The comma, the colon, etc.  They cause the
parser to think it is an end-of-line, so on.  'Inf' isn't particularly
graceful, is it?  I suppose I chose that as a symbol because it was the
only thing I could think of to mean "indefinite".  However, how about a
syntax that is similar to the way variables are declared in C?

array [10][5]

Would that parse a little better?  That would be six tokens, I think, that
the parser would send back.  The syntax would allow for the 'indefinite'
dimension length

array [10][5][]

which has the look of C.

Dan