Re: Trouble with contours and binary arrays

[Ethan A M. <me...@uw...>]

(apologies if you see this twice)

On Monday, 16 Mμay 2022 15:39:25 PDT Dima Kogan wrote:
> Hi. I'm having trouble with contoured plots, and I'm guessing there's at
> least one gnuplot bug here. Can I please get some pointers?
> 
> I make a binary data file. This is an regular matrix of 32-bit floats:
> 
>   perl -E 'for $i (0..99) { for $j (0..89) { $f=sin($i/30.)+cos($j/5.); print pack("f",$f); }}' > /tmp/sincos.float32
> 
> For convenience I'm attaching this data.
> 
> I can successfully plot a heatmap:
> 
>   splot "sincos.float32" binary array=(90,100) format="%float" with image
> 
> I can also transform the data and plot:
> 
>   splot "sincos.float32" binary array=(90,100) format="%float" using ($1+100) with image


Image data is treated as a special case, so keep that in mind as we continue through this.


> This all works. Without the transform, I can also plot contours:
> 
>   set view map
>   set contour base
>   splot "sincos.float32" binary array=(90,100) format="%float" with lines nosurface
> 
> But if I add the transform, it is ignored by the contours. THIS doesn't
> work:
> 
>   set view map
>   set contour base
>   splot "sincos.float32" binary array=(90,100) format="%float" using ($1+100) with lines nosurface
> 
> There's no error message. It just produces the untransformed data.


Let's simplify by  just plotting the surface rather than contouring.
We find that
*     splot "sincos.float32" binary array=(90,100) format="%float" with lines*
works as expected.


However
*     splot "sincos.float32" binary array=(90,100) format="%float" using ($1+100)*
seemingly ignores the using spec.  This is the same thing you ran into with contouring.


Why?  
The handling of binary array/matrix data is really weird.  
I hate it, but what happens is this:


1) The program uses the "binary array=(x,y)" clause to fill in the x and y coordinates


2) The program uses the plot type to fill in the z coordinate directly from the data
    provided by the format specifier "%float".


Now it gets weird


3) The program advances the count of data columns consumed by 2 ("plot") or 3 ("splot").


4) Only now does the program look at the using specifier.  But since it advanced the 
    column count in step (3), the column numbers in the using spec are now off by either
    2 or 3 depending on whether it's a "plot" or "splot" command.   
    This means that the operation ($1+100) is actually used to stuff data value 4
     rather than data value 1 (which is x) or z (which was stuffed in data value 3).


5) If there are additional using specs, they are used for data values 5, 6, ... and so on.


The upshot is that the using specifiers _can_ affect the plot, but not as x/y/z coordinates.
For instance you could do:


*    splot "sincos.float32" binary array=(90,100) format="%float" using ($1*$1) lc variable*


The x/y/z coordinates of the surface will be exactly the same as before.
However the variable color is taken from data value 4 so the using spec can now be seen
to have an actual effect.


Where does that leave us?
If we were designing the binary input code starting now, I would argue for
doing it very differently.  But redesign of the existing commands would completely
break backwards compatibility so I don't think that's an option.
Introducing a new binary input syntax in parallel while keeping the old 
one sounds just as unattractive.  So I am at a loss how to improve things.


The image handling code also plays fast and loose with the data columns,
although that is a separate code path.  It is not currently possible, for example,
to make a 3D histogram plot of the Green component of a color image.
Exactly analogous to your matrix contouring case, you cannot re-assign the
x/y/z coordinates based on image pixel values.


> Second issue. I'd like to plot boxed contour labels. 


I have not looked at this one yet.
It may or may not come down to the same thing.


    Ethan


> This works OK with
> ASCII matrix data, but with binary data it doesn't work. I run this
> gunplot script:
> 
>   set view map
>   set contour base
>   splot "sincos.float32" binary array=(90,100) format="%float" with labels boxed nosurface
> 
> And I see this:
> 
>   "cmd" line 3: warning: Couldn't slurp 72000 bytes (return was 36000)
> 
>   splot "sincos.float32" binary array=(90,100) format="%float" with labels boxed nosurface
>                                                                                           ^
>   "cmd" line 3: All points x value undefined
> 
> For whatever reason it's trying to read 8 bytes per point instead of 4.
> Some brief debugging tells me it thinks there're two values to read from
> each point, when in reality there's just one.
> 
> Thoughts?
> 
> Thanks!
> 
> 


-- 
Ethan A Merritt
Biomolecular Structure Center,  K-428 Health Sciences Bldg
MS 357742,   University of Washington, Seattle 98195-7742