Re: [Gnuplot-info] pm3d with sparse data

[Lawrence M. <law...@ed...>]

Hans-Bernhard Bröker wrote:
> But I'm afraid you didn't follow the first rule of histogram 
> generation: think before you bin.  You appear to have binned way finer 
> than your data supports.  As a rule of thumb histogram bins with zero 
> entries should be the exception, not the rule.  Statistically the 
> average count per bin should be at least roughly 5, because smaller 
> counts are dominated by randomness.
>
> Put another way: there's no way the display of your histogram can 
> faithfully reproduce 256 MBytes' worth of data --- even a rather 
> high-res screen image stores nowhere near that much information.  I.e. 
> even if gnuplot managed to plot that file, the result would be 
> completely useless.
Thanks for your response. I appreciate what you're saying, but the 
distributions I'm dealing with by nature consist of finely separated 
sharp peaks, such that the high-res binning is required to discriminate 
them. For example, out of 10 000 points, some bins may have over 1000 
points even at this resolution. This is a characteristic of the data, it 
is not due to noise. Yes, you can adjust the xrange and yrange to zoom 
in on clusters of peaks and get a more typical histogram for local 
regions, but it's nice to get an overall picture as well, which is what 
I'm trying to achieve here.

Perhaps I wasn't clear also, the data files I'm working with are 150 KB 
at most, it's only when padding them with zeros to give a dense grid 
structure for gnuplot that they become unwieldy, up to 250 MB. It's 
rather like sparse vs dense matrix representations.

Plotting with impulses or just the isolines, without forming a surface, 
seems to achieve reasonable results, although I think the pm3d surface 
plot more clearly presents the data. Is there some way of interpolating 
and resampling between points of the isolines to join them up into a 
surface, without explicitly entering all the points into the data file?

Cheers,

Lawrence