Re: [Gnuplot-info] Plot matrix with GNUplot. Slow

[Ethan A M. <eam...@gm...>]

On Thursday, 16 May 2013, Yonghui wrote:
> Thanks Ethan,
> 
> I am actually making a movie. 200 pictures, each picture is the 100x100
> matrix. Here is the command:
 
> ------------plot_TDM_loop.gp"----------------
> splot '_TDTDM.dat' matrix ind i*10
> print i
> i=i+1
> if (i<200) reread
> 
> Please take a look. Thanks.

Ugh. 
So it opens the file all over again each time,
reads one line at a time until it gets to where it
finished during the previous iteration, then reads
one matrix and loops again.  That means the
time goes up as N^2 rather than N.

I'm not certain, but I would guess that it would be
faster if instead of using
  splot $FOO matrix index i*10        (By the way, why 10?  Why not 100?)
you used 
  splot $FOO matrix every 1:1:1:i::i

I may have mangled that "every" specfier right;
you may have to tweak it.  See "help every".
The idea is that in order to read the Nth block you should skip 
the preceding N-1 blocks altogether rather than processing
them line-by-line.   It still has to read in the data, but it
won't bother to parse the contents of the lines you skip.

But really I think a better idea is to split the input into
N separate files and then iterate over which file is read.
That way it goes back to O(N) rather than O(N^2).

If you split the data into files file1.dat file2.dat ... fileN.dat
your loop could look like this

set pm3d
unset surface
set view map
set palette defined (-0.01 "red", 0 "white", 0.01 "blue")
do for [i=1:N] { 
    splot 'file' . i . '.dat' matrix;
}
unset out


	Ethan


> 
> Yonghui
> 
> -----Original Message-----
> From: Ethan A Merritt [mailto:eam...@gm...] 
> Sent: Friday, May 17, 2013 12:18 AM
> To: gnu...@li...
> Cc: FortCpp
> Subject: Re: [Gnuplot-info] Plot matrix with GNUplot. Slow
> 
> On Thursday, 16 May 2013, FortCpp wrote:
> > I am using GNUplot for plotting a small matrix. The matrix is 100x100 
> > by size. e.g.
> > 
> > 1.23212 2.43123 -1.24312 ......
> > -4.23123 2.00458 5.60234 ......
> > ......
> > The data is not neatly stored in the file. So from C++ point of view, 
> > due to the lack of length per data, there is no way to load a whole 
> > number, but it has to check when the number is loading. I guess this 
> > should be the reason of the slow plotting speed.
> > 
> > Now I have 3 questions:
> > Q1: Is loading the bottle neck?
> > 
> > Q2: If I can make the data file neatly stored (setw). e.g.
> > 
> >  1.23212  2.43123 -1.24312 ......
> > -4.23123  2.00458  5.60234 ......
> > ......
> > Does the plotting speed get any improvement? (Maybe GNUplot can check 
> > what the pattern is. Thus improve the loading speed. Not sure.)
> 
> No. That won't make any difference.
> 
> > 
> > Q3: Any other options that I can set to make it faster?
> 
> gnuplot> help fpe
> 
>  The `set datafile nofpe_trap` command tells gnuplot not to re-initialize a
> floating point exception handler before every expression evaluation used
> while  reading data from an input file.  This can significantly speed data
> input from  very large files at the risk of program termination if a
> floating-point  exception is generated.
> 
> You haven't shown us the commands you use to read the file, so I don't know
> if this is relevant to your case.
> 
> There is a recent patch in the CVS version that hugely speeds up reading
> binary matrices, but since the matrix you show is in ascii that wouldn't
> help.
> 
> 	Ethan
> 
> 
> --
> Tradition is not the worship of ashes, but the preservation of fire.
>   - Gustav Mahler
> 
> 


-- 
Tradition is not the worship of ashes, but the preservation of fire.
  - Gustav Mahler