Re: [Gnuplot-info] Plot matrix with GNUplot. Slow

[Yonghui <lyh...@gm...>]

Hi Ethan,

Thanks for your suggestion. I tried every. But it didn't work. For example:
If the file (test.txt) is :
1 1 1 1
2 1 1 1
3 1 1 1
4 1 1 1

1 2 2 2
2 2 2 2
3 2 2 2
4 2 2 2

1 3 3 3
2 3 3 3
3 3 3 3
4 3 3 3

Then set pm3d, set view map, unset surf
The sp "test.txt" matrix ind 0 is exactly what I need. But I completely have
no idea on how to use every to make the same plot. If I do sp "test.txt"
matrix every :::0::0, it doesn't make anything.

Could you please tell me how to make it work?

Thanks,
Yonghui

-----Original Message-----
From: Ethan A Merritt [mailto:eam...@gm...] 
Sent: Friday, May 17, 2013 12:54 AM
To: Yonghui
Cc: gnu...@li...
Subject: Re: [Gnuplot-info] Plot matrix with GNUplot. Slow

On Thursday, 16 May 2013, Yonghui wrote:
> Thanks Ethan,
> 
> I am actually making a movie. 200 pictures, each picture is the 
> 100x100 matrix. Here is the command:
 
> ------------plot_TDM_loop.gp"----------------
> splot '_TDTDM.dat' matrix ind i*10
> print i
> i=i+1
> if (i<200) reread
> 
> Please take a look. Thanks.

Ugh. 
So it opens the file all over again each time, reads one line at a time
until it gets to where it finished during the previous iteration, then reads
one matrix and loops again.  That means the time goes up as N^2 rather than
N.

I'm not certain, but I would guess that it would be faster if instead of
using
  splot $FOO matrix index i*10        (By the way, why 10?  Why not 100?)
you used
  splot $FOO matrix every 1:1:1:i::i

I may have mangled that "every" specfier right; you may have to tweak it.
See "help every".
The idea is that in order to read the Nth block you should skip the
preceding N-1 blocks altogether rather than processing
them line-by-line.   It still has to read in the data, but it
won't bother to parse the contents of the lines you skip.

But really I think a better idea is to split the input into N separate files
and then iterate over which file is read.
That way it goes back to O(N) rather than O(N^2).

If you split the data into files file1.dat file2.dat ... fileN.dat your loop
could look like this

set pm3d
unset surface
set view map
set palette defined (-0.01 "red", 0 "white", 0.01 "blue") do for [i=1:N] { 
    splot 'file' . i . '.dat' matrix;
}
unset out


	Ethan


> 
> Yonghui
> 
> -----Original Message-----
> From: Ethan A Merritt [mailto:eam...@gm...]
> Sent: Friday, May 17, 2013 12:18 AM
> To: gnu...@li...
> Cc: FortCpp
> Subject: Re: [Gnuplot-info] Plot matrix with GNUplot. Slow
> 
> On Thursday, 16 May 2013, FortCpp wrote:
> > I am using GNUplot for plotting a small matrix. The matrix is 
> > 100x100 by size. e.g.
> > 
> > 1.23212 2.43123 -1.24312 ......
> > -4.23123 2.00458 5.60234 ......
> > ......
> > The data is not neatly stored in the file. So from C++ point of 
> > view, due to the lack of length per data, there is no way to load a 
> > whole number, but it has to check when the number is loading. I 
> > guess this should be the reason of the slow plotting speed.
> > 
> > Now I have 3 questions:
> > Q1: Is loading the bottle neck?
> > 
> > Q2: If I can make the data file neatly stored (setw). e.g.
> > 
> >  1.23212  2.43123 -1.24312 ......
> > -4.23123  2.00458  5.60234 ......
> > ......
> > Does the plotting speed get any improvement? (Maybe GNUplot can 
> > check what the pattern is. Thus improve the loading speed. Not 
> > sure.)
> 
> No. That won't make any difference.
> 
> > 
> > Q3: Any other options that I can set to make it faster?
> 
> gnuplot> help fpe
> 
>  The `set datafile nofpe_trap` command tells gnuplot not to 
> re-initialize a floating point exception handler before every 
> expression evaluation used while  reading data from an input file.  
> This can significantly speed data input from  very large files at the 
> risk of program termination if a floating-point  exception is generated.
> 
> You haven't shown us the commands you use to read the file, so I don't 
> know if this is relevant to your case.
> 
> There is a recent patch in the CVS version that hugely speeds up 
> reading binary matrices, but since the matrix you show is in ascii 
> that wouldn't help.
> 
> 	Ethan
> 
> 
> --
> Tradition is not the worship of ashes, but the preservation of fire.
>   - Gustav Mahler
> 
> 


--
Tradition is not the worship of ashes, but the preservation of fire.
  - Gustav Mahler