Menu
▾
▴
gnuplot-info
|
From: FortCpp <lyh...@gm...> - 2013-05-17 04:56:43
|
I am using GNUplot for plotting a small matrix. The matrix is 100x100 by size. e.g. 1.23212 2.43123 -1.24312 ...... -4.23123 2.00458 5.60234 ...... ...... The data is not neatly stored in the file. So from C++ point of view, due to the lack of length per data, there is no way to load a whole number, but it has to check when the number is loading. I guess this should be the reason of the slow plotting speed. Now I have 3 questions: Q1: Is loading the bottle neck? Q2: If I can make the data file neatly stored (setw). e.g. 1.23212 2.43123 -1.24312 ...... -4.23123 2.00458 5.60234 ...... ...... Does the plotting speed get any improvement? (Maybe GNUplot can check what the pattern is. Thus improve the loading speed. Not sure.) Q3: Any other options that I can set to make it faster? Thanks! -- View this message in context: http://gnuplot.10905.n7.nabble.com/Plot-matrix-with-GNUplot-Slow-tp17365.html Sent from the Gnuplot - User mailing list archive at Nabble.com. |
|
From: Ethan A M. <eam...@gm...> - 2013-05-17 05:18:31
|
On Thursday, 16 May 2013, FortCpp wrote: > I am using GNUplot for plotting a small matrix. The matrix is 100x100 by > size. e.g. > > 1.23212 2.43123 -1.24312 ...... > -4.23123 2.00458 5.60234 ...... > ...... > The data is not neatly stored in the file. So from C++ point of view, due to > the lack of length per data, there is no way to load a whole number, but it > has to check when the number is loading. I guess this should be the reason > of the slow plotting speed. > > Now I have 3 questions: > Q1: Is loading the bottle neck? > > Q2: If I can make the data file neatly stored (setw). e.g. > > 1.23212 2.43123 -1.24312 ...... > -4.23123 2.00458 5.60234 ...... > ...... > Does the plotting speed get any improvement? (Maybe GNUplot can check what > the pattern is. Thus improve the loading speed. Not sure.) No. That won't make any difference. > > Q3: Any other options that I can set to make it faster? gnuplot> help fpe The `set datafile nofpe_trap` command tells gnuplot not to re-initialize a floating point exception handler before every expression evaluation used while reading data from an input file. This can significantly speed data input from very large files at the risk of program termination if a floating-point exception is generated. You haven't shown us the commands you use to read the file, so I don't know if this is relevant to your case. There is a recent patch in the CVS version that hugely speeds up reading binary matrices, but since the matrix you show is in ascii that wouldn't help. Ethan -- Tradition is not the worship of ashes, but the preservation of fire. - Gustav Mahler |
|
From: Yonghui <lyh...@gm...> - 2013-05-17 05:24:05
|
Thanks Ethan, I am actually making a movie. 200 pictures, each picture is the 100x100 matrix. Here is the command: -----------plot_TDM.gp-------------- set term gif size 640,640 animate delay 10 set out '@movie_TDM.gif' unset key set cbrange[-0.01:0.01] i=0 set pm3d unset surface set view map set palette defined (-0.01 "red", 0 "white", 0.01 "blue") load "plot_TDM_loop.gp" unset out quit ------------plot_TDM_loop.gp"---------------- splot '_TDTDM.dat' matrix ind i*10 print i i=i+1 if (i<200) reread Please take a look. Thanks. Yonghui -----Original Message----- From: Ethan A Merritt [mailto:eam...@gm...] Sent: Friday, May 17, 2013 12:18 AM To: gnu...@li... Cc: FortCpp Subject: Re: [Gnuplot-info] Plot matrix with GNUplot. Slow On Thursday, 16 May 2013, FortCpp wrote: > I am using GNUplot for plotting a small matrix. The matrix is 100x100 > by size. e.g. > > 1.23212 2.43123 -1.24312 ...... > -4.23123 2.00458 5.60234 ...... > ...... > The data is not neatly stored in the file. So from C++ point of view, > due to the lack of length per data, there is no way to load a whole > number, but it has to check when the number is loading. I guess this > should be the reason of the slow plotting speed. > > Now I have 3 questions: > Q1: Is loading the bottle neck? > > Q2: If I can make the data file neatly stored (setw). e.g. > > 1.23212 2.43123 -1.24312 ...... > -4.23123 2.00458 5.60234 ...... > ...... > Does the plotting speed get any improvement? (Maybe GNUplot can check > what the pattern is. Thus improve the loading speed. Not sure.) No. That won't make any difference. > > Q3: Any other options that I can set to make it faster? gnuplot> help fpe The `set datafile nofpe_trap` command tells gnuplot not to re-initialize a floating point exception handler before every expression evaluation used while reading data from an input file. This can significantly speed data input from very large files at the risk of program termination if a floating-point exception is generated. You haven't shown us the commands you use to read the file, so I don't know if this is relevant to your case. There is a recent patch in the CVS version that hugely speeds up reading binary matrices, but since the matrix you show is in ascii that wouldn't help. Ethan -- Tradition is not the worship of ashes, but the preservation of fire. - Gustav Mahler |
|
From: Ethan A M. <eam...@gm...> - 2013-05-17 05:54:30
|
On Thursday, 16 May 2013, Yonghui wrote:
> Thanks Ethan,
>
> I am actually making a movie. 200 pictures, each picture is the 100x100
> matrix. Here is the command:
> ------------plot_TDM_loop.gp"----------------
> splot '_TDTDM.dat' matrix ind i*10
> print i
> i=i+1
> if (i<200) reread
>
> Please take a look. Thanks.
Ugh.
So it opens the file all over again each time,
reads one line at a time until it gets to where it
finished during the previous iteration, then reads
one matrix and loops again. That means the
time goes up as N^2 rather than N.
I'm not certain, but I would guess that it would be
faster if instead of using
splot $FOO matrix index i*10 (By the way, why 10? Why not 100?)
you used
splot $FOO matrix every 1:1:1:i::i
I may have mangled that "every" specfier right;
you may have to tweak it. See "help every".
The idea is that in order to read the Nth block you should skip
the preceding N-1 blocks altogether rather than processing
them line-by-line. It still has to read in the data, but it
won't bother to parse the contents of the lines you skip.
But really I think a better idea is to split the input into
N separate files and then iterate over which file is read.
That way it goes back to O(N) rather than O(N^2).
If you split the data into files file1.dat file2.dat ... fileN.dat
your loop could look like this
set pm3d
unset surface
set view map
set palette defined (-0.01 "red", 0 "white", 0.01 "blue")
do for [i=1:N] {
splot 'file' . i . '.dat' matrix;
}
unset out
Ethan
>
> Yonghui
>
> -----Original Message-----
> From: Ethan A Merritt [mailto:eam...@gm...]
> Sent: Friday, May 17, 2013 12:18 AM
> To: gnu...@li...
> Cc: FortCpp
> Subject: Re: [Gnuplot-info] Plot matrix with GNUplot. Slow
>
> On Thursday, 16 May 2013, FortCpp wrote:
> > I am using GNUplot for plotting a small matrix. The matrix is 100x100
> > by size. e.g.
> >
> > 1.23212 2.43123 -1.24312 ......
> > -4.23123 2.00458 5.60234 ......
> > ......
> > The data is not neatly stored in the file. So from C++ point of view,
> > due to the lack of length per data, there is no way to load a whole
> > number, but it has to check when the number is loading. I guess this
> > should be the reason of the slow plotting speed.
> >
> > Now I have 3 questions:
> > Q1: Is loading the bottle neck?
> >
> > Q2: If I can make the data file neatly stored (setw). e.g.
> >
> > 1.23212 2.43123 -1.24312 ......
> > -4.23123 2.00458 5.60234 ......
> > ......
> > Does the plotting speed get any improvement? (Maybe GNUplot can check
> > what the pattern is. Thus improve the loading speed. Not sure.)
>
> No. That won't make any difference.
>
> >
> > Q3: Any other options that I can set to make it faster?
>
> gnuplot> help fpe
>
> The `set datafile nofpe_trap` command tells gnuplot not to re-initialize a
> floating point exception handler before every expression evaluation used
> while reading data from an input file. This can significantly speed data
> input from very large files at the risk of program termination if a
> floating-point exception is generated.
>
> You haven't shown us the commands you use to read the file, so I don't know
> if this is relevant to your case.
>
> There is a recent patch in the CVS version that hugely speeds up reading
> binary matrices, but since the matrix you show is in ascii that wouldn't
> help.
>
> Ethan
>
>
> --
> Tradition is not the worship of ashes, but the preservation of fire.
> - Gustav Mahler
>
>
--
Tradition is not the worship of ashes, but the preservation of fire.
- Gustav Mahler
|
|
From: Clement L. <the...@gm...> - 2013-05-17 07:22:17
|
On 17 May 2013 06:54, Ethan A Merritt <eam...@gm...> wrote:
> set pm3d
> unset surface
> set view map
> set palette defined (-0.01 "red", 0 "white", 0.01 "blue")
> do for [i=1:N] {
> splot 'file' . i . '.dat' matrix;
> }
> unset out
I just want to point out that you'll need version 4.6 and above. I say
this because I'm aware some people still use older versions,
particularly on university machines.
|
|
From: Yonghui <lyh...@gm...> - 2013-05-17 20:37:44
|
Hi Ethan,
Thanks for your suggestion. I tried every. But it didn't work. For example:
If the file (test.txt) is :
1 1 1 1
2 1 1 1
3 1 1 1
4 1 1 1
1 2 2 2
2 2 2 2
3 2 2 2
4 2 2 2
1 3 3 3
2 3 3 3
3 3 3 3
4 3 3 3
Then set pm3d, set view map, unset surf
The sp "test.txt" matrix ind 0 is exactly what I need. But I completely have
no idea on how to use every to make the same plot. If I do sp "test.txt"
matrix every :::0::0, it doesn't make anything.
Could you please tell me how to make it work?
Thanks,
Yonghui
-----Original Message-----
From: Ethan A Merritt [mailto:eam...@gm...]
Sent: Friday, May 17, 2013 12:54 AM
To: Yonghui
Cc: gnu...@li...
Subject: Re: [Gnuplot-info] Plot matrix with GNUplot. Slow
On Thursday, 16 May 2013, Yonghui wrote:
> Thanks Ethan,
>
> I am actually making a movie. 200 pictures, each picture is the
> 100x100 matrix. Here is the command:
> ------------plot_TDM_loop.gp"----------------
> splot '_TDTDM.dat' matrix ind i*10
> print i
> i=i+1
> if (i<200) reread
>
> Please take a look. Thanks.
Ugh.
So it opens the file all over again each time, reads one line at a time
until it gets to where it finished during the previous iteration, then reads
one matrix and loops again. That means the time goes up as N^2 rather than
N.
I'm not certain, but I would guess that it would be faster if instead of
using
splot $FOO matrix index i*10 (By the way, why 10? Why not 100?)
you used
splot $FOO matrix every 1:1:1:i::i
I may have mangled that "every" specfier right; you may have to tweak it.
See "help every".
The idea is that in order to read the Nth block you should skip the
preceding N-1 blocks altogether rather than processing
them line-by-line. It still has to read in the data, but it
won't bother to parse the contents of the lines you skip.
But really I think a better idea is to split the input into N separate files
and then iterate over which file is read.
That way it goes back to O(N) rather than O(N^2).
If you split the data into files file1.dat file2.dat ... fileN.dat your loop
could look like this
set pm3d
unset surface
set view map
set palette defined (-0.01 "red", 0 "white", 0.01 "blue") do for [i=1:N] {
splot 'file' . i . '.dat' matrix;
}
unset out
Ethan
>
> Yonghui
>
> -----Original Message-----
> From: Ethan A Merritt [mailto:eam...@gm...]
> Sent: Friday, May 17, 2013 12:18 AM
> To: gnu...@li...
> Cc: FortCpp
> Subject: Re: [Gnuplot-info] Plot matrix with GNUplot. Slow
>
> On Thursday, 16 May 2013, FortCpp wrote:
> > I am using GNUplot for plotting a small matrix. The matrix is
> > 100x100 by size. e.g.
> >
> > 1.23212 2.43123 -1.24312 ......
> > -4.23123 2.00458 5.60234 ......
> > ......
> > The data is not neatly stored in the file. So from C++ point of
> > view, due to the lack of length per data, there is no way to load a
> > whole number, but it has to check when the number is loading. I
> > guess this should be the reason of the slow plotting speed.
> >
> > Now I have 3 questions:
> > Q1: Is loading the bottle neck?
> >
> > Q2: If I can make the data file neatly stored (setw). e.g.
> >
> > 1.23212 2.43123 -1.24312 ......
> > -4.23123 2.00458 5.60234 ......
> > ......
> > Does the plotting speed get any improvement? (Maybe GNUplot can
> > check what the pattern is. Thus improve the loading speed. Not
> > sure.)
>
> No. That won't make any difference.
>
> >
> > Q3: Any other options that I can set to make it faster?
>
> gnuplot> help fpe
>
> The `set datafile nofpe_trap` command tells gnuplot not to
> re-initialize a floating point exception handler before every
> expression evaluation used while reading data from an input file.
> This can significantly speed data input from very large files at the
> risk of program termination if a floating-point exception is generated.
>
> You haven't shown us the commands you use to read the file, so I don't
> know if this is relevant to your case.
>
> There is a recent patch in the CVS version that hugely speeds up
> reading binary matrices, but since the matrix you show is in ascii
> that wouldn't help.
>
> Ethan
>
>
> --
> Tradition is not the worship of ashes, but the preservation of fire.
> - Gustav Mahler
>
>
--
Tradition is not the worship of ashes, but the preservation of fire.
- Gustav Mahler
|
|
From: walter h. <wh...@bf...> - 2013-05-19 17:40:42
|
I am ot sure that i understand your problem
do you want to plot something like a heatmap ?
i routinely plot 4000 * n via plot matrix with no problem
re,
wh
Am 17.05.2013 22:36, schrieb Yonghui:
> Hi Ethan,
>
> Thanks for your suggestion. I tried every. But it didn't work. For example:
> If the file (test.txt) is :
> 1 1 1 1
> 2 1 1 1
> 3 1 1 1
> 4 1 1 1
>
> 1 2 2 2
> 2 2 2 2
> 3 2 2 2
> 4 2 2 2
>
> 1 3 3 3
> 2 3 3 3
> 3 3 3 3
> 4 3 3 3
>
> Then set pm3d, set view map, unset surf
> The sp "test.txt" matrix ind 0 is exactly what I need. But I completely have
> no idea on how to use every to make the same plot. If I do sp "test.txt"
> matrix every :::0::0, it doesn't make anything.
>
> Could you please tell me how to make it work?
>
> Thanks,
> Yonghui
>
> -----Original Message-----
> From: Ethan A Merritt [mailto:eam...@gm...]
> Sent: Friday, May 17, 2013 12:54 AM
> To: Yonghui
> Cc: gnu...@li...
> Subject: Re: [Gnuplot-info] Plot matrix with GNUplot. Slow
>
> On Thursday, 16 May 2013, Yonghui wrote:
>> Thanks Ethan,
>>
>> I am actually making a movie. 200 pictures, each picture is the
>> 100x100 matrix. Here is the command:
>
>> ------------plot_TDM_loop.gp"----------------
>> splot '_TDTDM.dat' matrix ind i*10
>> print i
>> i=i+1
>> if (i<200) reread
>>
>> Please take a look. Thanks.
>
> Ugh.
> So it opens the file all over again each time, reads one line at a time
> until it gets to where it finished during the previous iteration, then reads
> one matrix and loops again. That means the time goes up as N^2 rather than
> N.
>
> I'm not certain, but I would guess that it would be faster if instead of
> using
> splot $FOO matrix index i*10 (By the way, why 10? Why not 100?)
> you used
> splot $FOO matrix every 1:1:1:i::i
>
> I may have mangled that "every" specfier right; you may have to tweak it.
> See "help every".
> The idea is that in order to read the Nth block you should skip the
> preceding N-1 blocks altogether rather than processing
> them line-by-line. It still has to read in the data, but it
> won't bother to parse the contents of the lines you skip.
>
> But really I think a better idea is to split the input into N separate files
> and then iterate over which file is read.
> That way it goes back to O(N) rather than O(N^2).
>
> If you split the data into files file1.dat file2.dat ... fileN.dat your loop
> could look like this
>
> set pm3d
> unset surface
> set view map
> set palette defined (-0.01 "red", 0 "white", 0.01 "blue") do for [i=1:N] {
> splot 'file' . i . '.dat' matrix;
> }
> unset out
>
>
> Ethan
>
>
>>
>> Yonghui
>>
>> -----Original Message-----
>> From: Ethan A Merritt [mailto:eam...@gm...]
>> Sent: Friday, May 17, 2013 12:18 AM
>> To: gnu...@li...
>> Cc: FortCpp
>> Subject: Re: [Gnuplot-info] Plot matrix with GNUplot. Slow
>>
>> On Thursday, 16 May 2013, FortCpp wrote:
>>> I am using GNUplot for plotting a small matrix. The matrix is
>>> 100x100 by size. e.g.
>>>
>>> 1.23212 2.43123 -1.24312 ......
>>> -4.23123 2.00458 5.60234 ......
>>> ......
>>> The data is not neatly stored in the file. So from C++ point of
>>> view, due to the lack of length per data, there is no way to load a
>>> whole number, but it has to check when the number is loading. I
>>> guess this should be the reason of the slow plotting speed.
>>>
>>> Now I have 3 questions:
>>> Q1: Is loading the bottle neck?
>>>
>>> Q2: If I can make the data file neatly stored (setw). e.g.
>>>
>>> 1.23212 2.43123 -1.24312 ......
>>> -4.23123 2.00458 5.60234 ......
>>> ......
>>> Does the plotting speed get any improvement? (Maybe GNUplot can
>>> check what the pattern is. Thus improve the loading speed. Not
>>> sure.)
>>
>> No. That won't make any difference.
>>
>>>
>>> Q3: Any other options that I can set to make it faster?
>>
>> gnuplot> help fpe
>>
>> The `set datafile nofpe_trap` command tells gnuplot not to
>> re-initialize a floating point exception handler before every
>> expression evaluation used while reading data from an input file.
>> This can significantly speed data input from very large files at the
>> risk of program termination if a floating-point exception is generated.
>>
>> You haven't shown us the commands you use to read the file, so I don't
>> know if this is relevant to your case.
>>
>> There is a recent patch in the CVS version that hugely speeds up
>> reading binary matrices, but since the matrix you show is in ascii
>> that wouldn't help.
>>
>> Ethan
>>
>>
>> --
>> Tradition is not the worship of ashes, but the preservation of fire.
>> - Gustav Mahler
>>
>>
>
>
> --
> Tradition is not the worship of ashes, but the preservation of fire.
> - Gustav Mahler
>
>
> ------------------------------------------------------------------------------
> AlienVault Unified Security Management (USM) platform delivers complete
> security visibility with the essential security capabilities. Easily and
> efficiently configure, manage, and operate all of your security controls
> from a single console and one unified framework. Download a free trial.
> http://p.sf.net/sfu/alienvault_d2d
> _______________________________________________
> gnuplot-info mailing list
> gnu...@li...
> https://lists.sourceforge.net/lists/listinfo/gnuplot-info
|
