Menu
▾
▴
matplotlib-users
|
From: Andrew K. <ndr...@gm...> - 2010-04-01 21:17:50
|
Has anyone had any success in speeding up the mpl imports? "import matplotlib.pyplot as plt" ( or "from matplotlib.figure import Figure") takes 6 full seconds to load. That seems excessive. Any ideas? -Andy |
|
From: Friedrich R. <fri...@gm...> - 2010-04-01 21:44:25
|
Hmm, I wrote one time a lazy-import module, you create objects and use
their attributes, but the object imports the module not earlier than
the first attribute access. Thus these objects are used like the
module via "import module". I.e., module = Lazy('matplotlib.module').
There are also wrappers for classes for replacement of "from moudule
import Class" syntax. Is anybody interested, anyway?
Friedrich
|
|
From: Eric F. <ef...@ha...> - 2010-04-01 22:26:00
|
Andrew Kelly wrote: > Has anyone had any success in speeding up the mpl imports? > > "import matplotlib.pyplot as plt" > ( or "from matplotlib.figure import Figure") > > takes 6 full seconds to load. That seems excessive. Any ideas? Unless you have a very old machine, it sounds like something is wrong. On my laptop it is about 0.6 seconds. About 20% of that is for numpy. Are mpl and numpy installed locally on your machine, as opposed to being accessed over a samba or nfs mount? Eric > > -Andy > |
|
From: Eric F. <ef...@ha...> - 2010-04-01 23:17:12
|
Andrew Kelly wrote: > Has anyone had any success in speeding up the mpl imports? > > "import matplotlib.pyplot as plt" > ( or "from matplotlib.figure import Figure") > > takes 6 full seconds to load. That seems excessive. Any ideas? > > -Andy Andy, A couple replies came back directly to me (probably intended for the list, though), and both reported results similar to yours, on Windows machines only. What OS and version are you running? Eric |
|
From: Andrew K. <ndr...@gm...> - 2010-04-01 23:22:08
|
Eric, I am running it on a windows 7 machine and a windows XP machine. Odd that it does this only on win32. -Andy On Thu, Apr 1, 2010 at 4:17 PM, Eric Firing <ef...@ha...> wrote: > Andrew Kelly wrote: > >> Has anyone had any success in speeding up the mpl imports? >> >> "import matplotlib.pyplot as plt" ( or "from matplotlib.figure import >> Figure") >> >> takes 6 full seconds to load. That seems excessive. Any ideas? >> >> -Andy >> > > Andy, > > A couple replies came back directly to me (probably intended for the list, > though), and both reported results similar to yours, on Windows machines > only. What OS and version are you running? > > Eric > |
|
From: C M <cmp...@gm...> - 2010-04-02 00:57:14
|
On Thu, Apr 1, 2010 at 7:17 PM, Eric Firing <ef...@ha...> wrote: > Andrew Kelly wrote: >> Has anyone had any success in speeding up the mpl imports? >> >> "import matplotlib.pyplot as plt" >> ( or "from matplotlib.figure import Figure") >> >> takes 6 full seconds to load. That seems excessive. Any ideas? >> >> -Andy > > Andy, > > A couple replies came back directly to me (probably intended for the > list, though), and both reported results similar to yours, on Windows > machines only. What OS and version are you running? Sorry Eric, that was indeed intended for the list. Just for the list's sake, I'll repeat it: It takes longer than any other Python module for me, too, about 5-6 seconds on a "cold" load (on Windows), though faster on a "warm" load. I am running it locally on a laptop that is 1.7 GHz Intel Pentium laptop with 1Meg RAM. And I should add: I don't currently have Linux installed, but will soon again I hope, and I will take note of how long it takes on Linux. Thanks, Che |
|
From: Gökhan S. <gok...@gm...> - 2010-04-02 01:40:33
|
On Thu, Apr 1, 2010 at 7:57 PM, C M <cmp...@gm...> wrote:
> On Thu, Apr 1, 2010 at 7:17 PM, Eric Firing <ef...@ha...> wrote:
> > Andrew Kelly wrote:
> >> Has anyone had any success in speeding up the mpl imports?
> >>
> >> "import matplotlib.pyplot as plt"
> >> ( or "from matplotlib.figure import Figure")
> >>
> >> takes 6 full seconds to load. That seems excessive. Any ideas?
> >>
> >> -Andy
> >
> > Andy,
> >
> > A couple replies came back directly to me (probably intended for the
> > list, though), and both reported results similar to yours, on Windows
> > machines only. What OS and version are you running?
>
> Sorry Eric, that was indeed intended for the list. Just for the
> list's sake, I'll repeat it:
>
> It takes longer than any other Python module for me, too, about 5-6
> seconds on a "cold" load (on Windows), though faster on a "warm" load.
> I am running it locally on a laptop that is 1.7 GHz Intel Pentium
> laptop with 1Meg RAM.
>
> And I should add: I don't currently have Linux installed, but will
> soon again I hope, and I will take note of how long it takes on Linux.
>
> Thanks,
> Che
>
This is Intel Core 2 Duo 2.5Ghz with 4GB Ram.
================================================================================
Platform :
Linux-2.6.31.9-174.fc12.i686.PAE-i686-with-fedora-12-Constantine
Python : ('CPython', 'tags/r262', '71600')
IPython : 0.10
Matplotlib : 1.0.svn.rev8214
================================================================================
I[2]: %time import matplotlib.pyplot as plt
CPU times: user 0.35 s, sys: 0.09 s, total: 0.43 s
Wall time: 1.18 s
My test-bed is IPython -pylab and the first load always takes longer.
Probably it's caching at the first time to speed-up later imports.
--
Gökhan
|
|
From: Michael D. <md...@st...> - 2010-04-02 13:28:43
|
My gut says it's probably the GUI framework import that is dominating
the time. Which backend are you using? Does importing it take a large
amount of time as well?
Can you provide a profiler output file we can examine to narrow it
down? The following from a command prompt should be sufficient to write
out a file called "import.prof":
python.exe -c "import cProfile; prof=cProfile.Profile();
prof.run('import pylab', 'import.prof')"
Mike
C M wrote:
> On Thu, Apr 1, 2010 at 7:17 PM, Eric Firing <ef...@ha...> wrote:
>
>> Andrew Kelly wrote:
>>
>>> Has anyone had any success in speeding up the mpl imports?
>>>
>>> "import matplotlib.pyplot as plt"
>>> ( or "from matplotlib.figure import Figure")
>>>
>>> takes 6 full seconds to load. That seems excessive. Any ideas?
>>>
>>> -Andy
>>>
>> Andy,
>>
>> A couple replies came back directly to me (probably intended for the
>> list, though), and both reported results similar to yours, on Windows
>> machines only. What OS and version are you running?
>>
>
> Sorry Eric, that was indeed intended for the list. Just for the
> list's sake, I'll repeat it:
>
> It takes longer than any other Python module for me, too, about 5-6
> seconds on a "cold" load (on Windows), though faster on a "warm" load.
> I am running it locally on a laptop that is 1.7 GHz Intel Pentium
> laptop with 1Meg RAM.
>
> And I should add: I don't currently have Linux installed, but will
> soon again I hope, and I will take note of how long it takes on Linux.
>
> Thanks,
> Che
>
> ------------------------------------------------------------------------------
> Download Intel® Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> _______________________________________________
> Matplotlib-users mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-users
>
--
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA
|
|
From: Gökhan S. <gok...@gm...> - 2010-04-02 14:22:29
|
On Fri, Apr 2, 2010 at 8:28 AM, Michael Droettboom <md...@st...> wrote:
> My gut says it's probably the GUI framework import that is dominating
> the time. Which backend are you using? Does importing it take a large
> amount of time as well?
>
> Can you provide a profiler output file we can examine to narrow it
> down? The following from a command prompt should be sufficient to write
> out a file called "import.prof":
>
> python.exe -c "import cProfile; prof=cProfile.Profile();
> prof.run('import pylab', 'import.prof')"
>
> Mike
>
Just for the records,
It reads as:
python -c "import cProfile; cProfile.run('import pylab',
filename='test.out')
in Python 2.6.2
These helped me to load the profile output:
import pstats
stats = pstats.Stats("test.out")
stats.print_stats()
--
Gökhan
|
|
From: Michael D. <md...@st...> - 2010-04-02 17:55:56
|
Can you provide the actual saved profiler data? The output of the
command itself doesn't provide enough information to diagnose the
problem, since it doesn't have full file paths etc.
When you do (thanks Gökhan for the less verbose version):
python.exe -c "import cProfile; cProfile.run('import pylab', 'test.out')"
this should produce a binary file "test.out" that can be loaded with the
pstats module and used by GUI tools such as KCacheGrind to help us get
to the bottom of this.
Mike
Andrew Kelly wrote:
> I'm back.
>
> My backend is wx. "Import wx" does not really take much time to
> import at all. In fact time.time() before and after = 0.0
>
> Some computer details:
> Processor: AMD Phenom IIx4 810 Processor 2.6 GHz
> RAM: 8.00 GB
>
> As for the cProfiler output on pylab, I have attached the output as
> test.txt.
>
> -Andy
>
> On Fri, Apr 2, 2010 at 7:22 AM, Gökhan Sever <gok...@gm...
> <mailto:gok...@gm...>> wrote:
>
>
>
> On Fri, Apr 2, 2010 at 8:28 AM, Michael Droettboom
> <md...@st... <mailto:md...@st...>> wrote:
>
> My gut says it's probably the GUI framework import that is
> dominating
> the time. Which backend are you using? Does importing it
> take a large
> amount of time as well?
>
> Can you provide a profiler output file we can examine to narrow it
> down? The following from a command prompt should be
> sufficient to write
> out a file called "import.prof":
>
> python.exe -c "import cProfile; prof=cProfile.Profile();
> prof.run('import pylab', 'import.prof')"
>
> Mike
>
>
> Just for the records,
>
> It reads as:
>
> python -c "import cProfile; cProfile.run('import pylab',
> filename='test.out')
>
> in Python 2.6.2
>
> These helped me to load the profile output:
>
> import pstats
> stats = pstats.Stats("test.out")
> stats.print_stats()
>
> --
> Gökhan
>
> ------------------------------------------------------------------------------
> Download Intel® Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> _______________________________________________
> Matplotlib-users mailing list
> Mat...@li...
> <mailto:Mat...@li...>
> https://lists.sourceforge.net/lists/listinfo/matplotlib-users
>
>
--
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA
|
|
From: Michael D. <md...@st...> - 2010-04-02 19:31:59
|
It looks like most of the time is being taken up by pytz (timezone
library), which opens ~500 files. How does the total time of "import
pytz" compare?
Mike
Andrew Kelly wrote:
> I see. I was wondering why it spit out a binary file.
>
> test.out is attached...
>
> -Andy
>
> On Fri, Apr 2, 2010 at 10:55 AM, Michael Droettboom <md...@st...
> <mailto:md...@st...>> wrote:
>
> Can you provide the actual saved profiler data? The output of the
> command itself doesn't provide enough information to diagnose the
> problem, since it doesn't have full file paths etc.
>
> When you do (thanks Gökhan for the less verbose version):
>
> python.exe -c "import cProfile; cProfile.run('import pylab',
> 'test.out')"
>
> this should produce a binary file "test.out" that can be loaded
> with the pstats module and used by GUI tools such as KCacheGrind
> to help us get to the bottom of this.
>
> Mike
>
> Andrew Kelly wrote:
>
> I'm back.
>
> My backend is wx. "Import wx" does not really take much time
> to import at all. In fact time.time() before and after = 0.0
>
> Some computer details:
> Processor: AMD Phenom IIx4 810 Processor 2.6 GHz
> RAM: 8.00 GB
>
> As for the cProfiler output on pylab, I have attached the
> output as test.txt.
> -Andy
>
> On Fri, Apr 2, 2010 at 7:22 AM, Gökhan Sever
> <gok...@gm... <mailto:gok...@gm...>
> <mailto:gok...@gm... <mailto:gok...@gm...>>>
> wrote:
>
>
>
> On Fri, Apr 2, 2010 at 8:28 AM, Michael Droettboom
> <md...@st... <mailto:md...@st...>
> <mailto:md...@st... <mailto:md...@st...>>> wrote:
>
> My gut says it's probably the GUI framework import that is
> dominating
> the time. Which backend are you using? Does importing it
> take a large
> amount of time as well?
>
> Can you provide a profiler output file we can examine
> to narrow it
> down? The following from a command prompt should be
> sufficient to write
> out a file called "import.prof":
>
> python.exe -c "import cProfile; prof=cProfile.Profile();
> prof.run('import pylab', 'import.prof')"
>
> Mike
>
>
> Just for the records,
>
> It reads as:
>
> python -c "import cProfile; cProfile.run('import pylab',
> filename='test.out')
>
> in Python 2.6.2
>
> These helped me to load the profile output:
>
> import pstats
> stats = pstats.Stats("test.out")
> stats.print_stats()
>
> -- Gökhan
>
>
> ------------------------------------------------------------------------------
> Download Intel® Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling,
> find bugs
> proactively, and fine-tune applications for parallel
> performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> _______________________________________________
> Matplotlib-users mailing list
> Mat...@li...
> <mailto:Mat...@li...>
> <mailto:Mat...@li...
> <mailto:Mat...@li...>>
>
> https://lists.sourceforge.net/lists/listinfo/matplotlib-users
>
>
>
> --
> Michael Droettboom
> Science Software Branch
> Operations and Engineering Division
> Space Telescope Science Institute
> Operated by AURA for NASA
>
>
--
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA
|
|
From: Andrew K. <ndr...@gm...> - 2010-04-02 20:00:02
|
import pytz only took 0.0 seconds.
I actually just ran that pstats module and there is one line that stuck out
at me:
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.000 0.000 0.000 0.000
C:\Python26\lib\os.py:35(_get_exports_list)
560 3.107 0.006 3.107 0.006 {open}
That is ~50% of the load time. I have 0 idea what this is though.
Let me try this on my os machine.....
-Andy
On Fri, Apr 2, 2010 at 12:31 PM, Michael Droettboom <md...@st...> wrote:
> It looks like most of the time is being taken up by pytz (timezone
> library), which opens ~500 files. How does the total time of "import pytz"
> compare?
>
> Mike
>
> Andrew Kelly wrote:
>
>> I see. I was wondering why it spit out a binary file.
>>
>> test.out is attached...
>>
>> -Andy
>>
>> On Fri, Apr 2, 2010 at 10:55 AM, Michael Droettboom <md...@st...<mailto:
>> md...@st...>> wrote:
>>
>> Can you provide the actual saved profiler data? The output of the
>> command itself doesn't provide enough information to diagnose the
>> problem, since it doesn't have full file paths etc.
>>
>> When you do (thanks Gökhan for the less verbose version):
>>
>> python.exe -c "import cProfile; cProfile.run('import pylab',
>> 'test.out')"
>>
>> this should produce a binary file "test.out" that can be loaded
>> with the pstats module and used by GUI tools such as KCacheGrind
>> to help us get to the bottom of this.
>>
>> Mike
>>
>> Andrew Kelly wrote:
>>
>> I'm back.
>>
>> My backend is wx. "Import wx" does not really take much time
>> to import at all. In fact time.time() before and after = 0.0
>>
>> Some computer details:
>> Processor: AMD Phenom IIx4 810 Processor 2.6 GHz
>> RAM: 8.00 GB
>>
>> As for the cProfiler output on pylab, I have attached the
>> output as test.txt.
>> -Andy
>>
>> On Fri, Apr 2, 2010 at 7:22 AM, Gökhan Sever
>> <gok...@gm... <mailto:gok...@gm...>
>> <mailto:gok...@gm... <mailto:gok...@gm...>>>
>>
>> wrote:
>>
>>
>>
>> On Fri, Apr 2, 2010 at 8:28 AM, Michael Droettboom
>> <md...@st... <mailto:md...@st...>
>> <mailto:md...@st... <mailto:md...@st...>>> wrote:
>>
>> My gut says it's probably the GUI framework import that is
>> dominating
>> the time. Which backend are you using? Does importing it
>> take a large
>> amount of time as well?
>>
>> Can you provide a profiler output file we can examine
>> to narrow it
>> down? The following from a command prompt should be
>> sufficient to write
>> out a file called "import.prof":
>>
>> python.exe -c "import cProfile; prof=cProfile.Profile();
>> prof.run('import pylab', 'import.prof')"
>>
>> Mike
>>
>>
>> Just for the records,
>>
>> It reads as:
>>
>> python -c "import cProfile; cProfile.run('import pylab',
>> filename='test.out')
>>
>> in Python 2.6.2
>>
>> These helped me to load the profile output:
>>
>> import pstats
>> stats = pstats.Stats("test.out")
>> stats.print_stats()
>>
>> -- Gökhan
>>
>>
>> ------------------------------------------------------------------------------
>> Download Intel® Parallel Studio Eval
>> Try the new software tools for yourself. Speed compiling,
>> find bugs
>> proactively, and fine-tune applications for parallel
>> performance.
>> See why Intel Parallel Studio got high marks during beta.
>> http://p.sf.net/sfu/intel-sw-dev
>> _______________________________________________
>> Matplotlib-users mailing list
>> Mat...@li...
>> <mailto:Mat...@li...>
>> <mailto:Mat...@li...
>> <mailto:Mat...@li...>>
>>
>> https://lists.sourceforge.net/lists/listinfo/matplotlib-users
>>
>>
>>
>> -- Michael Droettboom
>> Science Software Branch
>> Operations and Engineering Division
>> Space Telescope Science Institute
>> Operated by AURA for NASA
>>
>>
>>
> --
> Michael Droettboom
> Science Software Branch
> Operations and Engineering Division
> Space Telescope Science Institute
> Operated by AURA for NASA
>
>
|
|
From: Michael D. <md...@st...> - 2010-04-05 13:36:58
|
All of those calls to "open" are being generated from the pytz import --
which is why pytz seems like the likely candidate. Is it possible you
have pytz installed as a compressed egg, or on a remote disk, or
something that may be causing a file reading penalty?
As Eric said, make sure you time the "import pytz" in a clean Python
session -- if a module is already imported in the Python interpreter, it
won't be reimported.
Mike
Andrew Kelly wrote:
> import pytz only took 0.0 seconds.
>
> I actually just ran that pstats module and there is one line that
> stuck out at me:
> ncalls tottime percall cumtime percall filename:lineno(function)
> 1 0.000 0.000 0.000 0.000
> C:\Python26\lib\os.py:35(_get_exports_list)
> 560 3.107 0.006 3.107 0.006 {open}
>
> That is ~50% of the load time. I have 0 idea what this is though.
>
> Let me try this on my os machine.....
>
> -Andy
>
> On Fri, Apr 2, 2010 at 12:31 PM, Michael Droettboom <md...@st...
> <mailto:md...@st...>> wrote:
>
> It looks like most of the time is being taken up by pytz (timezone
> library), which opens ~500 files. How does the total time of
> "import pytz" compare?
>
> Mike
>
> Andrew Kelly wrote:
>
> I see. I was wondering why it spit out a binary file.
>
> test.out is attached...
>
> -Andy
>
> On Fri, Apr 2, 2010 at 10:55 AM, Michael Droettboom
> <md...@st... <mailto:md...@st...>
> <mailto:md...@st... <mailto:md...@st...>>> wrote:
>
> Can you provide the actual saved profiler data? The output
> of the
> command itself doesn't provide enough information to
> diagnose the
> problem, since it doesn't have full file paths etc.
>
> When you do (thanks Gökhan for the less verbose version):
>
> python.exe -c "import cProfile; cProfile.run('import pylab',
> 'test.out')"
>
> this should produce a binary file "test.out" that can be loaded
> with the pstats module and used by GUI tools such as
> KCacheGrind
> to help us get to the bottom of this.
>
> Mike
>
> Andrew Kelly wrote:
>
> I'm back.
>
> My backend is wx. "Import wx" does not really take
> much time
> to import at all. In fact time.time() before and after
> = 0.0
>
> Some computer details:
> Processor: AMD Phenom IIx4 810 Processor 2.6 GHz
> RAM: 8.00 GB
>
> As for the cProfiler output on pylab, I have attached the
> output as test.txt.
> -Andy
>
> On Fri, Apr 2, 2010 at 7:22 AM, Gökhan Sever
> <gok...@gm... <mailto:gok...@gm...>
> <mailto:gok...@gm... <mailto:gok...@gm...>>
> <mailto:gok...@gm...
> <mailto:gok...@gm...> <mailto:gok...@gm...
> <mailto:gok...@gm...>>>>
>
> wrote:
>
>
>
> On Fri, Apr 2, 2010 at 8:28 AM, Michael Droettboom
> <md...@st... <mailto:md...@st...>
> <mailto:md...@st... <mailto:md...@st...>>
> <mailto:md...@st... <mailto:md...@st...>
> <mailto:md...@st... <mailto:md...@st...>>>> wrote:
>
> My gut says it's probably the GUI framework
> import that is
> dominating
> the time. Which backend are you using? Does
> importing it
> take a large
> amount of time as well?
>
> Can you provide a profiler output file we can
> examine
> to narrow it
> down? The following from a command prompt should be
> sufficient to write
> out a file called "import.prof":
>
> python.exe -c "import cProfile;
> prof=cProfile.Profile();
> prof.run('import pylab', 'import.prof')"
>
> Mike
>
>
> Just for the records,
>
> It reads as:
>
> python -c "import cProfile; cProfile.run('import pylab',
> filename='test.out')
>
> in Python 2.6.2
>
> These helped me to load the profile output:
>
> import pstats
> stats = pstats.Stats("test.out")
> stats.print_stats()
>
> -- Gökhan
>
>
> ------------------------------------------------------------------------------
> Download Intel® Parallel Studio Eval
> Try the new software tools for yourself. Speed
> compiling,
> find bugs
> proactively, and fine-tune applications for parallel
> performance.
> See why Intel Parallel Studio got high marks during
> beta.
> http://p.sf.net/sfu/intel-sw-dev
> _______________________________________________
> Matplotlib-users mailing list
> Mat...@li...
> <mailto:Mat...@li...>
> <mailto:Mat...@li...
> <mailto:Mat...@li...>>
> <mailto:Mat...@li...
> <mailto:Mat...@li...>
> <mailto:Mat...@li...
> <mailto:Mat...@li...>>>
>
>
> https://lists.sourceforge.net/lists/listinfo/matplotlib-users
>
>
>
> -- Michael Droettboom
> Science Software Branch
> Operations and Engineering Division
> Space Telescope Science Institute
> Operated by AURA for NASA
>
>
>
> --
> Michael Droettboom
> Science Software Branch
> Operations and Engineering Division
> Space Telescope Science Institute
> Operated by AURA for NASA
>
>
--
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA
|
|
From: Eric F. <ef...@ha...> - 2010-04-02 20:41:59
|
Andrew Kelly wrote:
> import pytz only took 0.0 seconds.
>
Sounds like it was already imported, so you were not really timing that
import.
On linux (ubuntu 9.10, Lenovo T60 laptop) importing pytz takes longer
than importing numpy:
efiring@manini:~$ time python -c "import pytz"
real 0m0.203s
user 0m0.144s
sys 0m0.052s
efiring@manini:~$ time python -c "import pylab"
real 0m0.626s
user 0m0.480s
sys 0m0.124s
efiring@manini:~$ time python -c "import numpy"
real 0m0.113s
user 0m0.088s
sys 0m0.020s
(Probably everything is in cache in these tests; repeats yielded similar
results.)
Eric
> I actually just ran that pstats module and there is one line that stuck
> out at me:
> ncalls tottime percall cumtime percall filename:lineno(function)
> 1 0.000 0.000 0.000 0.000
> C:\Python26\lib\os.py:35(_get_exports_list)
> 560 3.107 0.006 3.107 0.006 {open}
>
> That is ~50% of the load time. I have 0 idea what this is though.
>
> Let me try this on my os machine.....
>
> -Andy
>
> On Fri, Apr 2, 2010 at 12:31 PM, Michael Droettboom <md...@st...
> <mailto:md...@st...>> wrote:
>
> It looks like most of the time is being taken up by pytz (timezone
> library), which opens ~500 files. How does the total time of
> "import pytz" compare?
>
> Mike
>
> Andrew Kelly wrote:
>
> I see. I was wondering why it spit out a binary file.
>
> test.out is attached...
>
> -Andy
>
> On Fri, Apr 2, 2010 at 10:55 AM, Michael Droettboom
> <md...@st... <mailto:md...@st...>
> <mailto:md...@st... <mailto:md...@st...>>> wrote:
>
> Can you provide the actual saved profiler data? The output
> of the
> command itself doesn't provide enough information to diagnose the
> problem, since it doesn't have full file paths etc.
>
> When you do (thanks Gökhan for the less verbose version):
>
> python.exe -c "import cProfile; cProfile.run('import pylab',
> 'test.out')"
>
> this should produce a binary file "test.out" that can be loaded
> with the pstats module and used by GUI tools such as KCacheGrind
> to help us get to the bottom of this.
>
> Mike
>
> Andrew Kelly wrote:
>
> I'm back.
>
> My backend is wx. "Import wx" does not really take much time
> to import at all. In fact time.time() before and after = 0.0
>
> Some computer details:
> Processor: AMD Phenom IIx4 810 Processor 2.6 GHz
> RAM: 8.00 GB
>
> As for the cProfiler output on pylab, I have attached the
> output as test.txt.
> -Andy
>
> On Fri, Apr 2, 2010 at 7:22 AM, Gökhan Sever
> <gok...@gm... <mailto:gok...@gm...>
> <mailto:gok...@gm... <mailto:gok...@gm...>>
> <mailto:gok...@gm...
> <mailto:gok...@gm...> <mailto:gok...@gm...
> <mailto:gok...@gm...>>>>
>
> wrote:
>
>
>
> On Fri, Apr 2, 2010 at 8:28 AM, Michael Droettboom
> <md...@st... <mailto:md...@st...>
> <mailto:md...@st... <mailto:md...@st...>>
> <mailto:md...@st... <mailto:md...@st...>
> <mailto:md...@st... <mailto:md...@st...>>>> wrote:
>
> My gut says it's probably the GUI framework import
> that is
> dominating
> the time. Which backend are you using? Does
> importing it
> take a large
> amount of time as well?
>
> Can you provide a profiler output file we can examine
> to narrow it
> down? The following from a command prompt should be
> sufficient to write
> out a file called "import.prof":
>
> python.exe -c "import cProfile;
> prof=cProfile.Profile();
> prof.run('import pylab', 'import.prof')"
>
> Mike
>
>
> Just for the records,
>
> It reads as:
>
> python -c "import cProfile; cProfile.run('import pylab',
> filename='test.out')
>
> in Python 2.6.2
>
> These helped me to load the profile output:
>
> import pstats
> stats = pstats.Stats("test.out")
> stats.print_stats()
>
> -- Gökhan
>
>
> ------------------------------------------------------------------------------
> Download Intel® Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling,
> find bugs
> proactively, and fine-tune applications for parallel
> performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> _______________________________________________
> Matplotlib-users mailing list
> Mat...@li...
> <mailto:Mat...@li...>
> <mailto:Mat...@li...
> <mailto:Mat...@li...>>
> <mailto:Mat...@li...
> <mailto:Mat...@li...>
> <mailto:Mat...@li...
> <mailto:Mat...@li...>>>
>
>
> https://lists.sourceforge.net/lists/listinfo/matplotlib-users
>
>
>
> -- Michael Droettboom
> Science Software Branch
> Operations and Engineering Division
> Space Telescope Science Institute
> Operated by AURA for NASA
>
>
>
> --
> Michael Droettboom
> Science Software Branch
> Operations and Engineering Division
> Space Telescope Science Institute
> Operated by AURA for NASA
>
>
>
> ------------------------------------------------------------------------
>
> ------------------------------------------------------------------------------
> Download Intel® Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> Matplotlib-users mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-users
|
|
From: Andrew K. <ndr...@gm...> - 2010-04-02 21:49:56
|
Slow on my OSX machine as well. I also get this strange warning which I
have never seen before (I usually only use MPL on my windows machine).
Warning (from warnings module):
File
"/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/pytz/__init__.py",
line 29
from pkg_resources import resource_stream
UserWarning: Module numpy was already imported from
/Library/Frameworks/GDAL.framework/Versions/1.6/Python/site-packages/numpy/__init__.pyc,
but
/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages
is being added to sys.path
On Fri, Apr 2, 2010 at 1:41 PM, Eric Firing <ef...@ha...> wrote:
> Andrew Kelly wrote:
>
>> import pytz only took 0.0 seconds.
>>
>>
>
> Sounds like it was already imported, so you were not really timing that
> import.
>
> On linux (ubuntu 9.10, Lenovo T60 laptop) importing pytz takes longer than
> importing numpy:
>
>
> efiring@manini:~$ time python -c "import pytz"
>
> real 0m0.203s
> user 0m0.144s
> sys 0m0.052s
> efiring@manini:~$ time python -c "import pylab"
>
> real 0m0.626s
> user 0m0.480s
> sys 0m0.124s
>
> efiring@manini:~$ time python -c "import numpy"
>
> real 0m0.113s
> user 0m0.088s
> sys 0m0.020s
>
> (Probably everything is in cache in these tests; repeats yielded similar
> results.)
>
> Eric
>
>
> I actually just ran that pstats module and there is one line that stuck
>> out at me:
>> ncalls tottime percall cumtime percall filename:lineno(function)
>> 1 0.000 0.000 0.000 0.000
>> C:\Python26\lib\os.py:35(_get_exports_list)
>> 560 3.107 0.006 3.107 0.006 {open}
>> That is ~50% of the load time. I have 0 idea what this is though.
>> Let me try this on my os machine.....
>> -Andy
>>
>> On Fri, Apr 2, 2010 at 12:31 PM, Michael Droettboom <md...@st...<mailto:
>> md...@st...>> wrote:
>>
>> It looks like most of the time is being taken up by pytz (timezone
>> library), which opens ~500 files. How does the total time of
>> "import pytz" compare?
>>
>> Mike
>>
>> Andrew Kelly wrote:
>>
>> I see. I was wondering why it spit out a binary file.
>>
>> test.out is attached...
>>
>> -Andy
>>
>> On Fri, Apr 2, 2010 at 10:55 AM, Michael Droettboom
>> <md...@st... <mailto:md...@st...>
>> <mailto:md...@st... <mailto:md...@st...>>> wrote:
>>
>> Can you provide the actual saved profiler data? The output
>> of the
>> command itself doesn't provide enough information to diagnose
>> the
>> problem, since it doesn't have full file paths etc.
>>
>> When you do (thanks Gökhan for the less verbose version):
>>
>> python.exe -c "import cProfile; cProfile.run('import pylab',
>> 'test.out')"
>>
>> this should produce a binary file "test.out" that can be loaded
>> with the pstats module and used by GUI tools such as KCacheGrind
>> to help us get to the bottom of this.
>>
>> Mike
>>
>> Andrew Kelly wrote:
>>
>> I'm back.
>>
>> My backend is wx. "Import wx" does not really take much
>> time
>> to import at all. In fact time.time() before and after =
>> 0.0
>>
>> Some computer details:
>> Processor: AMD Phenom IIx4 810 Processor 2.6 GHz
>> RAM: 8.00 GB
>>
>> As for the cProfiler output on pylab, I have attached the
>> output as test.txt.
>> -Andy
>>
>> On Fri, Apr 2, 2010 at 7:22 AM, Gökhan Sever
>> <gok...@gm... <mailto:gok...@gm...>
>> <mailto:gok...@gm... <mailto:gok...@gm...>>
>> <mailto:gok...@gm...
>> <mailto:gok...@gm...> <mailto:gok...@gm...
>> <mailto:gok...@gm...>>>>
>>
>> wrote:
>>
>>
>>
>> On Fri, Apr 2, 2010 at 8:28 AM, Michael Droettboom
>> <md...@st... <mailto:md...@st...>
>> <mailto:md...@st... <mailto:md...@st...>>
>> <mailto:md...@st... <mailto:md...@st...>
>> <mailto:md...@st... <mailto:md...@st...>>>> wrote:
>>
>> My gut says it's probably the GUI framework import
>> that is
>> dominating
>> the time. Which backend are you using? Does
>> importing it
>> take a large
>> amount of time as well?
>>
>> Can you provide a profiler output file we can examine
>> to narrow it
>> down? The following from a command prompt should be
>> sufficient to write
>> out a file called "import.prof":
>>
>> python.exe -c "import cProfile;
>> prof=cProfile.Profile();
>> prof.run('import pylab', 'import.prof')"
>>
>> Mike
>>
>>
>> Just for the records,
>>
>> It reads as:
>>
>> python -c "import cProfile; cProfile.run('import pylab',
>> filename='test.out')
>>
>> in Python 2.6.2
>>
>> These helped me to load the profile output:
>>
>> import pstats
>> stats = pstats.Stats("test.out")
>> stats.print_stats()
>>
>> -- Gökhan
>>
>>
>> ------------------------------------------------------------------------------
>> Download Intel® Parallel Studio Eval
>> Try the new software tools for yourself. Speed compiling,
>> find bugs
>> proactively, and fine-tune applications for parallel
>> performance.
>> See why Intel Parallel Studio got high marks during beta.
>> http://p.sf.net/sfu/intel-sw-dev
>> _______________________________________________
>> Matplotlib-users mailing list
>> Mat...@li...
>> <mailto:Mat...@li...>
>> <mailto:Mat...@li...
>> <mailto:Mat...@li...>>
>> <mailto:Mat...@li...
>> <mailto:Mat...@li...>
>> <mailto:Mat...@li...
>> <mailto:Mat...@li...>>>
>>
>>
>> https://lists.sourceforge.net/lists/listinfo/matplotlib-users
>>
>>
>>
>> -- Michael Droettboom
>> Science Software Branch
>> Operations and Engineering Division
>> Space Telescope Science Institute
>> Operated by AURA for NASA
>>
>>
>>
>> -- Michael Droettboom
>> Science Software Branch
>> Operations and Engineering Division
>> Space Telescope Science Institute
>> Operated by AURA for NASA
>>
>>
>>
>> ------------------------------------------------------------------------
>>
>>
>>
>> ------------------------------------------------------------------------------
>> Download Intel® Parallel Studio Eval
>> Try the new software tools for yourself. Speed compiling, find bugs
>> proactively, and fine-tune applications for parallel performance.
>> See why Intel Parallel Studio got high marks during beta.
>> http://p.sf.net/sfu/intel-sw-dev
>>
>>
>> ------------------------------------------------------------------------
>>
>> _______________________________________________
>> Matplotlib-users mailing list
>> Mat...@li...
>> https://lists.sourceforge.net/lists/listinfo/matplotlib-users
>>
>
>
|
|
From: Eric F. <ef...@ha...> - 2010-04-05 18:54:38
|
Michael Droettboom wrote: > All of those calls to "open" are being generated from the pytz import -- > which is why pytz seems like the likely candidate. Is it possible you > have pytz installed as a compressed egg, or on a remote disk, or > something that may be causing a file reading penalty? Mike, The pytz import time, at nearly 1/3 of the total mpl import time, is crazy even on linux, given that it adds only a tiny bit of functionality, and as far as I can see, even that is only rarely used. Therefore I have committed a change so that it is imported only if and when it is required. Our examples still work, after I modified one that was trying to import timezone from mpl.dates (although it was not actually using timezone). This probably illustrates the way in which this change may break some user code: user programs requiring pytz.timezone and pytz.tzinfo will have to import them directly instead of getting them from mpl.dates. I hope this is acceptable; importing them from mpl.dates seems like bad practice anyway, since mpl.dates was not using them or modifying them but was just passing them on from pytz. I suspect pytz could be redesigned so that it would not be so horrendously slow to import, but I am not going to tackle that. After the change: efiring@manini:~$ time python -c "import pylab" real 0m0.441s user 0m0.372s sys 0m0.064s Before the change: efiring@manini:~$ time python -c "import pylab" real 0m0.626s user 0m0.480s sys 0m0.124s Again, this is recent linux on a 3-year-old laptop. Eric > > As Eric said, make sure you time the "import pytz" in a clean Python > session -- if a module is already imported in the Python interpreter, it > won't be reimported. > > Mike > > |
