#630 Check pydev performance

1.3.11
closed-fixed
nobody
None
5
2008-01-26
2008-01-17
Fabio Zadrozny
No

Hi,

First post on the list, so I first wanted to thank Fabio for the great product! We have been happily using the pydev for a year now, and we also use the extensions.

Unfortunately, PyDev version 1.3.9 is the last usable version for us, the performance dramatically dropped in 1.3.10, and I was silently hoping for 1.3.11to solve our issues, but it didn't.

We usually work in a large hierarchy of 300 python modules, which live on a network drive under SVN.
As I said, 1.3.9 is just fine to work with.
But with 1.3.10 and 1.3.11, for example hitting the Ctrl key freezes the UI for 1 second each time, CPU usage goes trough the roof for the time being, and makes the whole experience really painful. It also happens while typing...

Using a completely clean eclipse workspace didn't solve the issue, and removing the pydev extensions didn't make any difference. No other difference in other Eclipse plugins except from the PyDev version change.

So my question are, are we the only ones to have this problem, and what could we do to help solving the issue?

Thanks,

--
Aloys Baillet
Research & Development - Animal Logic

Discussion

<< < 1 2 3 > >> (Page 2 of 3)
  • Logged In: NO

    Hi,

    I am giving up on PyDev 1.3.11, it is just not usable, hanging every 3 words I type.
    BTW my system is XP32 with dual Intel procs at 3.2Ghz with 2gigs of Ram, which is not a low-end PC...
    I'd be happy to do some profiling, but I will need help to get started...

    Cheers,

    Aloys

     
  • Fabio Zadrozny
    Fabio Zadrozny
    2008-01-22

    Logged In: YES
    user_id=617340
    Originator: YES

    Can you give that report you gave before based on 1.3.9 (which you said was usable)?

     
  • Fabio Zadrozny
    Fabio Zadrozny
    2008-01-22

    Logged In: YES
    user_id=617340
    Originator: YES

    I'm taking a look here... are you using mark-occurrences? If you disable it, do things go faster?

     
  • Aloys
    Aloys
    2008-01-23

    Logged In: YES
    user_id=1616463
    Originator: NO

    Hi,

    I am using Mark Ocurrences, but disabling it doesn't make any difference.
    I actually removed the extensions from my plugins before sending the first email to be sure the problem was with the opensource pydev.
    It's funny it seems the problems get worse gradually. I used a completely clean install of eclipse and pydev 1.3.11 this morning, and so far it is alright (not fast and some small freezes, but nothing near yesterday's state). As I already did that a couple of times I know it won't solve the problem unfortunately...

    Cheers,

    Aloys

     
  • Aloys
    Aloys
    2008-01-23

    Logged In: YES
    user_id=1616463
    Originator: NO

    After 3 hours of work, it is now going to a non usable state again.
    Restarting Eclipse, cleaning the project, clicking on "force restore internal info" button do not help at all.
    Logically, it seems the problem could come from the main index cache...

     
  • Aloys
    Aloys
    2008-01-23

    Logged In: YES
    user_id=1616463
    Originator: NO

    OK, now I think the slowdowns are directly related to the package that uses win32com, even if the issue might not be win32com but my dummy python module that is used for code completion.
    All the modules where I import it just keep freezing when I edit them.
    Modules that do not import it (fewer unfortunately) seem fine (not fast and a couple of small freezes, but workable).
    I'm about to send you the bad module by email.

    Cheers,

    Aloys

     
  • Aloys
    Aloys
    2008-01-24

    Logged In: YES
    user_id=1616463
    Originator: NO

    I found out that if I delete the following folders, it doesn't make any difference on the freezes:
    workspace\.metadata\.plugins\org.eclipse.core.resources\.projects\MyProject\com.python.pydev.analysis
    workspace\.metadata\.plugins\org.eclipse.core.resources\.projects\MyProject\org.python.pydev
    workspace\.metadata\.plugins\org.python.pydev
    workspace\.metadata\.plugins\com.python.pydev.analysis
    I just don't understand what's happening... why a completely clean workspace is responsive, but a workspace with none of the above folders is not...

     
  • Tim Diggins
    Tim Diggins
    2008-01-24

    Logged In: YES
    user_id=263040
    Originator: NO

    alloys wrote:
    > I just don't understand what's happening... why a completely clean
    workspace is responsive, but a workspace with none of the above folders is
    not...

    I agree - I think a really important feature / documentation of pydev would be how to /completely/ remove any project-based pydev caches - but ideally not the pydev preferences (e.g. python interpreter location). Could be a documentation item, or could (ideally) be a project specific button (like force restore internal info)

     
  • Fabio Zadrozny
    Fabio Zadrozny
    2008-01-24

    Logged In: YES
    user_id=617340
    Originator: YES

    In the project properties in pydev -> pythonpath you can clear the caches related to a given project -- there's a button there -- so, that feature request is already taken care of :)

    Removing the system caches should make no difference (as it's not changed after it's calculated the 1st time).

    I was taking a look at things, and I think that most of the time, things get slow because:
    1. You're working on a huge file (reparsing was already optimized a lot, but you can still feel that on large files).
    -- if you work with lot's of small modules pydev handles it much better (as most of the times the modules tend to be cached already and don't need a reparse)
    2. Some constructs can slow down the type inference engine (which I think is Aloys problem, but I still couldn't find it...)

    I usually work with lots of small files (my workspace has almost 9.000 files and pydev seems to handle it well -- tweaking the Xmx settings a bit)...

    So, what I'm aiming at right now is finding those constructs and try to work around it in faster ways...

    Cheers,

    Fabio

     
<< < 1 2 3 > >> (Page 2 of 3)