Learn how easy it is to sync an existing GitHub or Google Code repo to a SourceForge project! See Demo

Close

#630 Check pydev performance

1.3.11
closed-fixed
nobody
None
5
2008-01-26
2008-01-17
Fabio Zadrozny
No

Hi,

First post on the list, so I first wanted to thank Fabio for the great product! We have been happily using the pydev for a year now, and we also use the extensions.

Unfortunately, PyDev version 1.3.9 is the last usable version for us, the performance dramatically dropped in 1.3.10, and I was silently hoping for 1.3.11to solve our issues, but it didn't.

We usually work in a large hierarchy of 300 python modules, which live on a network drive under SVN.
As I said, 1.3.9 is just fine to work with.
But with 1.3.10 and 1.3.11, for example hitting the Ctrl key freezes the UI for 1 second each time, CPU usage goes trough the roof for the time being, and makes the whole experience really painful. It also happens while typing...

Using a completely clean eclipse workspace didn't solve the issue, and removing the pydev extensions didn't make any difference. No other difference in other Eclipse plugins except from the PyDev version change.

So my question are, are we the only ones to have this problem, and what could we do to help solving the issue?

Thanks,

--
Aloys Baillet
Research & Development - Animal Logic

Discussion

  • Fabio Zadrozny
    Fabio Zadrozny
    2008-01-17

    Logged In: YES
    user_id=617340
    Originator: YES

    I'll make a profile on code-completion to see what may have triggered
    that change... (I don't actually experience that on some large
    projects -- between 5-7 related projects each with around 300 - 900
    modules -- so, it may be some feature in code-completion that
    triggered that for you depending on some constructs you have).

    Just wanted to know some things about your environment:
    - are you using it with jython or python?
    - you said it happens while typing... are you using the 'autocomplete
    on all chars?'
    - if you uncheck the options for autocomplete do things work fast?
    - Do you pass some memory flags to the eclipse vm? Which (if you're
    reaching the maximum amount of memory in your install, that can be
    bad) -- have you tried raising the Xmx flag (you can edit it in
    eclipse.ini -- the default is 256m, try changing it to 512m and see if
    it works)
    - Do you have something in your error log?
    - Are you using only the Eclipse runtime or are you using the Eclipse SDK?
    - From 1.3.10 to 1.3.11, did you feel it got faster, slower or kept the same?

    Cheers,

    Fabio

     
  • Fabio Zadrozny
    Fabio Zadrozny
    2008-01-17

    Logged In: YES
    user_id=617340
    Originator: YES

    Thanks for your answers and questions, I'm not at work now so I can't try the memory yet, but I'm quite hopeful on this one, as we are using CDT + Mylyn + Pydev on a regular Eclipse, maybe the memory footprint of Pydev increased and that's why we are experiencing the problem.
    BTW I also monitored file access with filemon to see if our network could be the problem, but it didn't seem to be.
    See my answers below.

    Cheers,

    Aloys

    On Jan 16, 2008 8:46 PM, Fabio Zadrozny < fabiofz@gmail.com> wrote:

    > Unfortunately, PyDev version 1.3.9 is the last usable version for us, the
    > performance dramatically dropped in 1.3.10, and I was silently hoping for
    > 1.3.11to solve our issues, but it didn't.
    >
    > We usually work in a large hierarchy of 300 python modules, which live on a
    > network drive under SVN.
    > As I said, 1.3.9 is just fine to work with.
    > But with 1.3.10 and 1.3.11, for example hitting the Ctrl key freezes the UI
    > for 1 second each time, CPU usage goes trough the roof for the time being,
    > and makes the whole experience really painful. It also happens while
    > typing...
    >
    > Using a completely clean eclipse workspace didn't solve the issue, and
    > removing the pydev extensions didn't make any difference. No other
    > difference in other Eclipse plugins except from the PyDev version change.
    >
    > So my question are, are we the only ones to have this problem, and what
    > could we do to help solving the issue?

    I'll make a profile on code-completion to see what may have triggered
    that change... (I don't actually experience that on some large
    projects -- between 5-7 related projects each with around 300 - 900
    modules -- so, it may be some feature in code-completion that
    triggered that for you depending on some constructs you have).

    Just wanted to know some things about your environment:
    - are you using it with jython or python?

    Python only

    - you said it happens while typing... are you using the 'autocomplete
    on all chars?'

    Nope

    - if you uncheck the options for autocomplete do things work fast?

    I'll try that

    - Do you pass some memory flags to the eclipse vm? Which (if you're
    reaching the maximum amount of memory in your install, that can be
    bad) -- have you tried raising the Xmx flag (you can edit it in
    eclipse.ini -- the default is 256m, try changing it to 512m and see if
    it works)

    And that as well...

    - Do you have something in your error log?

    Not sure... where should I look for?

    - Are you using only the Eclipse runtime or are you using the Eclipse SDK?

    I believe the SDK, I need to check that. What would be the impact?

    - From 1.3.10 to 1.3.11, did you feel it got faster, slower or kept the same?

    It seems the same, but it's just a feeling... I could try to measure it.

    Cheers,

    Fabio

     
  • Fabio Zadrozny
    Fabio Zadrozny
    2008-01-17

    Logged In: YES
    user_id=617340
    Originator: YES

    > > - Do you have something in your error log?
    > Not sure... where should I look for?

    If you have the SDK installed go to window->show view->other->PDE
    Runtime->Error log. (otherwise, you have to find the log file at
    your_workspace/.metadata/.log)

    >
    > >
    > > - Are you using only the Eclipse runtime or are you using the Eclipse SDK?
    > I believe the SDK, I need to check that. What would be the impact?

    The impact is that the SDK has dependencies on JDT (so, your memory
    footprint is likely higher on the SDK than on the runtime -- it would
    be a requisite if you want to develop with jython, but as you're not,
    the runtime is recommended -- the only drawback is that you don't have
    the error log -- but you can actually get it from the SDK and just
    copy that plugin to the runtime).

    >
    > >
    > > - From 1.3.10 to 1.3.11, did you feel it got faster, slower or kept the
    > same?
    > It seems the same, but it's just a feeling... I could try to measure it.

    Ok (it's mostly so that I know at which moment to profile -- so, from
    1.3.9 to 1.3.10 seems the point to check for changes).

    Cheers,

    Fabio

     
  • Fabio Zadrozny
    Fabio Zadrozny
    2008-01-17

    Logged In: YES
    user_id=617340
    Originator: YES

    Fabio / Aloys / List -

    I also have experienced a general slowing down - however I notice that my slowing down is not on Ctrl depressing, but instead whenever pydev has to analyze a file, and particularly when the file contains classes that are heavily used ( e.g. near the top of a inheritence hierarchy). I've had to make pydev build & analyze on only on save, and then I have to put up with 3-10 seconds of total user interface blocking (e.g. can't save another file!) after that. There is then a long delay before the rest of the project is then reanalyzed ( e.g. if a method is removed and this then results in problems in other files, the problems view won't report on these for many minutes - and usually not until I open the file in question)

    My feeling is that this HASN'T got any better over the last few releases, but has got worse. I'm now moving to a newer (faster) machine, and will report back.

    (this may not be relevant to Aloys' problem, but given the title of the thread, thought it might be appropriate).

    Tim

     
  • Fabio Zadrozny
    Fabio Zadrozny
    2008-01-17

    Logged In: YES
    user_id=617340
    Originator: YES

    Hi,

    So after a day of using PyDev 1.3.11 with the following eclipse.ini:
    -showsplash
    org.eclipse.platform
    --launcher.XXMaxPermSize
    512m
    -vmargs
    -Xms80m
    -Xmx512m

    I can say it is better... but still not as good as 1.3.9, and just on the usable/not usable line...
    I haven't seen a great deal of errors, apart from one time where I had a really bad syntax error, and dozens of errors happened without really slowing the edition... Anyway, I attached a zip of the errors from today...

    Pressing Ctrl (and releasing it) still makes the UI freeze for a second or more, which is quite annoying when copying and pasting...
    But there seemed to be less freezing while editing.

    If there is any profiling I could run please let me know.

    Cheers,

    Aloys

     
  • Fabio Zadrozny
    Fabio Zadrozny
    2008-01-17

    Logged In: YES
    user_id=617340
    Originator: YES

    I've checked those errors and they should not interfere on the performance... I just took a look, and I think that the main change from 1.3.9 to 1.3.10 is that the zips/eggs are also used, so, maybe I can improve that handling to see if I can make things faster in that area (and if you have many zips/eggs, pydev will need more memory to index those, so, that's probably the reason the memory constraints got a bit higher). Can you confirm if you use many eggs/zips?

     
  • Aloys
    Aloys
    2008-01-17

    Logged In: YES
    user_id=1616463
    Originator: NO

    Hi,

    No we don't use eggs for development, only for production. And there's none of them in the PYTHONPATH.

    This morning I noticed really big slow downs that I didn't get yesterday. freezes of 2 to 15 seconds... I had them as well on 1.3.10, but never on 1.3.9. Running filemon during these showed that the win32com module was constantly being accessed on disk...

    One of my modules uses the win32com module, and when running outside of the host application (which provides an API using COM) it falls back on a regular python module to enable code completion. This particular module is in the list of "forced builtins" ...

    Here's an excerpt of the filemon logs:

    898 9:42:49.256 AM eclipse.exe:6696 IRP_MJ_CREATE C:\Documents and Settings\aloysb\workspace\.metadata\.plugins\org.eclipse.core.resources\.projects\ALPythonLibs\com.python.pydev.analysis\indexcache\AL.apps.RigSetup.test.chopper.indexcache SUCCESS Options: OverwriteIf Access: All
    899 9:42:49.256 AM eclipse.exe:6696 IRP_MJ_CLOSE C:\Documents and Settings\aloysb\workspace\.metadata\.plugins\org.eclipse.core.resources\.projects\ALPythonLibs\com.python.pydev.analysis\indexcache\AL.apps.RigSetup.test.chopper.indexcache SUCCESS
    900 9:42:49.256 AM eclipse.exe:6696 IRP_MJ_WRITE C:\Documents and Settings\aloysb\workspace\.metadata\.plugins\org.eclipse.core.resources\.projects\ALPythonLibs\com.python.pydev.analysis\indexcache\AL.apps.RigSetup.test.chopper.indexcache SUCCESS Offset: 0 Length: 924
    901 9:42:49.256 AM eclipse.exe:6696 IRP_MJ_CLEANUP C:\Documents and Settings\aloysb\workspace\.metadata\.plugins\org.eclipse.core.resources\.projects\ALPythonLibs\com.python.pydev.analysis\indexcache\AL.apps.RigSetup.test.chopper.indexcache SUCCESS
    902 9:42:49.256 AM eclipse.exe:6696 IRP_MJ_CREATE C:\Documents and Settings\aloysb\workspace\.metadata\.plugins\org.eclipse.core.resources\.projects\ALPythonLibs\com.python.pydev.analysis\indexcache\AL.apps.RigSetup.test.chopper.indexcache SUCCESS Options: Open Access: All
    903 9:42:49.256 AM eclipse.exe:6696 FASTIO_QUERY_BASIC_INFO C:\Documents and Settings\aloysb\workspace\.metadata\.plugins\org.eclipse.core.resources\.projects\ALPythonLibs\com.python.pydev.analysis\indexcache\AL.apps.RigSetup.test.chopper.indexcache SUCCESS Attributes: A
    904 9:42:49.256 AM eclipse.exe:6696 IRP_MJ_SET_INFORMATION C:\Documents and Settings\aloysb\workspace\.metadata\.plugins\org.eclipse.core.resources\.projects\ALPythonLibs\com.python.pydev.analysis\indexcache\AL.apps.RigSetup.test.chopper.indexcache SUCCESS FileBasicInformation
    905 9:42:49.256 AM eclipse.exe:6696 FASTIO_QUERY_STANDARD_INFO C:\Documents and Settings\aloysb\workspace\.metadata\.plugins\org.eclipse.core.resources\.projects\ALPythonLibs\com.python.pydev.analysis\indexcache\AL.apps.RigSetup.test.chopper.indexcache SUCCESS Length: 924
    906 9:42:49.256 AM eclipse.exe:6696 IRP_MJ_READ C:\Documents and Settings\aloysb\workspace\.metadata\.plugins\org.eclipse.core.resources\.projects\ALPythonLibs\com.python.pydev.analysis\indexcache\AL.apps.RigSetup.test.chopper.indexcache SUCCESS Offset: 0 Length: 8
    907 9:42:49.256 AM System:6696 IRP_MJ_CLEANUP C:\Documents and Settings\aloysb\workspace\.metadata\.plugins\org.eclipse.core.resources\.projects\ALPythonLibs\com.python.pydev.analysis\indexcache\AL.apps.RigSetup.test.chopper.indexcache SUCCESS
    908 9:42:49.256 AM System:6696 IRP_MJ_CLOSE C:\Documents and Settings\aloysb\workspace\.metadata\.plugins\org.eclipse.core.resources\.projects\ALPythonLibs\com.python.pydev.analysis\indexcache\AL.apps.RigSetup.test.chopper.indexcache SUCCESS
    909 9:42:49.256 AM eclipse.exe:6696 IRP_MJ_CREATE C:\Documents and Settings\aloysb\workspace\.metadata\.plugins\org.eclipse.core.resources\.projects\ALPythonLibs\com.python.pydev.analysis\indexcache\AL.apps.RigSetup.test.chopper.indexcache SUCCESS Options: Open Access: All
    910 9:42:49.256 AM eclipse.exe:6696 IRP_MJ_QUERY_INFORMATION C:\Documents and Settings\aloysb\workspace\.metadata\.plugins\org.eclipse.core.resources\.projects\ALPythonLibs\com.python.pydev.analysis\indexcache\AL.apps.RigSetup.test.chopper.indexcache SUCCESS FileBasicInformation
    911 9:42:49.256 AM System:6696 IRP_MJ_CLEANUP C:\Documents and Settings\aloysb\workspace\.metadata\.plugins\org.eclipse.core.resources\.projects\ALPythonLibs\com.python.pydev.analysis\indexcache\AL.apps.RigSetup.test.chopper.indexcache SUCCESS
    912 9:42:49.256 AM System:6696 IRP_MJ_CLOSE C:\Documents and Settings\aloysb\workspace\.metadata\.plugins\org.eclipse.core.resources\.projects\ALPythonLibs\com.python.pydev.analysis\indexcache\AL.apps.RigSetup.test.chopper.indexcache SUCCESS
    913 9:42:49.256 AM eclipse.exe:6696 IRP_MJ_CLOSE C:\Documents and Settings\aloysb\workspace\.metadata\.plugins\org.eclipse.core.resources\.projects\ALPythonLibs\com.python.pydev.analysis\indexcache\AL.apps.RigSetup.test.chopper.indexcache SUCCESS
    914 9:42:49.256 AM eclipse.exe:6696 IRP_MJ_CREATE C:\Python24\Lib\logging\__init__.py SUCCESS Options: Open Access: All
    915 9:42:49.256 AM eclipse.exe:6696 FASTIO_QUERY_BASIC_INFO C:\Python24\Lib\logging\__init__.py SUCCESS Attributes: A
    916 9:42:49.256 AM eclipse.exe:6696 IRP_MJ_CLEANUP C:\Python24\Lib\logging\__init__.py SUCCESS
    917 9:42:49.256 AM eclipse.exe:6696 IRP_MJ_CLOSE C:\Python24\Lib\logging\__init__.py SUCCESS
    918 9:42:49.256 AM eclipse.exe:6696 IRP_MJ_CREATE C:\Python24\Lib\unittest.py SUCCESS Options: Open Access: All
    919 9:42:49.256 AM eclipse.exe:6696 FASTIO_QUERY_BASIC_INFO C:\Python24\Lib\unittest.py SUCCESS Attributes: A
    920 9:42:49.256 AM eclipse.exe:6696 IRP_MJ_CLEANUP C:\Python24\Lib\unittest.py SUCCESS
    921 9:42:49.256 AM eclipse.exe:6696 IRP_MJ_CLOSE C:\Python24\Lib\unittest.py SUCCESS
    922 9:42:49.537 AM eclipse.exe:6696 IRP_MJ_CREATE C:\Python24\Lib\site-packages\win32com\client\__init__.py SUCCESS Options: Open Access: All
    923 9:42:49.537 AM eclipse.exe:6696 FASTIO_QUERY_BASIC_INFO C:\Python24\Lib\site-packages\win32com\client\__init__.py SUCCESS Attributes: A
    924 9:42:49.537 AM eclipse.exe:6696 IRP_MJ_CLEANUP C:\Python24\Lib\site-packages\win32com\client\__init__.py SUCCESS
    925 9:42:49.537 AM eclipse.exe:6696 IRP_MJ_CLOSE C:\Python24\Lib\site-packages\win32com\client\__init__.py SUCCESS
    926 9:42:49.537 AM eclipse.exe:6696 FASTIO_QUERY_OPEN C:\Python24\Lib\site-packages\win32com\client\CLSIDToClass.py SUCCESS Attributes: A
    927 9:42:49.537 AM eclipse.exe:6696 FASTIO_QUERY_OPEN C:\Python24\Lib\site-packages\win32com\client\__init__.py SUCCESS Attributes: A
    928 9:42:49.537 AM eclipse.exe:6696 FASTIO_QUERY_OPEN C:\Python24\Lib\site-packages\win32com\client\build.py SUCCESS Attributes: A
    929 9:42:49.537 AM eclipse.exe:6696 FASTIO_QUERY_OPEN C:\Python24\Lib\site-packages\win32com\client\combrowse.py SUCCESS Attributes: A
    930 9:42:49.537 AM eclipse.exe:6696 FASTIO_QUERY_OPEN C:\Python24\Lib\site-packages\win32com\client\connect.py SUCCESS Attributes: A
    931 9:42:49.537 AM eclipse.exe:6696 FASTIO_QUERY_OPEN C:\Python24\Lib\site-packages\win32com\client\dynamic.py SUCCESS Attributes: A
    932 9:42:49.537 AM eclipse.exe:6696 FASTIO_QUERY_OPEN C:\Python24\Lib\site-packages\win32com\client\gencache.py SUCCESS Attributes: A
    933 9:42:49.537 AM eclipse.exe:6696 FASTIO_QUERY_OPEN C:\Python24\Lib\site-packages\win32com\client\genpy.py SUCCESS Attributes: A
    934 9:42:49.537 AM eclipse.exe:6696 FASTIO_QUERY_OPEN C:\Python24\Lib\site-packages\win32com\client\makepy.py SUCCESS Attributes: A
    935 9:42:49.537 AM eclipse.exe:6696 FASTIO_QUERY_OPEN C:\Python24\Lib\site-packages\win32com\client\selecttlb.py SUCCESS Attributes: A
    936 9:42:49.537 AM eclipse.exe:6696 FASTIO_QUERY_OPEN C:\Python24\Lib\site-packages\win32com\client\tlbrowse.py SUCCESS Attributes: A
    937 9:42:49.537 AM eclipse.exe:6696 FASTIO_QUERY_OPEN C:\Python24\Lib\site-packages\win32com\client\util.py SUCCESS Attributes: A
    938 9:42:49.537 AM eclipse.exe:6696 FASTIO_QUERY_OPEN C:\Python24\Lib\site-packages\win32com\client\CLSIDToClass.py SUCCESS Attributes: A
    939 9:42:49.537 AM eclipse.exe:6696 FASTIO_QUERY_OPEN C:\Python24\Lib\site-packages\win32com\client\__init__.py SUCCESS Attributes: A
    940 9:42:49.537 AM eclipse.exe:6696 FASTIO_QUERY_OPEN C:\Python24\Lib\site-packages\win32com\client\build.py SUCCESS Attributes: A
    941 9:42:49.537 AM eclipse.exe:6696 FASTIO_QUERY_OPEN C:\Python24\Lib\site-packages\win32com\client\combrowse.py SUCCESS Attributes: A
    942 9:42:49.537 AM eclipse.exe:6696 FASTIO_QUERY_OPEN C:\Python24\Lib\site-packages\win32com\client\connect.py SUCCESS Attributes: A
    943 9:42:49.537 AM eclipse.exe:6696 FASTIO_QUERY_OPEN C:\Python24\Lib\site-packages\win32com\client\dynamic.py SUCCESS Attributes: A
    944 9:42:49.537 AM eclipse.exe:6696 FASTIO_QUERY_OPEN C:\Python24\Lib\site-packages\win32com\client\gencache.py SUCCESS Attributes: A
    945 9:42:49.537 AM eclipse.exe:6696 FASTIO_QUERY_OPEN C:\Python24\Lib\site-packages\win32com\client\genpy.py SUCCESS Attributes: A
    946 9:42:49.537 AM eclipse.exe:6696 FASTIO_QUERY_OPEN C:\Python24\Lib\site-packages\win32com\client\makepy.py SUCCESS Attributes: A
    947 9:42:49.537 AM eclipse.exe:6696 FASTIO_QUERY_OPEN C:\Python24\Lib\site-packages\win32com\client\selecttlb.py SUCCESS Attributes: A
    948 9:42:49.537 AM eclipse.exe:6696 FASTIO_QUERY_OPEN C:\Python24\Lib\site-packages\win32com\client\tlbrowse.py SUCCESS Attributes: A
    949 9:42:49.537 AM eclipse.exe:6696 FASTIO_QUERY_OPEN C:\Python24\Lib\site-packages\win32com\client\util.py SUCCESS Attributes: A

    Events 914 to 949 repeat 100+ times, then the last event before the freeze ends is:

    11200 9:43:03.146 AM eclipse.exe:6696 IRP_MJ_CLOSE C:\Python24\Lib\site-packages\win32com\client\__init__.py SUCCESS

    Cheers,

    Aloys

     
  • Aloys
    Aloys
    2008-01-18

    Logged In: YES
    user_id=1616463
    Originator: NO

    OK, I have a bit more information on the win32com.client being hammered by PyDev...
    From a brand new blank workspace, I added Python2.4 as a interpreter, waited for all the modules to be parsed, then restarted eclipse.
    When I added my project, during the initial build I could already see a lot of win32com modules being accessed on disc.
    And then just editing any file in there that includes a module that has a reference to win32com makes eclipse freeze, and filemon displays the same file accesses again and again...
    I'm gonna try to remove the code that calls win32com and see what happens now.

     
  • Aloys
    Aloys
    2008-01-18

    Logged In: YES
    user_id=1616463
    Originator: NO

    More information...
    In one of the modules I import in a lot of places, I had this code:

    from win32com.client import Dispatch
    try:
    xsi_app = Dispatch('XSI.Application')
    INSIDE_XSI_MODE = True
    except:
    print 'This module is now in OUTSIDE XSI mode...'
    INSIDE_XSI_MODE = False

    Leaving the first line and commenting the rest removes the win32com filesystem access problem, and removes the huge freezes that were happening.

    Now I still get big delays when I hit the Ctrl key, and looking at my friend filemon, I get:

    1 12:20:50.381 PM eclipse.exe:7400 FASTIO_QUERY_OPEN C:\Documents and Settings\aloysb\Application Data\Subversion SUCCESS Attributes: D
    2 12:20:50.381 PM eclipse.exe:7400 FASTIO_QUERY_OPEN C:\Documents and Settings\aloysb\Application Data\Subversion\auth SUCCESS Attributes: D
    3 12:20:50.381 PM eclipse.exe:7400 FASTIO_QUERY_OPEN C:\Documents and Settings\aloysb\Application Data\Subversion\auth\svn.simple SUCCESS Attributes: D
    4 12:20:50.381 PM eclipse.exe:7400 FASTIO_QUERY_OPEN C:\Documents and Settings\aloysb\Application Data\Subversion\auth\svn.username SUCCESS Attributes: D
    5 12:20:50.381 PM eclipse.exe:7400 FASTIO_QUERY_OPEN C:\Documents and Settings\aloysb\Application Data\Subversion\auth\svn.ssl.server SUCCESS Attributes: D
    6 12:20:50.396 PM eclipse.exe:7400 IRP_MJ_CREATE C:\Documents and Settings\aloysb\Application Data\Subversion\servers SUCCESS Options: Open Access: All
    7 12:20:50.396 PM eclipse.exe:7400 IRP_MJ_READ C:\Documents and Settings\aloysb\Application Data\Subversion\servers SUCCESS Offset: 0 Length: 16384
    8 12:20:50.396 PM eclipse.exe:7400 FASTIO_READ C:\Documents and Settings\aloysb\Application Data\Subversion\servers END OF FILE Offset: 3345 Length: 13039
    9 12:20:50.396 PM eclipse.exe:7400 IRP_MJ_CREATE C:\Program Files\Subversion\iconv\cp1252.so NOT FOUND Options: Open Access: All
    10 12:20:50.396 PM eclipse.exe:7400 FASTIO_QUERY_OPEN C:\Program Files\Subversion\iconv\cp1252.so NOT FOUND Attributes: Error
    11 12:20:50.396 PM eclipse.exe:7400 IRP_MJ_CREATE C:\Program Files\Subversion\iconv\windows-1252.so SUCCESS Options: Open Access: All
    12 12:20:50.396 PM eclipse.exe:7400 IRP_MJ_QUERY_VOLUME_INFORMATION C:\Program Files\Subversion\iconv\windows-1252.so SUCCESS FileFsVolumeInformation
    13 12:20:50.396 PM eclipse.exe:7400 IRP_MJ_QUERY_INFORMATION C:\Program Files\Subversion\iconv\windows-1252.so BUFFER OVERFLOW FileAllInformation
    14 12:20:50.396 PM eclipse.exe:7400 IRP_MJ_CLEANUP C:\Program Files\Subversion\iconv\windows-1252.so SUCCESS
    15 12:20:50.396 PM eclipse.exe:7400 IRP_MJ_CLOSE C:\Program Files\Subversion\iconv\windows-1252.so SUCCESS
    16 12:20:50.396 PM eclipse.exe:7400 FASTIO_QUERY_OPEN C:\Program Files\Subversion\iconv\windows-1252.so SUCCESS Attributes: A
    17 12:20:50.396 PM eclipse.exe:7400 IRP_MJ_CREATE C:\Program Files\Subversion\iconv\windows-1252.so SUCCESS Options: Open Access: Execute

    which repeats a lot with different .so from iconv...
    until

    8536 12:20:51.334 PM eclipse.exe:7400 IRP_MJ_CLEANUP C:\Program Files\Subversion\iconv\_tbl_simple.so SUCCESS
    8537 12:20:51.334 PM eclipse.exe:7400 IRP_MJ_CLOSE C:\Program Files\Subversion\iconv\_tbl_simple.so SUCCESS

    That that matches exactly the 1 second delay I get when I hit Ctrl!

    It seems my problem are system/configuration related, but I can't explain why 1.3.9 doesn't have this particular problem...
    Maybe the win32com could be logged as a separate bug?

    Cheers,

    Aloys

     
  • Aloys
    Aloys
    2008-01-18

    Logged In: YES
    user_id=1616463
    Originator: NO

    Hi again,

    The Subversion "problem" might not be related to the pydev slowdown, as I get the same filemon output on C++ files, but the text gets underligned without delay in CDT.
    Sorry wrong path...

    Cheers,

    Aloys

     
  • Logged In: NO

    Hi,

    I am giving up on PyDev 1.3.11, it is just not usable, hanging every 3 words I type.
    BTW my system is XP32 with dual Intel procs at 3.2Ghz with 2gigs of Ram, which is not a low-end PC...
    I'd be happy to do some profiling, but I will need help to get started...

    Cheers,

    Aloys

     
  • Fabio Zadrozny
    Fabio Zadrozny
    2008-01-22

    Logged In: YES
    user_id=617340
    Originator: YES

    Can you give that report you gave before based on 1.3.9 (which you said was usable)?

     
  • Fabio Zadrozny
    Fabio Zadrozny
    2008-01-22

    Logged In: YES
    user_id=617340
    Originator: YES

    I'm taking a look here... are you using mark-occurrences? If you disable it, do things go faster?

     
  • Aloys
    Aloys
    2008-01-23

    Logged In: YES
    user_id=1616463
    Originator: NO

    Hi,

    I am using Mark Ocurrences, but disabling it doesn't make any difference.
    I actually removed the extensions from my plugins before sending the first email to be sure the problem was with the opensource pydev.
    It's funny it seems the problems get worse gradually. I used a completely clean install of eclipse and pydev 1.3.11 this morning, and so far it is alright (not fast and some small freezes, but nothing near yesterday's state). As I already did that a couple of times I know it won't solve the problem unfortunately...

    Cheers,

    Aloys

     
  • Aloys
    Aloys
    2008-01-23

    Logged In: YES
    user_id=1616463
    Originator: NO

    After 3 hours of work, it is now going to a non usable state again.
    Restarting Eclipse, cleaning the project, clicking on "force restore internal info" button do not help at all.
    Logically, it seems the problem could come from the main index cache...

     
  • Aloys
    Aloys
    2008-01-23

    Logged In: YES
    user_id=1616463
    Originator: NO

    OK, now I think the slowdowns are directly related to the package that uses win32com, even if the issue might not be win32com but my dummy python module that is used for code completion.
    All the modules where I import it just keep freezing when I edit them.
    Modules that do not import it (fewer unfortunately) seem fine (not fast and a couple of small freezes, but workable).
    I'm about to send you the bad module by email.

    Cheers,

    Aloys

     
  • Aloys
    Aloys
    2008-01-24

    Logged In: YES
    user_id=1616463
    Originator: NO

    I found out that if I delete the following folders, it doesn't make any difference on the freezes:
    workspace\.metadata\.plugins\org.eclipse.core.resources\.projects\MyProject\com.python.pydev.analysis
    workspace\.metadata\.plugins\org.eclipse.core.resources\.projects\MyProject\org.python.pydev
    workspace\.metadata\.plugins\org.python.pydev
    workspace\.metadata\.plugins\com.python.pydev.analysis
    I just don't understand what's happening... why a completely clean workspace is responsive, but a workspace with none of the above folders is not...

     
  • Tim Diggins
    Tim Diggins
    2008-01-24

    Logged In: YES
    user_id=263040
    Originator: NO

    alloys wrote:
    > I just don't understand what's happening... why a completely clean
    workspace is responsive, but a workspace with none of the above folders is
    not...

    I agree - I think a really important feature / documentation of pydev would be how to /completely/ remove any project-based pydev caches - but ideally not the pydev preferences (e.g. python interpreter location). Could be a documentation item, or could (ideally) be a project specific button (like force restore internal info)

     
  • Fabio Zadrozny
    Fabio Zadrozny
    2008-01-24

    Logged In: YES
    user_id=617340
    Originator: YES

    In the project properties in pydev -> pythonpath you can clear the caches related to a given project -- there's a button there -- so, that feature request is already taken care of :)

    Removing the system caches should make no difference (as it's not changed after it's calculated the 1st time).

    I was taking a look at things, and I think that most of the time, things get slow because:
    1. You're working on a huge file (reparsing was already optimized a lot, but you can still feel that on large files).
    -- if you work with lot's of small modules pydev handles it much better (as most of the times the modules tend to be cached already and don't need a reparse)
    2. Some constructs can slow down the type inference engine (which I think is Aloys problem, but I still couldn't find it...)

    I usually work with lots of small files (my workspace has almost 9.000 files and pydev seems to handle it well -- tweaking the Xmx settings a bit)...

    So, what I'm aiming at right now is finding those constructs and try to work around it in faster ways...

    Cheers,

    Fabio

     
  • Fabio Zadrozny
    Fabio Zadrozny
    2008-01-26

    • status: open --> closed-fixed
     
  • Fabio Zadrozny
    Fabio Zadrozny
    2008-01-29

    Logged In: YES
    user_id=617340
    Originator: YES

    Ok, the release is out... please comment on it regarding this issue.

     
  • Aloys
    Aloys
    2008-02-11

    Logged In: YES
    user_id=1616463
    Originator: NO

    Hi Fabio,

    Sorry I couldn't answer earlier due to an unexpected travel.
    Yes indeed, the issues are resolved now!
    It feels much better.

    Thanks a lot for your time!

    Aloys