#113 ImportFile (Ctrl-I) iter bug

closed-fixed
nobody
None
5
2012-01-19
2011-11-15
kxroberto
No

diff -ur --strip _orig/scriptutils.py ./scriptutils.py
--- _orig/scriptutils.py 2010-08-25 17:46:40 +0000
+++ ./scriptutils.py 2011-11-15 18:37:26 +0000
@@ -391,7 +391,7 @@
path, modName = os.path.split(pathName)
modName, modExt = os.path.splitext(modName)
newPath = None
- for key, mod in sys.modules.iteritems():
+ for key, mod in list(sys.modules.iteritems()):
if hasattr(mod, '__file__'):
fname = mod.__file__
base, ext = os.path.splitext(fname)

solves:

Ctrl-I / FileImport:

>>> Traceback (most recent call last):
File "C:\Python27\Lib\site-packages\pythonwin\pywin\framework\intpyapp.py", line 330, in OnFileImport
scriptutils.ImportFile()
File "C:\Python27\Lib\site-packages\pythonwin\pywin\framework\scriptutils.py", line 394, in ImportFile
for key, mod in sys.modules.iteritems():
RuntimeError: dictionary changed size during iteration
win32ui.error: Error in Command Message handler for command ID 36867, Code 0

Note: in pure Python2 "sys.modules.iteritems()" would be ok. But for 2to3 the list(sys.modules.iteritems()) should be consistent

Discussion

  • kxroberto
    kxroberto
    2011-11-15

    Note: in pure Python2 "sys.modules.items()" would be ok. But for 2to3 the list(sys.modules.iteritems()) should be consistent

     
  • Mark Hammond
    Mark Hammond
    2011-11-16

    Wouldt sys,modules.items() be even better?

     
  • Mark Hammond
    Mark Hammond
    2011-11-16

    oops - just saw your comment :) I admit I like the items() version better as it is more "consistent" WRT other existing code - the fact that 2to3 changes it is actually a feature

     
  • kxroberto
    kxroberto
    2011-11-16

    well, I do not completely understand the comment and 2to3 and if/how its used for pywin32 (auto-)maintenance ;-)

    (I don't really use Py3 except a raw one for esoteric fun tests. The problems start with my fingers not wanting to type all those extra braces for the many print statements ...).

    Isn't the problem that Py3 has only .items() but that behaves (oversmart & intransparently) like Py2's .iteritems().
    Thus the problem of iteration over a dicts items while the dict itself changes, remains.
    But 2to3 I guess (?) doesn't auto-translate d.items() to list(d.items()) ?
    If so I see no other possib than "list(sys.modules.iteritems())" to trick it for working universal.

    C:\Python23\Lib>python32
    Python 3.2.2 (default, Sep 4 2011, 09:51:08) [MSC v.1500 32 bit (Intel)] on win32
    Type "help", "copyright", "credits" or "license" for more information.
    File "C:\bin\pythonrc.py", line 19
    print "-------------- First: PRESS ENTER 2x in GDB! -------------"
    ^
    SyntaxError: invalid syntax
    >>> d={2:3,5:6}
    >>> d.items
    <built-in method items of dict object at 0x00B25930>
    >>> d.iteritems
    Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
    AttributeError: 'dict' object has no attribute 'iteritems'
    >>> for x in d.keys(): del d[x]
    ...
    Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
    RuntimeError: dictionary changed size during iteration
    >>> for x in list(d.keys()): del d[x]
    ...
    >>>
    >>>

     
  • Mark Hammond
    Mark Hammond
    2012-01-02

    Sorry it has taken me so long to get back to this, but I'm interested in how sys,modules managed to change during this iteration. Did you have some other threads or similar running at the same time which may have imported modules? I can't repro it...

     
  • kxroberto
    kxroberto
    2012-01-14

    I removed the patch again - and it was quiet for a while.
    But when e.g. the email.* package is somehow in sys.modules the error is reproducable:

    >>> import email
    [press Ctrl-I for any source window]

     
  • Mark Hammond
    Mark Hammond
    2012-01-19

    Thanks - it is the "lazy importer" in the email package which is responsible for this. Fixed in 4184:8fc48a195771

     
  • Mark Hammond
    Mark Hammond
    2012-01-19

    • status: open --> closed-fixed