Menu ▾ ▴

Strange behaviour when debugging os.walk

2007-10-09
2013-03-15
  • Ivan Zderadicka

    Ivan Zderadicka - 2007-10-09

    Hi,

    pydev 1.3.9
    eclipse 3.3.1
    python 2.4.3 on Ubuntu 6.06

    when debugging following code:

    import os, sys

    if len(sys.argv)<2:
        print >>sys.stderr, "need dir"
        sys.exit(1)
       
    dir = unicode(sys.argv[1])
    for a,b,c in os.walk(dir,):
        print a,b,c

    os.walk fails - not dirs are listed (actually the os.listdir  within os.walk throws following error - OSError: [Errno 11] Resource temporarily unavailable: '/home/ivan/music')

    - when runs normally everything is ok, problem is only in the debbuger
    - when dir is not unicode works fine
    - also did worked in previous versions (1.3.5 I think), I've just recently upgraded eclipse+pydev.

    Any idea what might be wrong?

    Thanks

    Ivan

     
    • Ivan Zderadicka

      Ivan Zderadicka - 2007-10-09

      Actually version where debugging this code works in 1.3.4
      I.

       
      • Fabio Zadrozny

        Fabio Zadrozny - 2007-10-09

        Can you please report that as a bug? -- I need to check it better before giving you a better diagnosis about it...

        Cheers,

        Fabio

         
    • Ivan Zderadicka

      Ivan Zderadicka - 2007-10-18

      OK, I'll do
      I.

       
    • Mikko Ohtamaa

      Mikko Ohtamaa - 2008-07-07

      Hi,

      I can confirm the same behavior. I have Plone 3.1.1 run under Eclipse, Ubuntu 8.04 Hardy Heron on AMD64. Increasing the maximum number of file descriptors doesn't seem to help.

      1. In Run mode everything works

      2. In debug mode, I get this failure at os.listdir:

      2008-07-07 05:31:26 ERROR Application Couldn't install Five
      Traceback (most recent call last):
        File "/home/moo/workspace/PloneInstance/parts/zope2/lib/python/OFS/Application.py", line 786, in install_product
          initmethod(context)
        File "/home/moo/workspace/PloneInstance/parts/zope2/lib/python/Products/Five/__init__.py", line 28, in initialize
          zcml.load_site()
        File "/home/moo/workspace/PloneInstance/parts/zope2/lib/python/Products/Five/zcml.py", line 53, in load_site
          _context = xmlconfig.file(file)
        File "/home/moo/workspace/PloneInstance/parts/zope2/lib/python/zope/configuration/xmlconfig.py", line 579, in file
          include(context, name, package)
        File "/home/moo/workspace/PloneInstance/parts/zope2/lib/python/zope/configuration/xmlconfig.py", line 515, in include
          processxmlfile(f, context)
        File "/home/moo/workspace/PloneInstance/parts/zope2/lib/python/zope/configuration/xmlconfig.py", line 370, in processxmlfile
          parser.parse(src)
        File "/usr/lib/python2.4/xml/sax/expatreader.py", line 107, in parse
          xmlreader.IncrementalParser.parse(self, source)
        File "/usr/lib/python2.4/xml/sax/xmlreader.py", line 123, in parse
          self.feed(buffer)
        File "/usr/lib/python2.4/xml/sax/expatreader.py", line 207, in feed
          self._parser.Parse(data, isFinal)
        File "/usr/lib/python2.4/xml/sax/expatreader.py", line 348, in end_element_ns
          self._cont_handler.endElementNS(pair, None)
        File "/home/moo/workspace/PloneInstance/parts/zope2/lib/python/zope/configuration/xmlconfig.py", line 349, in endElementNS
          self.context.end()
        File "/home/moo/workspace/PloneInstance/parts/zope2/lib/python/zope/configuration/config.py", line 544, in end
          self.stack.pop().finish()
        File "/home/moo/workspace/PloneInstance/parts/zope2/lib/python/zope/configuration/config.py", line 692, in finish
          actions = self.handler(context, **args)
        File "/home/moo/workspace/PloneInstance/parts/zope2/lib/python/zope/configuration/xmlconfig.py", line 515, in include
          processxmlfile(f, context)
        File "/home/moo/workspace/PloneInstance/parts/zope2/lib/python/zope/configuration/xmlconfig.py", line 370, in processxmlfile
          parser.parse(src)
        File "/usr/lib/python2.4/xml/sax/expatreader.py", line 107, in parse
          xmlreader.IncrementalParser.parse(self, source)
        File "/usr/lib/python2.4/xml/sax/xmlreader.py", line 123, in parse
          self.feed(buffer)
        File "/usr/lib/python2.4/xml/sax/expatreader.py", line 207, in feed
          self._parser.Parse(data, isFinal)
        File "/usr/lib/python2.4/xml/sax/expatreader.py", line 348, in end_element_ns
          self._cont_handler.endElementNS(pair, None)
        File "/home/moo/workspace/PloneInstance/parts/zope2/lib/python/zope/configuration/xmlconfig.py", line 349, in endElementNS
          self.context.end()
        File "/home/moo/workspace/PloneInstance/parts/zope2/lib/python/zope/configuration/config.py", line 544, in end
          self.stack.pop().finish()
        File "/home/moo/workspace/PloneInstance/parts/zope2/lib/python/zope/configuration/config.py", line 692, in finish
          actions = self.handler(context, **args)
        File "/home/moo/workspace/PloneInstance/parts/zope2/lib/python/zope/configuration/xmlconfig.py", line 515, in include
          processxmlfile(f, context)
        File "/home/moo/workspace/PloneInstance/parts/zope2/lib/python/zope/configuration/xmlconfig.py", line 370, in processxmlfile
          parser.parse(src)
        File "/usr/lib/python2.4/xml/sax/expatreader.py", line 107, in parse
          xmlreader.IncrementalParser.parse(self, source)
        File "/usr/lib/python2.4/xml/sax/xmlreader.py", line 123, in parse
          self.feed(buffer)
        File "/usr/lib/python2.4/xml/sax/expatreader.py", line 207, in feed
          self._parser.Parse(data, isFinal)
        File "/usr/lib/python2.4/xml/sax/expatreader.py", line 348, in end_element_ns
          self._cont_handler.endElementNS(pair, None)
        File "/home/moo/workspace/PloneInstance/parts/zope2/lib/python/zope/configuration/xmlconfig.py", line 349, in endElementNS
          self.context.end()
        File "/home/moo/workspace/PloneInstance/parts/zope2/lib/python/zope/configuration/config.py", line 544, in end
          self.stack.pop().finish()
        File "/home/moo/workspace/PloneInstance/parts/zope2/lib/python/zope/configuration/config.py", line 692, in finish
          actions = self.handler(context, **args)
        File "/home/moo/workspace/PloneInstance/parts/plone/PlacelessTranslationService/patches.py", line 27, in compile_translations
          func(*args, **kwargs)
        File "/home/moo/workspace/PloneInstance/parts/zope2/lib/python/zope/i18n/zcml.py", line 49, in registerTranslations
          for language in os.listdir(path):
      ZopeXMLConfigurationError: File "/home/moo/workspace/PloneInstance/parts/instance/etc/site.zcml", line 5.2-5.37
          ZopeXMLConfigurationError: File "/home/moo/workspace/PloneInstance/parts/zope2/lib/python/Products/Five/configure.zcml", line 6.2-6.30
          ZopeXMLConfigurationError: File "/home/moo/workspace/PloneInstance/parts/zope2/lib/python/Products/Five/i18n.zcml", line 24.4-24.52
          OSError: [Errno 11] Resource temporarily unavailable: '/home/moo/workspace/PloneInstance/parts/zope2/lib/python/zope/app/locales'
      Traceback (most recent call last):
        File "/home/moo/.eclipse/org.eclipse.sdk.ide/updates/eclipse/plugins/org.python.pydev.debug_1.3.18/pysrc/pydevd.py", line 852, in ?
          debugger.run(setup['file'], None, None)
        File "/home/moo/.eclipse/org.eclipse.sdk.ide/updates/eclipse/plugins/org.python.pydev.debug_1.3.18/pysrc/pydevd.py", line 689, in run
          execfile(file, globals, locals) #execute the script
        File "/home/moo/workspace/PloneInstance/bin/idelauncher.py", line 57, in ?
          exec(f)
        File "/home/moo/workspace/PloneInstance/parts/zope2/lib/python/Zope2/Startup/run.py", line 58, in ?
          run()
        File "/home/moo/workspace/PloneInstance/parts/zope2/lib/python/Zope2/Startup/run.py", line 21, in run
          starter.prepare()
        File "/home/moo/workspace/PloneInstance/parts/zope2/lib/python/Zope2/Startup/__init__.py", line 102, in prepare
          self.startZope()
        File "/home/moo/workspace/PloneInstance/parts/zope2/lib/python/Zope2/Startup/__init__.py", line 278, in startZope
          Zope2.startup()
        File "/home/moo/workspace/PloneInstance/parts/zope2/lib/python/Zope2/__init__.py", line 47, in startup
          _startup()
        File "/home/moo/workspace/PloneInstance/parts/zope2/lib/python/Zope2/App/startup.py", line 102, in startup
          OFS.Application.initialize(application)
        File "/home/moo/workspace/PloneInstance/parts/zope2/lib/python/OFS/Application.py", line 309, in initialize
          initializer.initialize()
        File "/home/moo/workspace/PloneInstance/parts/zope2/lib/python/OFS/Application.py", line 338, in initialize
          self.install_products()
        File "/home/moo/workspace/PloneInstance/parts/zope2/lib/python/OFS/Application.py", line 603, in install_products
          return install_products(app)
        File "/home/moo/workspace/PloneInstance/parts/zope2/lib/python/OFS/Application.py", line 634, in install_products
          folder_permissions, raise_exc=debug_mode)
        File "/home/moo/workspace/PloneInstance/parts/zope2/lib/python/OFS/Application.py", line 786, in install_product
          initmethod(context)
        File "/home/moo/workspace/PloneInstance/parts/zope2/lib/python/Products/Five/__init__.py", line 28, in initialize
          zcml.load_site()
        File "/home/moo/workspace/PloneInstance/parts/zope2/lib/python/Products/Five/zcml.py", line 53, in load_site
          _context = xmlconfig.file(file)
        File "/home/moo/workspace/PloneInstance/parts/zope2/lib/python/zope/configuration/xmlconfig.py", line 579, in file
          include(context, name, package)
        File "/home/moo/workspace/PloneInstance/parts/zope2/lib/python/zope/configuration/xmlconfig.py", line 515, in include
          processxmlfile(f, context)
        File "/home/moo/workspace/PloneInstance/parts/zope2/lib/python/zope/configuration/xmlconfig.py", line 370, in processxmlfile
          parser.parse(src)
        File "/usr/lib/python2.4/xml/sax/expatreader.py", line 107, in parse
          xmlreader.IncrementalParser.parse(self, source)
        File "/usr/lib/python2.4/xml/sax/xmlreader.py", line 123, in parse
          self.feed(buffer)
        File "/usr/lib/python2.4/xml/sax/expatreader.py", line 207, in feed
          self._parser.Parse(data, isFinal)
        File "/usr/lib/python2.4/xml/sax/expatreader.py", line 348, in end_element_ns
          self._cont_handler.endElementNS(pair, None)
        File "/home/moo/workspace/PloneInstance/parts/zope2/lib/python/zope/configuration/xmlconfig.py", line 349, in endElementNS
          self.context.end()
        File "/home/moo/workspace/PloneInstance/parts/zope2/lib/python/zope/configuration/config.py", line 544, in end
          self.stack.pop().finish()
        File "/home/moo/workspace/PloneInstance/parts/zope2/lib/python/zope/configuration/config.py", line 692, in finish
          actions = self.handler(context, **args)
        File "/home/moo/workspace/PloneInstance/parts/zope2/lib/python/zope/configuration/xmlconfig.py", line 515, in include
          processxmlfile(f, context)
        File "/home/moo/workspace/PloneInstance/parts/zope2/lib/python/zope/configuration/xmlconfig.py", line 370, in processxmlfile
          parser.parse(src)
        File "/usr/lib/python2.4/xml/sax/expatreader.py", line 107, in parse
          xmlreader.IncrementalParser.parse(self, source)
        File "/usr/lib/python2.4/xml/sax/xmlreader.py", line 123, in parse
          self.feed(buffer)
        File "/usr/lib/python2.4/xml/sax/expatreader.py", line 207, in feed
          self._parser.Parse(data, isFinal)
        File "/usr/lib/python2.4/xml/sax/expatreader.py", line 348, in end_element_ns
          self._cont_handler.endElementNS(pair, None)
        File "/home/moo/workspace/PloneInstance/parts/zope2/lib/python/zope/configuration/xmlconfig.py", line 349, in endElementNS
          self.context.end()
        File "/home/moo/workspace/PloneInstance/parts/zope2/lib/python/zope/configuration/config.py", line 544, in end
          self.stack.pop().finish()
        File "/home/moo/workspace/PloneInstance/parts/zope2/lib/python/zope/configuration/config.py", line 692, in finish
          actions = self.handler(context, **args)
        File "/home/moo/workspace/PloneInstance/parts/zope2/lib/python/zope/configuration/xmlconfig.py", line 515, in include
          processxmlfile(f, context)
        File "/home/moo/workspace/PloneInstance/parts/zope2/lib/python/zope/configuration/xmlconfig.py", line 370, in processxmlfile
          parser.parse(src)
        File "/usr/lib/python2.4/xml/sax/expatreader.py", line 107, in parse
          xmlreader.IncrementalParser.parse(self, source)
        File "/usr/lib/python2.4/xml/sax/xmlreader.py", line 123, in parse
          self.feed(buffer)
        File "/usr/lib/python2.4/xml/sax/expatreader.py", line 207, in feed
          self._parser.Parse(data, isFinal)
        File "/usr/lib/python2.4/xml/sax/expatreader.py", line 348, in end_element_ns
          self._cont_handler.endElementNS(pair, None)
        File "/home/moo/workspace/PloneInstance/parts/zope2/lib/python/zope/configuration/xmlconfig.py", line 349, in endElementNS
          self.context.end()
        File "/home/moo/workspace/PloneInstance/parts/zope2/lib/python/zope/configuration/config.py", line 544, in end
          self.stack.pop().finish()
        File "/home/moo/workspace/PloneInstance/parts/zope2/lib/python/zope/configuration/config.py", line 692, in finish
          actions = self.handler(context, **args)
        File "/home/moo/workspace/PloneInstance/parts/plone/PlacelessTranslationService/patches.py", line 27, in compile_translations
          func(*args, **kwargs)
        File "/home/moo/workspace/PloneInstance/parts/zope2/lib/python/zope/i18n/zcml.py", line 49, in registerTranslations
          for language in os.listdir(path):
      zope.configuration.xmlconfig.ZopeXMLConfigurationError: File "/home/moo/workspace/PloneInstance/parts/instance/etc/site.zcml", line 5.2-5.37
          ZopeXMLConfigurationError: File "/home/moo/workspace/PloneInstance/parts/zope2/lib/python/Products/Five/configure.zcml", line 6.2-6.30
          ZopeXMLConfigurationError: File "/home/moo/workspace/PloneInstance/parts/zope2/lib/python/Products/Five/i18n.zcml", line 24.4-24.52
          OSError: [Errno 11] Resource temporarily unavailable: '/home/moo/workspace/PloneInstance/parts/zope2/lib/python/zope/app/locales'
      Unhandled exception in thread started by
      Error in sys.excepthook:

      Original exception was:

      ==========================================
      Relevant strace:

      open("/home/moo/workspace/PloneInstance/parts/zope2/lib/python/zope/app/locales/en/LC_MESSAGES", O_RDONLY|O_NONBLOCK|O_DIRECTORY|0x80000) = 15
      fstat(15, {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
      getdents(15, /* 4 entries */, 4096)     = 112
      getdents(15, /* 0 entries */, 4096)     = 0
      close(15)                               = 0
      stat("/home/moo/workspace/PloneInstance/parts/zope2/lib/python/zope/app/locales/en/LC_MESSAGES/zope.po", {st_mode=S_IFREG|0644, st_size=460, ...}) = 0
      stat("/home/moo/workspace/PloneInstance/parts/zope2/lib/python/zope/app/locales/en/LC_MESSAGES/zope.mo", {st_mode=S_IFREG|0644, st_size=361, ...}) = 0
      stat("/home/moo/workspace/PloneInstance/parts/zope2/lib/python/zope/app/locales/en/LC_MESSAGES/zope.mo", {st_mode=S_IFREG|0644, st_size=361, ...}) = 0
      getrlimit(RLIMIT_NOFILE, {rlim_cur=4*1024, rlim_max=4*1024}) = 0
      write(1, "Got path:/home/moo/workspace/Plo"..., 87) = 87
      write(1, "\n", 1)                       = 1
      open("/home/moo/workspace/PloneInstance/parts/zope2/lib/python/zope/app/locales", O_RDONLY|O_NONBLOCK|O_DIRECTORY|0x80000) = 15
      fstat(15, {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
      getdents(15, /* 24 entries */, 4096)    = 680
      lstat("/usr", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
      lstat("/usr/lib", {st_mode=S_IFDIR|0755, st_size=45056, ...}) = 0
      lstat("/usr/lib/python2.4", {st_mode=S_IFDIR|0755, st_size=16384, ...}) = 0
      lstat("/usr/lib/python2.4/encodings", {st_mode=S_IFDIR|0755, st_size=12288, ...}) = 0
      lstat("/usr/lib/python2.4/encodings/utf_8.py", {st_mode=S_IFREG|0644, st_size=535, ...}) = 0
      getdents(15, /* 0 entries */, 4096)     = 0
      close(15)                               = 0
      stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=1883, ...}) = 0
      stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=1883, ...}) = 0
      lstat("/usr", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
      lstat("/usr/lib", {st_mode=S_IFDIR|0755, st_size=45056, ...}) = 0
      lstat("/usr/lib/python2.4", {st_mode=S_IFDIR|0755, st_size=16384, ...}) = 0
      lstat("/usr/lib/python2.4/traceback.py", {st_mode=S_IFREG|0644, st_size=10464, ...}) = 0
      lstat("/usr", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
      lstat("/usr/lib", {st_mode=S_IFDIR|0755, st_size=45056, ...}) = 0
      lstat("/usr/lib/python2.4", {st_mode=S_IFDIR|0755, st_size=16384, ...}) = 0
      lstat("/usr/lib/python2.4/linecache.py", {st_mode=S_IFREG|0644, st_size=2880, ...}) = 0
      stat("/home/moo/workspace/PloneInstance/parts/zope2/lib/python/OFS/Application.py", {st_mode=S_IFREG|0644, st_size=39011, ...}) = 0
      open("/home/moo/workspace/PloneInstance/parts/zope2/lib/python/OFS/Application.py", O_RDONLY) = 15
      fstat(15, {st_mode=S_IFREG|0644, st_size=39011, ...}) = 0
      fstat(15, {st_mode=S_IFREG|0644, st_size=39011, ...}) = 0
      mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fc82c3be000
      read(15, "################################"..., 8192) = 8192
      read(15, "                    m = list(ob."..., 4096) = 4096
      read(15, "f = getattr(app, \'temp_folder\', "..., 4096) = 4096
      read(15, "ady installed one\n            re"..., 4096) = 4096
      read(15, "g(\'virtual_hosting\')\n           "..., 4096) = 4096
      read(15, "):\n        del Products._package"..., 4096) = 4096
      read(15, "=product_name\n        if not isd"..., 4096) = 4096
      read(15, "oved in Zope \"\n                 "..., 4096) = 4096
      read(15, " else:\n            continue\n    "..., 4096) = 2147
      read(15, "", 4096)                      = 0
      close(15)                               = 0
      munmap(0x7fc82c3be000, 4096)            = 0
      stat("/home/moo/workspace/PloneInstance/parts/zope2/lib/python/Products/Five/__init__.py", {st_mode=S_IFREG|0644, st_size=1246, ...}) = 0
      open("/home/moo/workspace/PloneInstance/parts/zope2/lib/python/Products/Five/__init__.py", O_RDONLY) = 15
      fstat(15, {st_mode=S_IFREG|0644, st_size=1246, ...}) = 0
      fstat(15, {st_mode=S_IFREG|0644, st_size=1246, ...}) = 0
      mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fc82c3be000
      read(15, "################################"..., 8192) = 1246
      read(15, "", 4096)                      = 0
      close(15)                               = 0
      munmap(0x7fc82c3be000, 4096)            = 0
      stat("/home/moo/workspace/PloneInstance/parts/zope2/lib/python/Products/Five/zcml.py", {st_mode=S_IFREG|0644, st_size=2399, ...}) = 0
      open("/home/moo/workspace/PloneInstance/parts/zope2/lib/python/Products/Five/zcml.py", O_RDONLY) = 15
      fstat(15, {st_mode=S_IFREG|0644, st_size=2399, ...}) = 0
      fstat(15, {st_mode=S_IFREG|0644, st_size=2399, ...}) = 0
      mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fc82c3be000
      read(15, "################################"..., 8192) = 2399

       
      • Fabio Zadrozny

        Fabio Zadrozny - 2008-07-07

        What may be happening is that for some reason the debugger is keeping the file descriptors alive... (which makes you run out of them). Which python/pydev version are you using?

        Can you report that as a bug?

        Cheers,

        Fabio