#532 Davmail provokes error with python imaplib

v3.6.5
closed-fixed
nobody
Imap (98)
5
2014-06-03
2013-06-12
Anonymous
No

Hi I've suddenly encountered an error using imaplib on some code that worked fine before.

import imaplib
m = imaplib.IMAP4('myserver','port')
m.login(r'username','password')
m.select()

gives me the error

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python2.7/imaplib.py", line 649, in select
typ, dat = self._simple_command(name, mailbox)
File "/usr/lib/python2.7/imaplib.py", line 1070, in _simple_command
return self._command_complete(name, self._command(name, *args))
File "/usr/lib/python2.7/imaplib.py", line 899, in _command_complete
raise self.abort('command: %s => %s' % (name, val))
imaplib.abort: command: SELECT => unexpected response: '* 1520 EXISTS'

I'm not sure what it means. Emails are otherwise coming through fine, and I'm using davmail as a server.

The program in its entirety saves attachments with a certain name in a specific folder.

I've stepped through it and its definitely the `m.select()` that is where its falling over.

This same program worked absolutely fine until recently.

What am I doing wrong, and how do I fix it?

The log of activity is as follows

>>> import imaplib
>>> m = imaplib.IMAP4('server','port')
>>> Debug=4
>>> m.debug
0
>>> m.debug=4
>>> m.debug
4
>>> m.login(r'username','password')
01:26.55 > HLFI1 LOGIN "username" "password"
01:30.76 < HLFI1 OK Authenticated
('OK', ['Authenticated'])
>>> m.list()
01:56.33 > HLFI2 LIST "" *
02:00.04 < * LIST (\HasNoChildren) "/" "Trash/Sent Messages"
02:00.04 < * LIST (\HasNoChildren) "/" "Sync Issues/Server Failures"
02:00.04 < * LIST (\HasNoChildren) "/" "Sync Issues/Local Failures"
02:00.04 < * LIST (\HasNoChildren) "/" "Sync Issues/Conflicts"
02:00.04 < * LIST (\HasChildren) "/" "Sync Issues"
02:00.04 < * LIST (\HasNoChildren) "/" "Junk E-mail"
02:00.04 < * LIST (\HasNoChildren) "/" "Drafts"
02:00.04 < * LIST (\HasChildren) "/" "Trash"
02:00.04 < * LIST (\HasNoChildren) "/" "Sent"
02:00.04 < * LIST (\HasNoChildren) "/" "Outbox"
02:00.04 < * LIST (\HasNoChildren) "/" "INBOX"
02:00.04 < HLFI2 OK LIST completed
('OK', ['(\\HasNoChildren) "/" "Trash/Sent Messages"', '(\\HasNoChildren) "/" "Sync Issues/Server Failures"', '(\\HasNoChildren) "/" "Sync Issues/Local Failures"', '(\\HasNoChildren) "/" "Sync Issues/Conflicts"', '(\\HasChildren) "/" "Sync Issues"', '(\\HasNoChildren) "/" "Junk E-mail"', '(\\HasNoChildren) "/" "Drafts"', '(\\HasChildren) "/" "Trash"', '(\\HasNoChildren) "/" "Sent"', '(\\HasNoChildren) "/" "Outbox"', '(\\HasNoChildren) "/" "INBOX"'])
>>> m.select()
02:21.37 > HLFI3 SELECT INBOX
02:30.87 < * 1548 EXISTS
02:30.87 last 4 IMAP4 interactions:
00:16.73 < * OK [CAPABILITY IMAP4REV1 AUTH=LOGIN MOVE] IMAP4rev1 DavMail 4.3.0-2125 server ready
00:16.73 > HLFI0 CAPABILITY
00:16.74 < * CAPABILITY IMAP4REV1 AUTH=LOGIN MOVE
00:16.77 < HLFI0 OK CAPABILITY completed
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python2.7/imaplib.py", line 649, in select
typ, dat = self._simple_command(name, mailbox)
File "/usr/lib/python2.7/imaplib.py", line 1070, in _simple_command
return self._command_complete(name, self._command(name, *args))
File "/usr/lib/python2.7/imaplib.py", line 899, in _command_complete
raise self.abort('command: %s => %s' % (name, val))
imaplib.abort: command: SELECT => unexpected response: '* 1548 EXISTS'

I understand that this seems to be occurring because of the extra spaces in the final RETURN from a query on stack overflow

http://stackoverflow.com/questions/16911238/python-imaplib-error-unexpected-repsonse

and it was suggested I file an issue report.

http://http://bugs.python.org/issue18153

also filed on python bug tracker. Suggesting it is a davmail issue

I'm using Davmail as the server on Ubuntu 13.04 and the server works fine with Thunderbird, Fetchmail and Evolution

My first attempt at filing an issue so apologies if I've done something wrong.

Discussion

  • This is a side effect of the keepalive space mechanism: As SELECT can be very slow with the backend Exchange server some IMAP clients tend to timeout during folder or message load.

    As there is no standard keep alive response IMAP RFC, a simple way to avoid client timeouts is to send a space character periodically on the connection while loading large folder/message. I do agree that this behaviour is not RFC compliant, but most IMAP clients just ignore additional spaces.

     
  • Hi Mickael, do you think you could add a setting to enable/disable this behavior manually?

    Maybe the setting could be a list of IMAP clients that do timeout so they could benefit from the keepalive... if there's a way to detect the client, of course.

     
  • tahnoon
    tahnoon
    2013-06-15

    Hi Mickael. I've continued to post on https://bugs.python.org and the imaplib maintainer has agreed to look at fixing this on imaplib if I can provide the appropriate unit test to replicate the problem. I'm not really a programmer so any pointers you can give me on how to set up a server simulation that approximates davmail or if you have one already set up as part of your development that would help solve this at the client end. Thanks

     
  • Le 15/06/2013 08:19, tahnoon a écrit :

    Hi Mickael. I've continued to post on https://bugs.python.org and the
    imaplib maintainer has agreed to look at fixing this on imaplib if I
    can provide the appropriate unit test to replicate the problem. I'm
    not really a programmer so any pointers you can give me on how to set
    up a server simulation that approximates davmail or if you have one
    already set up as part of your development that would help solve this
    at the client end. Thanks

    Well, it's not easy to reproduce without a backend Exchange server.
    Basically it should accept multiple spaces after star in untagged response.

    untested fix of imaplib.py:
    replace
    Untagged_response = re.compile(r'* (?P<type>[A-Z-]+)( (?P<data>.))?')
    with
    Untagged_response = re.compile(r'*[ ]+(?P<type>[A-Z-]+)( (?P<data>.
    ))?')

    Note that I disabled this non standard behavior in latest DavMail
    release, you can enable it in DavMail settings (KeepAlive).

    Regards,

    --
    Mickael Guessant
    mailto:mguessan@free.fr

     
  • tahnoon
    tahnoon
    2013-06-17

    Thanks Mickael. That is terrific news. I'll download the new version.

     
    • status: open --> open-fixed
    • Group: --> v3.6.5
     
  • Fixed in 4.3.3

     
    • status: open-fixed --> closed-fixed