#8 untagged responses: an empty string is added when a literal is received

[Pilou]

Literal strings sent by IMAP servers are not correctly handled by imaplib2: an empty string is added.

How to reproduce:

1. create a directory containing quotes, for example: 'INBOX.OLItest "1"'
2. list the directory using the code below:

    :::python
    import imaplib2
    import getpass
    from pprint import pprint

    server = 'imap.xxx.yyy'
    user = 'zzz'

    imapobj = imaplib2.IMAP4_SSL(server, debug=10)
    imapobj.login(user, getpass.getpass())
    res_t, data = imapobj.list("INBOX", "*OLI*")
    pprint(data)

• the ouput is:

  36:18.14 MainThread [async] LIST ('INBOX', '*OLI*')
  36:18.14 MainThread state_change_pending.acquire
  36:18.14 MainThread state_change_pending.release
  36:18.14 MainThread _request_push(ENFI3, LIST, {}) = ENFI3
  36:18.14 MainThread data=ENFI3 LIST INBOX "*OLI*"
  36:18.14 MainThread LIST:ENFI3.ready.wait
  36:18.14 imap.ir5.eu writer > ENFI3 LIST INBOX "*OLI*"\r\n
  36:18.15 imap.ir5.eu reader poll => [(3, 1)]
  36:18.15 imap.ir5.eu reader rcvd 79
  36:18.15 imap.ir5.eu reader < * LIST (\HasNoChildren) "." {17}\r\n
  36:18.15 imap.ir5.eu reader < INBOX.OLItest "1"\r\n
  36:18.15 imap.ir5.eu handler _put_response(* LIST (\HasNoChildren) "." {17})
  36:18.15 imap.ir5.eu handler read literal size 17
  36:18.15 imap.ir5.eu reader < ENFI3 OK List completed.\r\n
  36:18.15 imap.ir5.eu handler untagged_responses[LIST] 0 += ["('(\\HasNoChildren) "." {17}', 'INBOX.OLItest "1"')"]
  36:18.15 imap.ir5.eu handler _put_response()
  36:18.15 imap.ir5.eu handler untagged_responses[LIST] 1 += [""]
  36:18.15 imap.ir5.eu handler literal completed
  36:18.15 imap.ir5.eu handler _put_response(ENFI3 OK List completed.)
  36:18.15 imap.ir5.eu handler state_change_free.set
  36:18.15 imap.ir5.eu handler _request_pop(ENFI3, ('OK', ['List completed.'])) = ENFI3
  36:18.15 imap.ir5.eu handler LIST:ENFI3.ready.set
  36:18.15 MainThread _get_untagged_response(LIST) => [('(\\HasNoChildren) "." {17}', 'INBOX.OLItest "1"'), '']
  36:18.15 MainThread _untagged_response(OK, ?, LIST) => [('(\\HasNoChildren) "." {17}', 'INBOX.OLItest "1"'), '']
  [('(\\HasNoChildren) "." {17}', 'INBOX.OLItest "1"'), '']
  

With the attached patch, the empty string is removed:

33:49.21 MainThread state_change_pending.release
33:49.21 MainThread [async] LIST ('INBOX', '*OLI*')
33:49.21 MainThread state_change_pending.acquire
33:49.21 MainThread state_change_pending.release
33:49.21 MainThread _request_push(DAME3, LIST, {}) = DAME3
33:49.21 MainThread data=DAME3 LIST INBOX "*OLI*"
33:49.21 MainThread LIST:DAME3.ready.wait
33:49.21 imap.ir5.eu writer > DAME3 LIST INBOX "*OLI*"\r\n
33:49.21 imap.ir5.eu reader poll => [(3, 1)] 
33:49.21 imap.ir5.eu reader rcvd 79
33:49.21 imap.ir5.eu reader < * LIST (\HasNoChildren) "." {17}\r\n
33:49.21 imap.ir5.eu reader < INBOX.OLItest "1"\r\n
33:49.21 imap.ir5.eu handler _put_response(* LIST (\HasNoChildren) "." {17})
33:49.21 imap.ir5.eu reader < DAME3 OK List completed.\r\n
33:49.21 imap.ir5.eu handler read literal size 17
33:49.21 imap.ir5.eu handler untagged_responses[LIST] 0 += ["('(\\HasNoChildren) "." {17}', 'INBOX.OLItest "1"')"]
33:49.21 imap.ir5.eu handler _put_response()
33:49.21 imap.ir5.eu handler literal completed
33:49.21 imap.ir5.eu handler _put_response(DAME3 OK List completed.)
33:49.21 imap.ir5.eu handler state_change_free.set
33:49.21 imap.ir5.eu handler _request_pop(DAME3, ('OK', ['List completed.'])) = DAME3
33:49.21 imap.ir5.eu handler LIST:DAME3.ready.set
33:49.21 MainThread _get_untagged_response(LIST) => [('(\\HasNoChildren) "." {17}', 'INBOX.OLItest "1"')]
33:49.21 MainThread _untagged_response(OK, ?, LIST) => [('(\\HasNoChildren) "." {17}', 'INBOX.OLItest "1"')]
[('(\\HasNoChildren) "." {17}', 'INBOX.OLItest "1"')]

[Piers Lauder]

Thanks for yet another bug detection! However, that change breaks other responses where all expected data has been received but there is still a tail to retrieve. Please try changing your code to be just:
if dat:
self._append_untagged(typ, dat) # Tail
and see if that also fixes your example.

[Pilou]

The updated patch is attached, it fixes my test case.

[Piers Lauder]

Excellent! Glad that worked, and thanks for another bug fix!