Work at SourceForge, help us to make it a better place! We have an immediate need for a Support Technician in our San Francisco or Denver office.

Close

#105 Can't get SB (Windows) to work with IMAP server

open
nobody
None
5
2005-12-23
2005-12-23
Greg Bullock
No

I loved SpamBayes when I had an POP3 server. Now that
I'm using an IMAP email server, I'm trying to get it to
work with that. The setup seemed to work OK, but the
filtering gives an error message and aborts.

I've just installed Python 2.4 and SpamBayes 1.0.4 on
my Windows XP system with Outlook Express 6.

The "Configuration", "Configure folders to filter" and
"Configure folders to train" steps went fine, and
SpamBayes correctly identified the available IMAP folders.

But I can't seem to get the actual filtering to work.
Here's the command line and error response:

C:\Python24\Scripts>sb_imapfilter.py -c -t -l 5
SpamBayes IMAP Filter Version 0.6 (January 2005)
and engine SpamBayes Engine Version 0.3 (January 2004).

Traceback (most recent call last):
File "C:\Python24\Scripts\sb_imapfilter.py", line
947, in ?
run()
File "C:\Python24\Scripts\sb_imapfilter.py", line
933, in run
imap_filter.Train()
File "C:\Python24\Scripts\sb_imapfilter.py", line
766, in Train
num_ham_trained = folder.Train(self.classifier, False)
File "C:\Python24\Scripts\sb_imapfilter.py", line
686, in Train
for msg in self:
File "C:\Python24\Scripts\sb_imapfilter.py", line
613, in __iter__
yield self[key]
File "C:\Python24\Scripts\sb_imapfilter.py", line
662, in __getitem__
msg.Save()
File "C:\Python24\Scripts\sb_imapfilter.py", line
545, in Save
response = imap.uid("SEARCH", "(UNDELETED HEADER %s
\"%s\")" % \ File "C:\Python24\lib\imaplib.py", line 725, in uid
typ, dat = self._simple_command(name, command, *args)
File "C:\Python24\lib\imaplib.py", line 1028, in
_simple_command
return self._command_complete(name,
self._command(name, *args))
File "C:\Python24\lib\imaplib.py", line 865, in
_command_complete
raise self.error('%s command error: %s %s' % (name,
typ, data))
imaplib.error: UID command error: BAD ['UID invalid
arguments']

Any help most appreciated.

Regards.
Greg

Discussion

  • Greg Bullock
    Greg Bullock
    2005-12-24

    Logged In: YES
    user_id=1131257

    I just switched to SpamBayes 1.1a1 and tried again. I get a
    different error message now, but I'm still just as stumped
    as before about how to solve this or even how to look into
    the error.

    Here's the command line and new error message (this time
    with some debugging information):

    C:\Python24\Scripts>sb_imapfilter.py -i 4 -t
    SpamBayes IMAP Filter Version 1.1a1 (April 2005).

    22:51.52 > CBHD1 LOGIN "*********@******.com" "********"
    22:51.57 < CBHD1 OK LOGIN completed
    22:51.57 > CBHD2 SELECT Chisato
    22:51.63 < * 0 EXISTS
    22:51.82 < * 0 RECENT
    22:51.82 < * OK [UIDVALIDITY 0] UIDs valid
    22:51.82 < * FLAGS (\Answered \Flagged \Deleted \Seen \Draft)
    22:51.82 < * OK [PERMANENTFLAGS (\Answered \Flagged
    \Deleted \Seen \Draft)]
    22:51.82 < CBHD2 OK [READ-WRITE] SELECT completed
    22:51.82 > CBHD3 UID SEARCH UNDELETED
    22:51.88 < CBHD3 OK UID completed
    22:51.88 > CBHD4 SELECT Personal
    22:51.95 < * 85 EXISTS
    22:52.13 < * 0 RECENT
    22:52.13 < * OK [UIDVALIDITY 0] UIDs valid
    22:52.13 < * FLAGS (\Answered \Flagged \Deleted \Seen \Draft)
    22:52.13 < * OK [PERMANENTFLAGS (\Answered \Flagged
    \Deleted \Seen \Draft)]
    22:52.13 < CBHD4 OK [READ-WRITE] SELECT completed
    22:52.13 > CBHD5 UID SEARCH UNDELETED
    22:52.18 < * SEARCH 131 133 143 146 147 148 158 161 162
    166 170 171 172 174 17
    7 178 179 180 186 190 191 192 194 198 199 206 211 212 214
    215 216 220 224 228 23
    2 237 238 242 265 266 267 273 278 279 287 289 290 294 295
    296 298 299 300 310 32
    3 324 325 326 328 330 335 336 337 338 340 341 342 343 344
    345 347 348 349 350 35
    1 352 353 354 356 357 358 359 361
    22:52.33 < CBHD5 OK UID completed
    22:52.33 > CBHD6 UID FETCH 131 RFC822.HEADER
    22:52.53 < * 2 FETCH (RFC822.HEADER {935}
    22:52.54 read literal size 935
    22:52.54 < UID 131)
    22:52.54 < CBHD6 OK UID completed
    Traceback (most recent call last):
    File "C:\Python24\Scripts\sb_imapfilter.py", line 1203, in ?
    run()
    File "C:\Python24\Scripts\sb_imapfilter.py", line 1181, in run
    imap_filter.Train()
    File "C:\Python24\Scripts\sb_imapfilter.py", line 945, in
    Train
    num_trained = folder.Train(self.classifier, is_spam)
    File "C:\Python24\Scripts\sb_imapfilter.py", line 829, in
    Train
    for msg in self:
    File "C:\Python24\Scripts\sb_imapfilter.py", line 740, in
    __iter__
    yield self[key]
    File "C:\Python24\Scripts\sb_imapfilter.py", line 772, in
    __getitem__
    data = self.imap_server.extract_fetch_data(response_data)
    File "C:\Python24\Scripts\sb_imapfilter.py", line 399, in
    extract_fetch_data
    msg_data = self._extract_fetch_data(msg)
    File "C:\Python24\Scripts\sb_imapfilter.py", line 372, in
    _extract_fetch_data
    raise BadIMAPResponseError("FETCH response", response)
    __main__.BadIMAPResponseError: The command 'FETCH response'
    failed to give an OK
    response.
    (' UID 131)',)

    Thanks.
    Greg

     
  • nzlinus
    nzlinus
    2006-08-16

    Logged In: YES
    user_id=1266208

    Hi Greg,
    I just came across this same problem today, too. I'm using
    1.1a2.

    I don't know if it's the particular server that we're using,
    but it's sending back data in a slightly different format to
    what the scripts are expecting (I think). I've managed to
    hack around that issue and another (separate) one. Being a
    python newbie, it might not be the most elegant solution,
    but if anyone else can offer any suggestions, I'd be
    interested to hear them.

    So, here's my workaround(in sb_imapfilter.py): I've added
    another re called
    FALSE_UID_PART_RE = re.compile(r"^[\s]*(UID) ([\d]+)\)$")

    and around line 414, I've changed the line
    if part == ')' to also try a match on the new RE, so that it
    catches the " UID xx)" data coming back without falling
    over. See below:

    def _extract_fetch_data(self, response):
    """This does the real work of extracting the data,
    for each message
    number.
    """
    # We support the following FETCH items:
    # FLAGS
    # INTERNALDATE
    # RFC822
    # UID
    # RFC822.HEADER
    # BODY.PEEK
    # All others are ignored.

    if isinstance(response, types.StringTypes):
    response = (response,)

    data = {}
    expected_literal = None
    for part in response:

    # We ignore parentheses by themselves, for
    convenience.
    if part == ')' or
    self.FALSE_UID_PART_RE.match(part):
    continue
    if expected_literal:
    # This should be a literal of a certain size.
    key, expected_size = expected_literal
    ## if len(part) != expected_size:
    ## raise BadIMAPResponseError(\ ## "FETCH response (wrong size
    literal %d != %d)" % \ ## (len(part), expected_size), response)
    data[key] = part
    expected_literal = None
    continue

    I noticed another issue while training, which has to do with
    the SEARCH commands that are being issued. Again, it seems
    like the server doesn't like the syntax being used. Rather
    than
    CNGH30 UID SEARCH (UNDELETED HEADER X-Spambayes-MailId
    "1155728489") it seems to like the following:
    CNGH30 UID SEARCH UNDELETED HEADER "X-Spambayes-MailId"
    1155728489

    So, I made the following small change around line 732:
    # search_string = "(UNDELETED HEADER %s \"%s\")" % \ # (options["Headers",
    "mailid_header_name"],
    #
    self.id.replace('\\',r'\\').replace('"',r'\"'))
    search_string =
    ["UNDELETED","HEADER",options["Headers", "mailid_header_name"],

    self.id.replace('\\',r'\\').replace('"',r'\"'),]
    response = self.imap_server.uid("SEARCH",
    *search_string)
    data = self.imap_server.check_response("search " + "
    ".join(search_string),
    response)

    I've removed the parentheses and the quote marks (which are
    added where necessary in the IMAP library), and split the
    string, so that the IMAP library will quote just one piece
    of the string at a time. And as a consequence, I've had to
    add the " ".join(search_string) so that the concatenation works.

    Remember, this is very hacky, and might cause problems
    elsewhere, but for now it seems to have resolved this issue
    for me.

    Cheers,
    Lin.

     
  • Greg Bullock
    Greg Bullock
    2006-08-16

    Logged In: YES
    user_id=1131257

    Thank you very much, Lin. I'll give that a try.

    Regards.
    Greg