#2566 Ignore bogus RFC822.SIZEs

closed-invalid
Options (155)
5
2009-07-27
2008-02-11
jidanni
No

In http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=464016
we see the case of an IMAP server producing incorrect RFC822.SIZEs. SquirrelMail should have a user-clickable option to ignore them, and thus act like getmail, lest
one's mail get truncated.

Or just ignore them in the first place.

Discussion

  • Stefan "Bebbo" Franke

    Logged In: YES
    user_id=406985
    Originator: NO

    I suggest posting this bug at the IMAP server's bug list.
    => no bogus RFC822.SIZE => no workaround necessary.

     
  • jidanni

    jidanni - 2008-02-26

    Logged In: YES
    user_id=1971011
    Originator: YES

    Yes, but Sam V., the Courier IMAP maintainer insists that he intends to give that false RFC822.SIZE and refuses to discuss it further.

    Thank God at least getmail knows better than to swallow false RFC822.SIZEs whole.

    But what about when one is on the road with only one's ISP's
    squirrelmail to use?

    Why is there no alternative than having squirrelmail believe this
    false number when there are the extra bytes, right there in the stream
    going into squirrelmail from IMAP, but because of this false number,
    they get thrown on the floor!

    There those bytes were, already there received and stored in a file,
    but due to this false number, the user has no access to them and
    indeed they get thrown away when he clicks delete on the message he
    thought he saw the whole of but didn't!

     
  • Jonathan Angliss

    Without some serious digging, the only place I can see we use the size is the display in the mailbox view. We don't use it to do any fetching that I can see with a quick skim. What appears to be the actual issue?

     
  • jidanni

    jidanni - 2008-12-27

    If 1) a message is recieved with more than 150 MIME parts, and 2) it is received via courier-imap, then the user
    will never know or be able to access the rest of the parts beyond #150!

    Do like getmail does, and ignore lies from such IMAPs.

     
  • Jonathan Angliss

    MIME parts are not the same as RFC822.SIZE. MIME parts impacts BODYSTRUCTURE calls. RFC822.SIZE impacts the reported size in the mailbox view (at least for us anyway). If courier is limiting the message to 150 MIME parts, then that'd probably be a pretty large message (mail list digest?), and would generally be best viewed in a normal client anyway. For us to parse the message would require us to fetch the entire message, and parse in memory, which can/will be quite memory hungry. As far as I know, getmail just fetches the entire message body, and does no parsing itself.

     
  • jidanni

    jidanni - 2008-12-29

    All I know is I make these tiny
    http://jidanni.org/comp/spam/spamdealer.html
    mime parts, and the combination of squirrelmail and that sneaky
    courier-imap, which tinkers with the RFC822.SIZE to fool squirrelmail
    into thinking there is only the size that 150 of them would use, means
    I would never have an inkling that I have been robbed of the knowledge
    that the remaining MIME parts have been absconded with.

    On the road many users have no choice but to use squirrel web mail.

    Anyway, my cool spam reviewing program is thwarted by this cabal.

    Anyway, here we see two of my innocent tiny parts below.

    Yes, it would be nice if that courier-imap guy would play fair, but he won't, and it shouldn't drag others along.

    ------------=_1230520001-2119-0

    Date: Mon, 29 Dec 2008 02:45:06 +0000
    From: owner@bugs.debian.org (Debian Bug Tracking System)
    Subject: Bug#510078: Acknowledgement (add a Short-Description to
    /etc/init.d/ script)
    X-Spam-Status: Yes, score=9.5 required=1.9 tests=J_DEBIAN_ACK,J_REFERENCES
    X-Spam-Languages: en
    X-Size: 4243
    X-File: Mail/probably-spam/new/1230518844.21212_0.hoffa

    ------------=_1230520001-2119-0

    Date: Mon, 29 Dec 2008 02:45:04 +0000
    From: owner@bugs.debian.org (Debian Bug Tracking System)
    Subject: Bug#510077: Acknowledgement (first try Short-Description for more
    accurate descriptions)
    X-Spam-Status: Yes, score=9.5 required=1.9 tests=J_DEBIAN_ACK,J_REFERENCES
    X-Spam-Languages: en
    X-Size: 4226
    X-File: Mail/probably-spam/new/1230518868.30186_0.hoffa

    ------------=_1230520001-2119-0

     
  • jidanni

    jidanni - 2009-07-26

    Try the test script in the above Debian bug via courier-imap, you'll see the truncation at 150:

    Bla 149 0.5 k [ message/rfc822 ] Bllla Download | View
    untitled-[150] 0 k [ message/rfc822 ] Unknown sender Download | View

    Please add a configuration option that functions/imap_messages.php will use
    to ignore lying courier-imap SIZEs.

    Maybe make it part ot
    2. Server Settings
    A. Update IMAP Settings : localhost:143 (courier)

    Maybe call it Ignore RFC.822SIZE or something.

     
  • Jonathan Angliss

    Again, we use your imap server to identify the message using BODYSTRUCTURE, if your imap server gives up after 150 parts, then there isn't anything we can do about it. getmail is doing a fetch of the entire message, SquirrelMail cannot do that without causing a MASSIVE spike in memory usage, which would be far worse for your system than the users not being able to see the 150 message parts.

    Again, this has nothing to do with sizes, but BODYSTRUCTURE. You can test it out yourself, by enabling the "info" plugin shipped with any of the latest SquirrelMail releases. Once enabled, go to your folder where you have one of these "bad" messages, and hover your mouse over it, you'll want to look at the status bar for the passed_id variable. You should see something like this:

    src/read_body?mailbox=INBOX&passed_id=1234

    You'll need to remember the mailbox, and passed_id.

    Now go to Options | IMAP Server Information. Enable check 0, and change the mailbox name to match, then enable check 9, and put:

    FETCH # (BODYSTRUCTURE)

    Change the # to be the number you got from the passed_id. You should get something back that looks like this:

    FETCH (BODYSTRUCTURE (("text" "plain" ("charset" "iso-8859-1") NIL NIL "8bit" 188 14 NIL NIL NIL)("text" "html" ("charset" "iso-8859-1") NIL NIL "8bit" 226 4 NIL NIL NIL) "alternative" ("boundary" "----=_20090708143331_89027") NIL NIL))

    In this case, 188, and 226 are the size of the two parts on this message. If you don't get something similar back, then your IMAP server is at fault. We're not going to work around a non-RFC compliant IMAP server in this case because it will be too memory intensive. We'd have to download the entire message to memory (you already said it had 150+ parts), then cycle through each part to figure out the size, body parts, and attachment information. This all has to be done IN MEMORY.

    We're not going to put a "hack" in place just for courier because they won't read the full parts correctly. It'd require an extensive amount of code just to handle a one off for just this server. More than simply putting in an option to ignore something which is not what you're reporting on anyway. SIZE is not BODYSTRUCTURE.

     
  • Jonathan Angliss

    • assigned_to: nobody --> jangliss
    • status: open --> closed-invalid
     

Log in to post a comment.

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:

JavaScript is required for this form.





No, thanks