Re: [Fetchmail-devel] [Fetchmail-users] some question and thoughts about imap and uids

[Matthias A. <mat...@gm...>]

NOTE: Followup-To set to a public mailing list (which requires
subscription for postings) - be sure to double-check where your mailer
is going to send your reply!


Am 09.10.18 um 18:44 schrieb Kamil Jońca:
> I have read imap.c code and (it is possible I have overlooked something)

Hello Kamil,

Thanks for looking into this. I think this should continue on
fetchmail-devel@, I am redirecting there and please do subscribe so you
can post.

The model that fetchmail followed so far is that it relied on the \Seen
flags to track which messages had been downloaded, and without --keep,
it would mark messages as deleted.

> 1. (imap_search)
>    sometimes fetchmail issues "SEARCH UNSEEN" and sometimes "SEARCH
>    UNSEEN UNDELETED" (it depends on "keep" flag )
>    What is use case to issue "SEARCH UNSEEN" (without undeleted?)

ad 1. What the original intention was, I don't know. Part of it was
compatibility with IMAP2 (see the revision log for dca4a906), and part
may be the assumption that without --keep, we mark messages for deletion
but possibly without an immediate EXPUNGE, so disconnects or crashes
could theoretically leave deleted messages somehow. I am Cc:'ing Sunil
Shetye who contributed this code in early 2010 if he remembers. I hope
his mail address works, haven't heard in a long time.

> commit dca4a906d60a197b09159bc8d8a16625b1790215
> Committer: Matthias Andree <mat...@gm...>
> Date:   Thu Feb 4 09:51:01 2010 +0000
>
>     IMAP SEARCH fixes & FETCH fallback by Sunil Shetye
>     
>     * The IMAP client now uses "SEARCH UNSEEN" rather than "SEARCH
> UNSEEN NOT
>       DELETED" again on IMAP2, to fix a regression in fetchmail 6.2.5
> reported by
>       Will Stringer in June 2004. (Sunil Shetye)
>     * The IMAP client now uses "SEARCH UNSEEN UNDELETED" on IMAP4 and
> IMAP4r1
>       servers (Sunil Shetye).
>     * Workaround: The IMAP client now falls back to "FETCH n:m FLAGS"
> if the server
>       does not support "SEARCH". (Sunil Shetye)
>     * The IMAP client now requests message numbers in batches of 1,000
> to avoid
>       problems if there are more than 1860 unseen messages. (Sunil Shetye)
>         Note that this wasn't security relevant because fetchmail
> would only read up
>       to the maximum buffer size and leave the remainder of the string
> unread, going
>       out of synch afterwards.
>     
>     svn path=/branches/BRANCH_6-3/; revision=5468

> 2. After fetchin we set "\Seen" flag. Will be this neccessary with uids
> tracking?

It will no longer be necessary. We might make it an option, or we might
forego it entirely.

> 3. my first idea is to keep "not seen" ranges attached to (host,folder)
> ie something like 5:6,1000:* this can easilly be mapped to
> SEARCH UNSEEN UNDELETED UID <ranges here>

Makes sense. If you find the current .fetchids format non-workable, we
can address that, too. We used to have a different layout for .fetchids
that was more efficient, and split up, however the patches were for
6.2.5, surely will no longer apply, and I haven't forward-ported them to
6.4.x beta, let alone 7.0.x alpha.  And if you find awkward things that
need fixing, but should not distract from your current work, feel free
to file issues on Gitlab even without patches.

Thanks again for looking into this!

Cheers,
Matthias