Re: mbsync slow gmail All Email folder with +100K emails

[Oswald B. <osw...@gm...>]

On Mon, Apr 18, 2022 at 11:34:59AM +0200, Martin Clausen wrote:
>Looks to me like the time is spent on a gigantic number of the 4 
>operations I listed all for the All Mail folder.
>
as i expected, by far most time is spent receiving IMAP FETCH responses.  
so the network is definitely the limiting factor, and it's an expected 
effect of syncing 130k+ messages. to quote myself from a recent thread:

>> the base imap protocol that mbsync uses isn't very efficient for 
>> resyncs. a proper fix would require using the QRESYNC extension, but 
>> that's a major project (also in the TODO). as a workaround, you can 
>> do partial syncs, though it may be challenging to automate that 
>> sufficiently to make it both reliable and convenient when working 
>> with multiple clients.

but note that gmail doesn't support QRESYNC anyway. it supports 
CONDSTORE, for which i also have an TODO item, but that won't help you 
anytime soon.

so practically, you need to use --pull-new and --push, and do full syncs 
only occasionally.

note that you're syncing both "All Mail" and some labels (== virtual 
folders), which causes duplicate work. for performance, it would be best 
to exclude All Mail entirely, and sync only "views" that are currently 
relevant. you may consider the MaxMessages option as well.