#45 Journal naming scheme leads to "File name too long" errors

[J.J.]

For IMAP subfolders with long names, including multiple subfolders with long names, the way mbsync names the journal files results in "File name too long" and "Error: cannot read sync state" failures. I encountered this with five of a user's folder structure:

The following is a sample of the file name mbsync reated:
':gmail-account_for_sync:Internal Plans and Information!Proposals and Contracts!In-Building!Simon Fraser University Sustainable Energy and Environmental Engineering (SE3P) Building_:local-account_for_sync:Internal Plans and Information!Proposals and Contracts!In-Building!Simon Fraser University Sustainable Energy and Environmental Engineering (SE3P) Building'

As an alternative (I have no idea what the performance impact would be) would it not be better to use an index file (per channel) to store those long-ass names and have the journal file name be a hash of that name referenced from the index file.

For example:
mychannel_index

...contains:
'long-ass name1'/[hash of long-ass name]
'long-ass name2'/[hash of long-ass name]

...then the journal file would be:
[hash of long-ass name].journal
[hash of long-ass name].journal
...etc...

I'm using mbsync 1.3.0 on Ubuntu 18.04.x

Thanks

[Oswald Buddenhagen]

did you try SyncState *?

[J.J.]

No. I'll give it a try later on and see what happens. That user's mailbox is over 20 GB, so I'll give it a day or two so Google doesn't flip out.

[J.J.]

Just tried with a smaller mailbox and it looks like "SyncState *" is not appropriate for my use case. I'm syncing from Gmail to a local IMAP server.

Based on the error message, "Error: store 'local-mychannel' does not support in-box sync state," I'm assuming that "SyncState *" is only meaningful if you're syncing to the filesystem.

[Oswald Buddenhagen]

right. you could work around that by using a local store with two connected channels as a proxy. you'd run --pull chan1, then chan2, and finally --push chan1.

fair enough, hashing it is then. note that this doesn't require a mapping file; i'd only put a comment with the original name into each state file itself for purely informative purposes.

[J.J.]

OK. Thank you for considering a hashing solution. I will make do with what I can till then.

[Oswald Buddenhagen]

urgh, update to the suggested workaround: you can't have the state files of two channels in one mailbox. that defeats the purpose of the proxy, which is having the sync states in the mailbox.
there is a workaround for that in turn, but it's getting a bit messy: for each mailbox, you create a "shadow" directory which contains symlinks to the new/cur/tmp directories and the .uidvalidity file, but not the .mbsyncstate, and you create a second local store referencing these shadows for the second channel.
of course, this gets even messier when new boxes can appear on the second server as well, because then the shadowing may need to go either way.

the situation is a lot easier if this is a one-shot sync (i.e., a migration): you first sync (pull) to a local store, then change the config of the remote store to the second server and delete the .mbsyncstate files, and then sync (push) to the (redirected) remote.

i wouldn't count on a particularly timely implementation of the hashing solution unless you want to contribute it yourself (i can give you some hints) or shell out a three digit figure (that's community tariff).
have you looked at alternatives to mbsync? in particular for a one-shot that might be the best option if something else works well enough.

[J.J.]

Wow. I'm a bit less adventurous than that, so I'm currently exploring gmvault for the interim. My object-oriented skills are pretty much non-existent, so I wouldn't be much help beyond possibly pseudocode.

Thanks