isync-devel — All-purpose list, actually - but RTFM first. Moderated for non-subscribers for SPAM protection.

Re: Issue with Office365

[Oswald B. <osw...@gm...>]

On Thu, Aug 15, 2024 at 08:15:34AM -0400, Laurent Michel wrote:
>Gotcha. Though, I must say I did not have a “.mbsync” folder. That
>is still mysterious to me (I compilde isync from the git repo directly,
>main branch).
>
git master since two years (and now 1.5.0) defaults to
$XDG_STATE_HOME/isync/ (usually ~/.local/state/isync/), as per man page.
you should delete this after moving to SyncState *.

i strongly recommend not to just adjust the UidValidity if isync fails
to revalidate it. this will usually happen for very small folders, so
compare the messages manually first (you can use thunderbird for that,
enabling the "receive order" column to get the uids). if there are
discrepancies, you need to adjust/delete the bad uid pairs further down
in the state file. otherwise you'll lose these mails, if you ever
propagate deletions (you currently don't).

Re: Issue with Office365

[Marton B. <ba...@gm...>]

I'm pretty sure it's Microsoft rather than your IT department. Happens to a lot of people on M365.

Best wishes,
Marton

On Thu, Aug 15, 2024 at 08:15:34AM -0400, Laurent Michel wrote:
> Gotcha. Though, I must say I did not have a “.mbsync” folder. That is still
> mysterious to me (I compilde isync from the git repo directly, main branch). No
> matter, you saved me! Thank you!
> 
> The IT department does not know what they are doing. Things break when they go
> to the next version of software from the evil empire. I know it will happen
> again. Still, very glad to have the functionality restored.
> 
> Thanks again!
> 
> All the best,
> 
> —
> Laurent
> 
> 
> On 15 Aug 2024, at 2:17, Marton Balazs wrote:
> 
>     Great!
> 
>     I suspect the whole time the old UidValidity values were kept from your ~
>     /.mbsync/ folder. Turning on SyncState resulted in mbsync now disregarding
>     those and starting with fresh values in the new .mbsyncstate files.
> 
>     That is, the next time you run into this error (and, believe me you
>     eventually will - thanks Microsoft!) you'll still need to remember to find
>     and reset the UidValidity values in the .mbsyncstate files.
> 
>     Best wishes,
>     Marton
> 
>     On Thu, 15 Aug 2024, 00:05 Laurent Michel, <ld...@th...>
>     wrote:
> 
> 
>         You are likely going to be a life savior for me. I added the SyncState
>         stanza
>         and it started synchronizing! Fingers crossed that it will do the whole
>         bit, but that is amazing.
> 
>         A .mbsyncstate(.journal | .lock | .new) set of files appeared too.
> 
>         Thank you so much, it seems like I’m out of the woods!
> 
>         —
>         Laurent
> 
>         On 14 Aug 2024, at 18:12, Marton Balazs wrote:
> 
>             ooops, forgot to cc the List :-)
> 
>             ----- Forwarded message from Marton Balazs ba...@gm... -----
> 
>             Date: Wed, 14 Aug 2024 23:10:57 +0100
>             From: Marton Balazs ba...@gm...
>             To: Laurent Michel ld...@th...
>             Subject: Re: Issue with Office365
> 
>             Do you have
>             ~/.mbsync/ ?
> 
>             My .mbsyncrc starts with
>             SyncState *
> 
>             meaning the sync state files .mbsyncstate are in the local maildir
>             folders. Without this it seems you'll have them in ~/.mbsync/ , so
>             you may need to find and edit the UidValidity parameters there
>             somewhere?
> 
>             "
>             SyncState {*|path}
> 
>             Set the location of this Channel’s synchronization state files. *
>             means that the state should be saved in a file named .mbsyncstate
>             in the near side mailbox itself; this has the advantage that you do
>             not need to handle the state file separately if you delete the
>             mailbox, but it works only with Maildir mailboxes, obviously.
>             Otherwise this is interpreted as a string to prepend to the near
>             side mailbox name to make up a complete path.
>             This option can be used outside any section for a global effect. In
>             this case the appended string is made up according to the pattern
>             :far-store:far-box_:near-store:near-box (see also FieldDelimiter
>             below).
>             (Global default: ~/.mbsync/).
>             "
>             https://isync.sourceforge.io/mbsync.html
> 
>             Best wishes,
>             Marton
> 
>             On Wed, Aug 14, 2024 at 04:11:03PM GMT, Laurent Michel wrote:
> 
>                 Thanks.
> 
>                 I tried a few things based on your email.
> 
>                 First I do not have any file anywhere going by the name
>                 “.mbsyncstate”
> 
>                 ldm@pildm:~ $ find . -name ".mbsyncstate"
>                 ldm@pildm:~ $
> 
>                 I changed my mbsyncrc as you suggested to
> 
>                 IMAPAccount work
>                 Host outlook.office365.com
>                 User …..
>                 PassCmd /home/ldm/bin/oauth2ms
>                 AuthMechs XOAUTH2
>                 TLSType IMAPS
>                 PipelineDepth 1
> 
>                 Remote storage
> 
>                 IMAPStore work-remote
>                 Account work
> 
>                 Local storage
> 
>                 MaildirStore work-local
>                 SubFolders Maildir++
>                 Inbox ~/FOO/work/Inbox
> 
>                 Channel work-inbox
>                 Far :work-remote:"INBOX"
>                 Near :work-local:INBOX
>                 Patterns *
>                 Create Near
>                 Sync None
>                 Expunge None
>                 Remove None
> 
>                 But it ended with the same UIDVALIDITY complain:
> 
>                 ldm@pildm:~ $ mbsync -aV
>                 Reading configuration file /home/ldm/.mbsyncrc
>                 Channel work-inbox
>                 Opening far side store work-remote...
>                 Resolving outlook.office365.com...
>                 Opening near side store work-local...
>                 Connecting to outlook.office365.com (52.96.109.130:993)...
>                 Connection is now encrypted
>                 Logging in...
>                 Authenticating with SASL mechanism XOAUTH2...
>                 Opening far side box INBOX...
>                 Opening near side box INBOX...
>                 Loading far side box...
>                 Loading near side box...
>                 near side: 0 messages, 0 recent
>                 far side: 1973 messages, 0 recent
>                 Error: channel work-inbox, near side box INBOX: Unable to
>                 recover from UIDVALIDITY change.
>                 Processed 1 box(es) in 1 channel(s).
> 
>                 I’m puzzled. This is what the foo folder contains
> 
>                 ldm@pildm:~ $ cd FOO/
>                 ldm@pildm:~/FOO $ ls
>                 work
>                 ldm@pildm:~/FOO $ cd work/Inbox/
>                 cur/ new/ tmp/
>                 ldm@pildm:~/FOO $ cd work/Inbox/
>                 ldm@pildm:~/FOO/work/Inbox $ ls -al
>                 total 24
>                 drwx------ 5 ldm ldm 4096 Aug 14 15:55 .
>                 drwx------ 3 ldm ldm 4096 Aug 14 15:55 ..
>                 drwx------ 2 ldm ldm 4096 Aug 14 15:55 cur
>                 drwx------ 2 ldm ldm 4096 Aug 14 15:55 new
>                 drwx------ 2 ldm ldm 4096 Aug 14 15:55 tmp
>                 -rw------- 1 ldm ldm 13 Aug 14 15:55 .uidvalidity
>                 ldm@pildm:~/FOO/work/Inbox $ cat .uidvalidity
>                 1723665330
>                 0
> 
>                 On 14 Aug 2024, at 15:45, Marton Balazs wrote:
> 
>                 As far as I understand, every now and then MS screws up UidValidity, which
>                 it really shouldn't do -- but it's MS. On these occasions Mbsync usually
>                 recovers most of my folders from this but on a few I get the error you
>                 mentioned. I then create an empty directory and pull all of my folders from
>                 MS to there using
> 
>                 Patterns *
>                 Create Near
>                 Sync None
>                 Expunge None
>                 Remove None
> 
>                 This creates the folders and the
>                 .mbsyncstate
>                 .uidvalidity
> 
>                 files in them but won't download the actual emails so it runs fast. I don't
>                 know why you never mentioned .mbsyncstate? This is the one I need to look
>                 at: if its first line like
> 
>                 FarUidValidity 14
> 
>                 differs between the pulled empty folders and my existing local email
>                 folder, that's when I get the errors. So, I manually edit this parameter in
>                 my local emails folder to match the one I just pulled from MS and magically
>                 everything works ok again.
> 
>                 I'm not sure you have the same problem but it would be worth looking at the
>                 .mbsyncstate files and the FarUidValidity parameter in them.
> 
>                 Best wishes,
>                 Marton
> 
>                 On Wed, Aug 14, 2024 at 02:50:19PM GMT, Laurent Michel wrote:
> 
>                     Dear All,
> 
>                     I had isync running flawlessly for a few years with OAUTH2. Recently,
>                     it seems
>                     like my organization must have done an “upgrade” to Office365. And now,
>                     everything is broken on my end.
> 
>                     Specifically, each sync end as so:
> 
>                     ldm@pildm:~ $ mbsync -aV
>                     Reading configuration file /home/ldm/.mbsyncrc
>                     Channel work-inbox
>                     Opening far side store work-remote...
>                     Resolving outlook.office365.com...
>                     Opening near side store work-local...
>                     Connecting to outlook.office365.com (52.96.97.162:993)...
>                     Connection is now encrypted
>                     Logging in...
>                     Authenticating with SASL mechanism XOAUTH2...
>                     Opening far side box INBOX...
>                     Opening near side box INBOX...
>                     Loading far side box...
>                     Loading near side box...
>                     near side: 0 messages, 0 recent
>                     far side: 1971 messages, 0 recent
>                     Error: channel work-inbox, near side box INBOX: Unable to recover from
>                     UIDVALIDITY change.
>                     Processed 1 box(es) in 1 channel(s),
>                     pulled 0 new message(s) and 0 flag update(s),
>                     expunged 0 message(s) from near side,
>                     pushed 0 new message(s) and 0 flag update(s),
>                     expunged 0 message(s) from far side.
> 
>                     As you can see, it’s not an authentication issue but an UIDVALIDITY
>                     issue. I
>                     looked on the web and tried pretty much every single thing I found,
>                     including
>                     deleting my local copy in its entirety:
> 
>                     rm -rf ~/Mail
> 
>                     And let it redownload everything. The .uidvalidity file(s) are held
>                     within the
>                     ~/Mail
> 
>                     and rerunning the command above after atomizing the folder yields this
>                     content
> 
>                     ldm@pildm:~ $ cd Mail/
>                     ldm@pildm:~/Mail $ tree
>                     .
>                     └── work
>                     └── In
>                     ├── cur
>                     ├── new
>                     └── tmp
> 
>                     6 directories, 0 files
>                     ldm@pildm:~/Mail $ ls -al
>                     total 12
>                     drwx------ 3 ldm ldm 4096 Aug 14 14:44 .
>                     drwxr-xr-x 57 ldm users 4096 Aug 14 14:44 ..
>                     drwx------ 3 ldm ldm 4096 Aug 14 14:44 work
>                     ldm@pildm:~/Mail $ ls -alR
>                     .:
>                     total 12
>                     drwx------ 3 ldm ldm 4096 Aug 14 14:44 .
>                     drwxr-xr-x 57 ldm users 4096 Aug 14 14:44 ..
>                     drwx------ 3 ldm ldm 4096 Aug 14 14:44 work
> 
>                     ./work:
>                     total 12
>                     drwx------ 3 ldm ldm 4096 Aug 14 14:44 .
>                     drwx------ 3 ldm ldm 4096 Aug 14 14:44 ..
>                     drwx------ 5 ldm ldm 4096 Aug 14 14:44 In
> 
>                     ./work/In:
>                     total 24
>                     drwx------ 5 ldm ldm 4096 Aug 14 14:44 .
>                     drwx------ 3 ldm ldm 4096 Aug 14 14:44 ..
>                     drwx------ 2 ldm ldm 4096 Aug 14 14:44 cur
>                     drwx------ 2 ldm ldm 4096 Aug 14 14:44 new
>                     drwx------ 2 ldm ldm 4096 Aug 14 14:44 tmp
>                     -rw------- 1 ldm ldm 13 Aug 14 14:44 .uidvalidity
> 
>                     ./work/In/cur:
>                     total 8
>                     drwx------ 2 ldm ldm 4096 Aug 14 14:44 .
>                     drwx------ 5 ldm ldm 4096 Aug 14 14:44 ..
> 
>                     ./work/In/new:
>                     total 8
>                     drwx------ 2 ldm ldm 4096 Aug 14 14:44 .
>                     drwx------ 5 ldm ldm 4096 Aug 14 14:44 ..
> 
>                     ./work/In/tmp:
>                     total 8
>                     drwx------ 2 ldm ldm 4096 Aug 14 14:44 .
>                     drwx------ 5 ldm ldm 4096 Aug 14 14:44 ..
> 
>                     I even have renamed my inbox from INBOX to “In” in fear that a hidden
>                     file was
>                     lurking somewhere. But no dice.
> 
>                     running
> 
>                     mbsync -aVD
> 
>                     produces a huge output. Only showing the very end below
> 
>                     uid=1051965 flags=S size=0 tuid=?
>                     uid=1052014 flags=S size=0 tuid=?
>                     uid=1052020 flags=S size=0 tuid=?
>                     uid=1052026 flags=S size=0 tuid=?
>                     uid=1052032 flags=RS size=0 tuid=?
>                     uid=1052050 flags=RS size=0 tuid=?
>                     uid=1052062 flags=S size=0 tuid=?
>                     uid=1052073 flags=S size=0 tuid=?
>                     uid=1052079 flags=S size=0 tuid=?
>                     uid=1052085 flags=S size=0 tuid=?
>                     uid=1052099 flags=S size=0 tuid=?
>                     uid=1052104 flags=S size=0 tuid=?
>                     uid=1052112 flags=S size=0 tuid=?
>                     far side: 1971 messages, 0 recent
>                     matching messages on far side against sync records
>                     trying to re-approve uid validity of near side
>                     Error: channel work-inbox, near side box INBOX: Unable to recover from
>                     UIDVALIDITY change.
>                     F: [ 7] Enter cancel_cmds
>                     F: [ 7] Callback enter cancel_cmds
>                     F: [ 7] Callback leave cancel_cmds
>                     F: [ 7] Leave cancel_cmds
>                     N: [ 8] Enter cancel_cmds
>                     N: [ 8] Callback enter cancel_cmds
>                     F: Enter free_store
>                     F: Leave free_store
>                     N: Enter free_store
>                     N: Leave free_store
>                     F: >>> 8 LOGOUT
>                     N: [ 8] Callback leave cancel_cmds
>                     N: [ 8] Leave cancel_cmds
>                     F: [ 5] Callback leave load_box
>                     F: * BYE Microsoft Exchange Server IMAP4 server signing off.
>                     F: 8 OK LOGOUT completed.
>                     Processed 1 box(es) in 1 channel(s),
>                     pulled 0 new message(s) and 0 flag update(s),
>                     expunged 0 message(s) from near side,
>                     pushed 0 new message(s) and 0 flag update(s),
>                     expunged 0 message(s) from far side.
>                     ldm@pildm:~/Mail $
> 
>                     There are 1971 messages in my inbox, so that value we see above is
>                     correct. I
>                     have no idea why this is failing. Any help would be much appreciated.
> 
>                     My .mbsyncrc is pretty vanilla (just removing my address)
> 
>                     ldm@pildm:~ $ cat .mbsyncrc
>                     #################################
>                     ######## Account work ###########
>                     #################################
> 
>                     IMAPAccount work
>                     Host outlook.office365.com
>                     User xx...@yy...
>                     PassCmd /home/ldm/bin/oauth2ms
>                     AuthMechs XOAUTH2
>                     TLSType IMAPS
>                     PipelineDepth 1
> 
>                     Remote storage
> 
>                     IMAPStore work-remote
>                     Account work
> 
>                     Local storage
> 
>                     MaildirStore work-local
>                     SubFolders Maildir++
>                     Inbox ~/Mail/work/In
> 
>                     Channel work-inbox
>                     Far :work-remote:"INBOX"
>                     Near :work-local:INBOX
>                     Create Both
>                     Sync Pull Push New Flags
>                     Expunge Both
> 
>                     Group work
>                     Channel work-inbox
> 
>                     The only .uidvalidity file I have is created in that tree and contains
>                     the
>                     following
> 
>                     ldm@pildm:~ $ cat Mail/work/In/.uidvalidity
>                     1723661072
>                     0
> 
>                     Help?
> 
>                     —
>                     Laurent
> 
>                     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
> 
>                     isync-devel mailing list
>                     isy...@li...
>                   https://lists.sourceforge.net/lists/listinfo/isync-devel
> 
>             ----- End forwarded message -----
> 
>             ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
>            
>             isync-devel mailing list
>             isy...@li...
>             https://lists.sourceforge.net/lists/listinfo/isync-devel
> 

Re: Issue with Office365

[Laurent M. <ld...@th...>]

Gotcha. Though, I must say I did not have a “.mbsync” folder. That 
is still mysterious to me (I compilde isync from the git repo directly, 
main branch). No matter, you saved me! Thank you!

The IT department does not know what they are doing. Things break when 
they go to the next version of software from the evil empire. I know it 
will happen again. Still, very glad to have the functionality restored.

Thanks again!

All the best,

—
  Laurent


On 15 Aug 2024, at 2:17, Marton Balazs wrote:

> Great!
>
> I suspect the whole time the old UidValidity values were kept from
> your ~/.mbsync/
> folder. Turning on SyncState resulted in mbsync now disregarding those 
> and
> starting with fresh values in the new .mbsyncstate files.
>
> That is, the next time you run into this error (and, believe me you
> eventually will - thanks Microsoft!) you'll still need to remember to 
> find
> and reset the UidValidity values in the .mbsyncstate files.
>
> Best wishes,
> Marton
>
> On Thu, 15 Aug 2024, 00:05 Laurent Michel, <ld...@th...>
> wrote:
>
>> You are likely going to be a life savior for me. I added the 
>> SyncState
>> stanza
>> and it started synchronizing! Fingers crossed that it will do the 
>> whole
>> bit, but that is amazing.
>>
>> A .mbsyncstate(.journal | .lock | .new) set of files appeared too.
>>
>> Thank you so much, it seems like I’m out of the woods!
>>
>> —
>> Laurent
>>
>> On 14 Aug 2024, at 18:12, Marton Balazs wrote:
>>
>> ooops, forgot to cc the List :-)
>>
>> ----- Forwarded message from Marton Balazs ba...@gm... -----
>>
>> Date: Wed, 14 Aug 2024 23:10:57 +0100
>> From: Marton Balazs ba...@gm...
>> To: Laurent Michel ld...@th...
>> Subject: Re: Issue with Office365
>>
>> Do you have
>> ~/.mbsync/ ?
>>
>> My .mbsyncrc starts with
>> SyncState *
>>
>> meaning the sync state files .mbsyncstate are in the local maildir
>> folders. Without this it seems you'll have them in ~/.mbsync/ , so 
>> you may
>> need to find and edit the UidValidity parameters there somewhere?
>>
>> "
>> SyncState {*|path}
>>
>> Set the location of this Channel’s synchronization state files. * 
>> means
>> that the state should be saved in a file named .mbsyncstate in the 
>> near
>> side mailbox itself; this has the advantage that you do not need to 
>> handle
>> the state file separately if you delete the mailbox, but it works 
>> only with
>> Maildir mailboxes, obviously. Otherwise this is interpreted as a 
>> string to
>> prepend to the near side mailbox name to make up a complete path.
>> This option can be used outside any section for a global effect. In 
>> this
>> case the appended string is made up according to the pattern
>> :far-store:far-box_:near-store:near-box (see also FieldDelimiter 
>> below).
>> (Global default: ~/.mbsync/).
>> "
>> https://isync.sourceforge.io/mbsync.html
>>
>> Best wishes,
>> Marton
>>
>> On Wed, Aug 14, 2024 at 04:11:03PM GMT, Laurent Michel wrote:
>>
>> Thanks.
>>
>> I tried a few things based on your email.
>>
>> First I do not have any file anywhere going by the name 
>> “.mbsyncstate”
>>
>> ldm@pildm:~ $ find . -name ".mbsyncstate"
>> ldm@pildm:~ $
>>
>> I changed my mbsyncrc as you suggested to
>>
>> IMAPAccount work
>> Host outlook.office365.com
>> User …..
>> PassCmd /home/ldm/bin/oauth2ms
>> AuthMechs XOAUTH2
>> TLSType IMAPS
>> PipelineDepth 1
>> Remote storage
>>
>> IMAPStore work-remote
>> Account work
>> Local storage
>>
>> MaildirStore work-local
>> SubFolders Maildir++
>> Inbox ~/FOO/work/Inbox
>>
>> Channel work-inbox
>> Far :work-remote:"INBOX"
>> Near :work-local:INBOX
>> Patterns *
>> Create Near
>> Sync None
>> Expunge None
>> Remove None
>>
>> But it ended with the same UIDVALIDITY complain:
>>
>> ldm@pildm:~ $ mbsync -aV
>> Reading configuration file /home/ldm/.mbsyncrc
>> Channel work-inbox
>> Opening far side store work-remote...
>> Resolving outlook.office365.com...
>> Opening near side store work-local...
>> Connecting to outlook.office365.com (52.96.109.130:993)...
>> Connection is now encrypted
>> Logging in...
>> Authenticating with SASL mechanism XOAUTH2...
>> Opening far side box INBOX...
>> Opening near side box INBOX...
>> Loading far side box...
>> Loading near side box...
>> near side: 0 messages, 0 recent
>> far side: 1973 messages, 0 recent
>> Error: channel work-inbox, near side box INBOX: Unable to recover 
>> from
>> UIDVALIDITY change.
>> Processed 1 box(es) in 1 channel(s).
>>
>> I’m puzzled. This is what the foo folder contains
>>
>> ldm@pildm:~ $ cd FOO/
>> ldm@pildm:~/FOO $ ls
>> work
>> ldm@pildm:~/FOO $ cd work/Inbox/
>> cur/ new/ tmp/
>> ldm@pildm:~/FOO $ cd work/Inbox/
>> ldm@pildm:~/FOO/work/Inbox $ ls -al
>> total 24
>> drwx------ 5 ldm ldm 4096 Aug 14 15:55 .
>> drwx------ 3 ldm ldm 4096 Aug 14 15:55 ..
>> drwx------ 2 ldm ldm 4096 Aug 14 15:55 cur
>> drwx------ 2 ldm ldm 4096 Aug 14 15:55 new
>> drwx------ 2 ldm ldm 4096 Aug 14 15:55 tmp
>> -rw------- 1 ldm ldm 13 Aug 14 15:55 .uidvalidity
>> ldm@pildm:~/FOO/work/Inbox $ cat .uidvalidity
>> 1723665330
>> 0
>>
>> On 14 Aug 2024, at 15:45, Marton Balazs wrote:
>>
>> As far as I understand, every now and then MS screws up UidValidity, 
>> which
>> it really shouldn't do -- but it's MS. On these occasions Mbsync 
>> usually
>> recovers most of my folders from this but on a few I get the error 
>> you
>> mentioned. I then create an empty directory and pull all of my 
>> folders from
>> MS to there using
>>
>> Patterns *
>> Create Near
>> Sync None
>> Expunge None
>> Remove None
>>
>> This creates the folders and the
>> .mbsyncstate
>> .uidvalidity
>>
>> files in them but won't download the actual emails so it runs fast. I 
>> don't
>> know why you never mentioned .mbsyncstate? This is the one I need to 
>> look
>> at: if its first line like
>>
>> FarUidValidity 14
>>
>> differs between the pulled empty folders and my existing local email
>> folder, that's when I get the errors. So, I manually edit this 
>> parameter in
>> my local emails folder to match the one I just pulled from MS and 
>> magically
>> everything works ok again.
>>
>> I'm not sure you have the same problem but it would be worth looking 
>> at the
>> .mbsyncstate files and the FarUidValidity parameter in them.
>>
>> Best wishes,
>> Marton
>>
>> On Wed, Aug 14, 2024 at 02:50:19PM GMT, Laurent Michel wrote:
>>
>>     Dear All,
>>
>>     I had isync running flawlessly for a few years with OAUTH2. 
>> Recently,
>>     it seems
>>     like my organization must have done an “upgrade” to 
>> Office365. And now,
>>     everything is broken on my end.
>>
>>     Specifically, each sync end as so:
>>
>>     ldm@pildm:~ $ mbsync -aV
>>     Reading configuration file /home/ldm/.mbsyncrc
>>     Channel work-inbox
>>     Opening far side store work-remote...
>>     Resolving outlook.office365.com...
>>     Opening near side store work-local...
>>     Connecting to outlook.office365.com (52.96.97.162:993)...
>>     Connection is now encrypted
>>     Logging in...
>>     Authenticating with SASL mechanism XOAUTH2...
>>     Opening far side box INBOX...
>>     Opening near side box INBOX...
>>     Loading far side box...
>>     Loading near side box...
>>     near side: 0 messages, 0 recent
>>     far side: 1971 messages, 0 recent
>>     Error: channel work-inbox, near side box INBOX: Unable to recover 
>> from
>>     UIDVALIDITY change.
>>     Processed 1 box(es) in 1 channel(s),
>>     pulled 0 new message(s) and 0 flag update(s),
>>     expunged 0 message(s) from near side,
>>     pushed 0 new message(s) and 0 flag update(s),
>>     expunged 0 message(s) from far side.
>>
>>     As you can see, it’s not an authentication issue but an 
>> UIDVALIDITY
>>     issue. I
>>     looked on the web and tried pretty much every single thing I 
>> found,
>>     including
>>     deleting my local copy in its entirety:
>>
>>     rm -rf ~/Mail
>>
>>     And let it redownload everything. The .uidvalidity file(s) are 
>> held
>>     within the
>>     ~/Mail
>>
>>     and rerunning the command above after atomizing the folder yields 
>> this
>>     content
>>
>>     ldm@pildm:~ $ cd Mail/
>>     ldm@pildm:~/Mail $ tree
>>     .
>>     └── work
>>     └── In
>>     ├── cur
>>     ├── new
>>     └── tmp
>>
>>     6 directories, 0 files
>>     ldm@pildm:~/Mail $ ls -al
>>     total 12
>>     drwx------ 3 ldm ldm 4096 Aug 14 14:44 .
>>     drwxr-xr-x 57 ldm users 4096 Aug 14 14:44 ..
>>     drwx------ 3 ldm ldm 4096 Aug 14 14:44 work
>>     ldm@pildm:~/Mail $ ls -alR
>>     .:
>>     total 12
>>     drwx------ 3 ldm ldm 4096 Aug 14 14:44 .
>>     drwxr-xr-x 57 ldm users 4096 Aug 14 14:44 ..
>>     drwx------ 3 ldm ldm 4096 Aug 14 14:44 work
>>
>>     ./work:
>>     total 12
>>     drwx------ 3 ldm ldm 4096 Aug 14 14:44 .
>>     drwx------ 3 ldm ldm 4096 Aug 14 14:44 ..
>>     drwx------ 5 ldm ldm 4096 Aug 14 14:44 In
>>
>>     ./work/In:
>>     total 24
>>     drwx------ 5 ldm ldm 4096 Aug 14 14:44 .
>>     drwx------ 3 ldm ldm 4096 Aug 14 14:44 ..
>>     drwx------ 2 ldm ldm 4096 Aug 14 14:44 cur
>>     drwx------ 2 ldm ldm 4096 Aug 14 14:44 new
>>     drwx------ 2 ldm ldm 4096 Aug 14 14:44 tmp
>>     -rw------- 1 ldm ldm 13 Aug 14 14:44 .uidvalidity
>>
>>     ./work/In/cur:
>>     total 8
>>     drwx------ 2 ldm ldm 4096 Aug 14 14:44 .
>>     drwx------ 5 ldm ldm 4096 Aug 14 14:44 ..
>>
>>     ./work/In/new:
>>     total 8
>>     drwx------ 2 ldm ldm 4096 Aug 14 14:44 .
>>     drwx------ 5 ldm ldm 4096 Aug 14 14:44 ..
>>
>>     ./work/In/tmp:
>>     total 8
>>     drwx------ 2 ldm ldm 4096 Aug 14 14:44 .
>>     drwx------ 5 ldm ldm 4096 Aug 14 14:44 ..
>>
>>     I even have renamed my inbox from INBOX to “In” in fear that 
>> a hidden
>>     file was
>>     lurking somewhere. But no dice.
>>
>>     running
>>
>>     mbsync -aVD
>>
>>     produces a huge output. Only showing the very end below
>>
>>     uid=1051965 flags=S size=0 tuid=?
>>     uid=1052014 flags=S size=0 tuid=?
>>     uid=1052020 flags=S size=0 tuid=?
>>     uid=1052026 flags=S size=0 tuid=?
>>     uid=1052032 flags=RS size=0 tuid=?
>>     uid=1052050 flags=RS size=0 tuid=?
>>     uid=1052062 flags=S size=0 tuid=?
>>     uid=1052073 flags=S size=0 tuid=?
>>     uid=1052079 flags=S size=0 tuid=?
>>     uid=1052085 flags=S size=0 tuid=?
>>     uid=1052099 flags=S size=0 tuid=?
>>     uid=1052104 flags=S size=0 tuid=?
>>     uid=1052112 flags=S size=0 tuid=?
>>     far side: 1971 messages, 0 recent
>>     matching messages on far side against sync records
>>     trying to re-approve uid validity of near side
>>     Error: channel work-inbox, near side box INBOX: Unable to recover 
>> from
>>     UIDVALIDITY change.
>>     F: [ 7] Enter cancel_cmds
>>     F: [ 7] Callback enter cancel_cmds
>>     F: [ 7] Callback leave cancel_cmds
>>     F: [ 7] Leave cancel_cmds
>>     N: [ 8] Enter cancel_cmds
>>     N: [ 8] Callback enter cancel_cmds
>>     F: Enter free_store
>>     F: Leave free_store
>>     N: Enter free_store
>>     N: Leave free_store
>>     F: >>> 8 LOGOUT
>>     N: [ 8] Callback leave cancel_cmds
>>     N: [ 8] Leave cancel_cmds
>>     F: [ 5] Callback leave load_box
>>     F: * BYE Microsoft Exchange Server IMAP4 server signing off.
>>     F: 8 OK LOGOUT completed.
>>     Processed 1 box(es) in 1 channel(s),
>>     pulled 0 new message(s) and 0 flag update(s),
>>     expunged 0 message(s) from near side,
>>     pushed 0 new message(s) and 0 flag update(s),
>>     expunged 0 message(s) from far side.
>>     ldm@pildm:~/Mail $
>>
>>     There are 1971 messages in my inbox, so that value we see above 
>> is
>>     correct. I
>>     have no idea why this is failing. Any help would be much 
>> appreciated.
>>
>>     My .mbsyncrc is pretty vanilla (just removing my address)
>>
>>     ldm@pildm:~ $ cat .mbsyncrc
>>     #################################
>>     ######## Account work ###########
>>     #################################
>>
>>     IMAPAccount work
>>     Host outlook.office365.com
>>     User xx...@yy...
>>     PassCmd /home/ldm/bin/oauth2ms
>>     AuthMechs XOAUTH2
>>     TLSType IMAPS
>>     PipelineDepth 1
>>
>>     Remote storage
>>
>>     IMAPStore work-remote
>>     Account work
>>
>>     Local storage
>>
>>     MaildirStore work-local
>>     SubFolders Maildir++
>>     Inbox ~/Mail/work/In
>>
>>     Channel work-inbox
>>     Far :work-remote:"INBOX"
>>     Near :work-local:INBOX
>>     Create Both
>>     Sync Pull Push New Flags
>>     Expunge Both
>>
>>     Group work
>>     Channel work-inbox
>>
>>     The only .uidvalidity file I have is created in that tree and 
>> contains
>>     the
>>     following
>>
>>     ldm@pildm:~ $ cat Mail/work/In/.uidvalidity
>>     1723661072
>>     0
>>
>>     Help?
>>
>>     —
>>     Laurent
>>
>>     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
>>
>>     isync-devel mailing list
>>     isy...@li...
>> https://lists.sourceforge.net/lists/listinfo/isync-devel
>>
>> ----- End forwarded message -----
>> ------------------------------
>>
>> isync-devel mailing list
>> isy...@li...
>> https://lists.sourceforge.net/lists/listinfo/isync-devel
>>
>>

Re: Issue with Office365

[Marton B. <ba...@gm...>]

Great!

I suspect the whole time the old UidValidity values were kept from
your ~/.mbsync/
folder. Turning on SyncState resulted in mbsync now disregarding those and
starting with fresh values in the new .mbsyncstate files.

That is, the next time you run into this error (and, believe me you
eventually will - thanks Microsoft!) you'll still need to remember to find
and reset the UidValidity values in the .mbsyncstate files.

Best wishes,
Marton

On Thu, 15 Aug 2024, 00:05 Laurent Michel, <ld...@th...>
wrote:

> You are likely going to be a life savior for me. I added the SyncState
> stanza
> and it started synchronizing! Fingers crossed that it will do the whole
> bit, but that is amazing.
>
> A .mbsyncstate(.journal | .lock | .new) set of files appeared too.
>
> Thank you so much, it seems like I’m out of the woods!
>
> —
> Laurent
>
> On 14 Aug 2024, at 18:12, Marton Balazs wrote:
>
> ooops, forgot to cc the List :-)
>
> ----- Forwarded message from Marton Balazs ba...@gm... -----
>
> Date: Wed, 14 Aug 2024 23:10:57 +0100
> From: Marton Balazs ba...@gm...
> To: Laurent Michel ld...@th...
> Subject: Re: Issue with Office365
>
> Do you have
> ~/.mbsync/ ?
>
> My .mbsyncrc starts with
> SyncState *
>
> meaning the sync state files .mbsyncstate are in the local maildir
> folders. Without this it seems you'll have them in ~/.mbsync/ , so you may
> need to find and edit the UidValidity parameters there somewhere?
>
> "
> SyncState {*|path}
>
> Set the location of this Channel’s synchronization state files. * means
> that the state should be saved in a file named .mbsyncstate in the near
> side mailbox itself; this has the advantage that you do not need to handle
> the state file separately if you delete the mailbox, but it works only with
> Maildir mailboxes, obviously. Otherwise this is interpreted as a string to
> prepend to the near side mailbox name to make up a complete path.
> This option can be used outside any section for a global effect. In this
> case the appended string is made up according to the pattern
> :far-store:far-box_:near-store:near-box (see also FieldDelimiter below).
> (Global default: ~/.mbsync/).
> "
> https://isync.sourceforge.io/mbsync.html
>
> Best wishes,
> Marton
>
> On Wed, Aug 14, 2024 at 04:11:03PM GMT, Laurent Michel wrote:
>
> Thanks.
>
> I tried a few things based on your email.
>
> First I do not have any file anywhere going by the name “.mbsyncstate”
>
> ldm@pildm:~ $ find . -name ".mbsyncstate"
> ldm@pildm:~ $
>
> I changed my mbsyncrc as you suggested to
>
> IMAPAccount work
> Host outlook.office365.com
> User …..
> PassCmd /home/ldm/bin/oauth2ms
> AuthMechs XOAUTH2
> TLSType IMAPS
> PipelineDepth 1
> Remote storage
>
> IMAPStore work-remote
> Account work
> Local storage
>
> MaildirStore work-local
> SubFolders Maildir++
> Inbox ~/FOO/work/Inbox
>
> Channel work-inbox
> Far :work-remote:"INBOX"
> Near :work-local:INBOX
> Patterns *
> Create Near
> Sync None
> Expunge None
> Remove None
>
> But it ended with the same UIDVALIDITY complain:
>
> ldm@pildm:~ $ mbsync -aV
> Reading configuration file /home/ldm/.mbsyncrc
> Channel work-inbox
> Opening far side store work-remote...
> Resolving outlook.office365.com...
> Opening near side store work-local...
> Connecting to outlook.office365.com (52.96.109.130:993)...
> Connection is now encrypted
> Logging in...
> Authenticating with SASL mechanism XOAUTH2...
> Opening far side box INBOX...
> Opening near side box INBOX...
> Loading far side box...
> Loading near side box...
> near side: 0 messages, 0 recent
> far side: 1973 messages, 0 recent
> Error: channel work-inbox, near side box INBOX: Unable to recover from
> UIDVALIDITY change.
> Processed 1 box(es) in 1 channel(s).
>
> I’m puzzled. This is what the foo folder contains
>
> ldm@pildm:~ $ cd FOO/
> ldm@pildm:~/FOO $ ls
> work
> ldm@pildm:~/FOO $ cd work/Inbox/
> cur/ new/ tmp/
> ldm@pildm:~/FOO $ cd work/Inbox/
> ldm@pildm:~/FOO/work/Inbox $ ls -al
> total 24
> drwx------ 5 ldm ldm 4096 Aug 14 15:55 .
> drwx------ 3 ldm ldm 4096 Aug 14 15:55 ..
> drwx------ 2 ldm ldm 4096 Aug 14 15:55 cur
> drwx------ 2 ldm ldm 4096 Aug 14 15:55 new
> drwx------ 2 ldm ldm 4096 Aug 14 15:55 tmp
> -rw------- 1 ldm ldm 13 Aug 14 15:55 .uidvalidity
> ldm@pildm:~/FOO/work/Inbox $ cat .uidvalidity
> 1723665330
> 0
>
> On 14 Aug 2024, at 15:45, Marton Balazs wrote:
>
> As far as I understand, every now and then MS screws up UidValidity, which
> it really shouldn't do -- but it's MS. On these occasions Mbsync usually
> recovers most of my folders from this but on a few I get the error you
> mentioned. I then create an empty directory and pull all of my folders from
> MS to there using
>
> Patterns *
> Create Near
> Sync None
> Expunge None
> Remove None
>
> This creates the folders and the
> .mbsyncstate
> .uidvalidity
>
> files in them but won't download the actual emails so it runs fast. I don't
> know why you never mentioned .mbsyncstate? This is the one I need to look
> at: if its first line like
>
> FarUidValidity 14
>
> differs between the pulled empty folders and my existing local email
> folder, that's when I get the errors. So, I manually edit this parameter in
> my local emails folder to match the one I just pulled from MS and magically
> everything works ok again.
>
> I'm not sure you have the same problem but it would be worth looking at the
> .mbsyncstate files and the FarUidValidity parameter in them.
>
> Best wishes,
> Marton
>
> On Wed, Aug 14, 2024 at 02:50:19PM GMT, Laurent Michel wrote:
>
>     Dear All,
>
>     I had isync running flawlessly for a few years with OAUTH2. Recently,
>     it seems
>     like my organization must have done an “upgrade” to Office365. And now,
>     everything is broken on my end.
>
>     Specifically, each sync end as so:
>
>     ldm@pildm:~ $ mbsync -aV
>     Reading configuration file /home/ldm/.mbsyncrc
>     Channel work-inbox
>     Opening far side store work-remote...
>     Resolving outlook.office365.com...
>     Opening near side store work-local...
>     Connecting to outlook.office365.com (52.96.97.162:993)...
>     Connection is now encrypted
>     Logging in...
>     Authenticating with SASL mechanism XOAUTH2...
>     Opening far side box INBOX...
>     Opening near side box INBOX...
>     Loading far side box...
>     Loading near side box...
>     near side: 0 messages, 0 recent
>     far side: 1971 messages, 0 recent
>     Error: channel work-inbox, near side box INBOX: Unable to recover from
>     UIDVALIDITY change.
>     Processed 1 box(es) in 1 channel(s),
>     pulled 0 new message(s) and 0 flag update(s),
>     expunged 0 message(s) from near side,
>     pushed 0 new message(s) and 0 flag update(s),
>     expunged 0 message(s) from far side.
>
>     As you can see, it’s not an authentication issue but an UIDVALIDITY
>     issue. I
>     looked on the web and tried pretty much every single thing I found,
>     including
>     deleting my local copy in its entirety:
>
>     rm -rf ~/Mail
>
>     And let it redownload everything. The .uidvalidity file(s) are held
>     within the
>     ~/Mail
>
>     and rerunning the command above after atomizing the folder yields this
>     content
>
>     ldm@pildm:~ $ cd Mail/
>     ldm@pildm:~/Mail $ tree
>     .
>     └── work
>     └── In
>     ├── cur
>     ├── new
>     └── tmp
>
>     6 directories, 0 files
>     ldm@pildm:~/Mail $ ls -al
>     total 12
>     drwx------ 3 ldm ldm 4096 Aug 14 14:44 .
>     drwxr-xr-x 57 ldm users 4096 Aug 14 14:44 ..
>     drwx------ 3 ldm ldm 4096 Aug 14 14:44 work
>     ldm@pildm:~/Mail $ ls -alR
>     .:
>     total 12
>     drwx------ 3 ldm ldm 4096 Aug 14 14:44 .
>     drwxr-xr-x 57 ldm users 4096 Aug 14 14:44 ..
>     drwx------ 3 ldm ldm 4096 Aug 14 14:44 work
>
>     ./work:
>     total 12
>     drwx------ 3 ldm ldm 4096 Aug 14 14:44 .
>     drwx------ 3 ldm ldm 4096 Aug 14 14:44 ..
>     drwx------ 5 ldm ldm 4096 Aug 14 14:44 In
>
>     ./work/In:
>     total 24
>     drwx------ 5 ldm ldm 4096 Aug 14 14:44 .
>     drwx------ 3 ldm ldm 4096 Aug 14 14:44 ..
>     drwx------ 2 ldm ldm 4096 Aug 14 14:44 cur
>     drwx------ 2 ldm ldm 4096 Aug 14 14:44 new
>     drwx------ 2 ldm ldm 4096 Aug 14 14:44 tmp
>     -rw------- 1 ldm ldm 13 Aug 14 14:44 .uidvalidity
>
>     ./work/In/cur:
>     total 8
>     drwx------ 2 ldm ldm 4096 Aug 14 14:44 .
>     drwx------ 5 ldm ldm 4096 Aug 14 14:44 ..
>
>     ./work/In/new:
>     total 8
>     drwx------ 2 ldm ldm 4096 Aug 14 14:44 .
>     drwx------ 5 ldm ldm 4096 Aug 14 14:44 ..
>
>     ./work/In/tmp:
>     total 8
>     drwx------ 2 ldm ldm 4096 Aug 14 14:44 .
>     drwx------ 5 ldm ldm 4096 Aug 14 14:44 ..
>
>     I even have renamed my inbox from INBOX to “In” in fear that a hidden
>     file was
>     lurking somewhere. But no dice.
>
>     running
>
>     mbsync -aVD
>
>     produces a huge output. Only showing the very end below
>
>     uid=1051965 flags=S size=0 tuid=?
>     uid=1052014 flags=S size=0 tuid=?
>     uid=1052020 flags=S size=0 tuid=?
>     uid=1052026 flags=S size=0 tuid=?
>     uid=1052032 flags=RS size=0 tuid=?
>     uid=1052050 flags=RS size=0 tuid=?
>     uid=1052062 flags=S size=0 tuid=?
>     uid=1052073 flags=S size=0 tuid=?
>     uid=1052079 flags=S size=0 tuid=?
>     uid=1052085 flags=S size=0 tuid=?
>     uid=1052099 flags=S size=0 tuid=?
>     uid=1052104 flags=S size=0 tuid=?
>     uid=1052112 flags=S size=0 tuid=?
>     far side: 1971 messages, 0 recent
>     matching messages on far side against sync records
>     trying to re-approve uid validity of near side
>     Error: channel work-inbox, near side box INBOX: Unable to recover from
>     UIDVALIDITY change.
>     F: [ 7] Enter cancel_cmds
>     F: [ 7] Callback enter cancel_cmds
>     F: [ 7] Callback leave cancel_cmds
>     F: [ 7] Leave cancel_cmds
>     N: [ 8] Enter cancel_cmds
>     N: [ 8] Callback enter cancel_cmds
>     F: Enter free_store
>     F: Leave free_store
>     N: Enter free_store
>     N: Leave free_store
>     F: >>> 8 LOGOUT
>     N: [ 8] Callback leave cancel_cmds
>     N: [ 8] Leave cancel_cmds
>     F: [ 5] Callback leave load_box
>     F: * BYE Microsoft Exchange Server IMAP4 server signing off.
>     F: 8 OK LOGOUT completed.
>     Processed 1 box(es) in 1 channel(s),
>     pulled 0 new message(s) and 0 flag update(s),
>     expunged 0 message(s) from near side,
>     pushed 0 new message(s) and 0 flag update(s),
>     expunged 0 message(s) from far side.
>     ldm@pildm:~/Mail $
>
>     There are 1971 messages in my inbox, so that value we see above is
>     correct. I
>     have no idea why this is failing. Any help would be much appreciated.
>
>     My .mbsyncrc is pretty vanilla (just removing my address)
>
>     ldm@pildm:~ $ cat .mbsyncrc
>     #################################
>     ######## Account work ###########
>     #################################
>
>     IMAPAccount work
>     Host outlook.office365.com
>     User xx...@yy...
>     PassCmd /home/ldm/bin/oauth2ms
>     AuthMechs XOAUTH2
>     TLSType IMAPS
>     PipelineDepth 1
>
>     Remote storage
>
>     IMAPStore work-remote
>     Account work
>
>     Local storage
>
>     MaildirStore work-local
>     SubFolders Maildir++
>     Inbox ~/Mail/work/In
>
>     Channel work-inbox
>     Far :work-remote:"INBOX"
>     Near :work-local:INBOX
>     Create Both
>     Sync Pull Push New Flags
>     Expunge Both
>
>     Group work
>     Channel work-inbox
>
>     The only .uidvalidity file I have is created in that tree and contains
>     the
>     following
>
>     ldm@pildm:~ $ cat Mail/work/In/.uidvalidity
>     1723661072
>     0
>
>     Help?
>
>     —
>     Laurent
>
>     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
>
>     isync-devel mailing list
>     isy...@li...
>   https://lists.sourceforge.net/lists/listinfo/isync-devel
>
> ----- End forwarded message -----
> ------------------------------
>
> isync-devel mailing list
> isy...@li...
> https://lists.sourceforge.net/lists/listinfo/isync-devel
>
>

Re: Issue with Office365

[Laurent M. <ld...@th...>]

Thanks.

I tried a few things based on your email.

First I do not have any file anywhere going by the name 
“.mbsyncstate”

```
ldm@pildm:~ $ find . -name ".mbsyncstate"
ldm@pildm:~ $
```

I changed my mbsyncrc as you suggested to

```
IMAPAccount work
Host outlook.office365.com
User …..
PassCmd /home/ldm/bin/oauth2ms
AuthMechs XOAUTH2
TLSType IMAPS
PipelineDepth 1



# Remote storage
IMAPStore work-remote
Account work

# Local storage
MaildirStore work-local
SubFolders Maildir++
Inbox ~/FOO/work/Inbox

Channel work-inbox
Far  :work-remote:"INBOX"
Near :work-local:INBOX
Patterns *
Create Near
Sync None
Expunge None
Remove None
```

But it ended with the same UIDVALIDITY complain:


```
ldm@pildm:~ $ mbsync -aV
Reading configuration file /home/ldm/.mbsyncrc
Channel work-inbox
Opening far side store work-remote...
Resolving outlook.office365.com...
Opening near side store work-local...
Connecting to outlook.office365.com (52.96.109.130:993)...
Connection is now encrypted
Logging in...
Authenticating with SASL mechanism XOAUTH2...
Opening far side box INBOX...
Opening near side box INBOX...
Loading far side box...
Loading near side box...
near side: 0 messages, 0 recent
far side: 1973 messages, 0 recent
Error: channel work-inbox, near side box INBOX: Unable to recover from 
UIDVALIDITY change.
Processed 1 box(es) in 1 channel(s).
```

I’m puzzled. This is what the foo folder contains


```
ldm@pildm:~ $ cd FOO/
ldm@pildm:~/FOO $ ls
work
ldm@pildm:~/FOO $ cd work/Inbox/
cur/ new/ tmp/
ldm@pildm:~/FOO $ cd work/Inbox/
ldm@pildm:~/FOO/work/Inbox $ ls -al
total 24
drwx------ 5 ldm ldm 4096 Aug 14 15:55 .
drwx------ 3 ldm ldm 4096 Aug 14 15:55 ..
drwx------ 2 ldm ldm 4096 Aug 14 15:55 cur
drwx------ 2 ldm ldm 4096 Aug 14 15:55 new
drwx------ 2 ldm ldm 4096 Aug 14 15:55 tmp
-rw------- 1 ldm ldm   13 Aug 14 15:55 .uidvalidity
ldm@pildm:~/FOO/work/Inbox $ cat .uidvalidity
1723665330
0
```


On 14 Aug 2024, at 15:45, Marton Balazs wrote:

> As far as I understand, every now and then MS screws up UidValidity, 
> which it really shouldn't do -- but it's MS. On these occasions Mbsync 
> usually recovers most of my folders from this but on a few I get the 
> error you mentioned. I then create an empty directory and pull all of 
> my folders from MS to there using
>
> Patterns *
> Create Near
> Sync None
> Expunge None
> Remove None
>
> This creates the folders and the
> .mbsyncstate
> .uidvalidity
>
> files in them but won't download the actual emails so it runs fast. I 
> don't know why you never mentioned .mbsyncstate? This is the one I 
> need to look at: if its first line like
>
> FarUidValidity 14
>
> differs between the pulled empty folders and my existing local email 
> folder, that's when I get the errors. So, I manually edit this 
> parameter in my local emails folder to match the one I just pulled 
> from MS and magically everything works ok again.
>
> I'm not sure you have the same problem but it would be worth looking 
> at the .mbsyncstate files and the FarUidValidity parameter in them.
>
> Best wishes,
> Marton
>
> On Wed, Aug 14, 2024 at 02:50:19PM GMT, Laurent Michel wrote:
>> Dear All,
>>
>> I had isync running flawlessly for a few years with OAUTH2. Recently, 
>> it seems
>> like my organization must have done an “upgrade” to Office365. 
>> And now,
>> everything is broken on my end.
>>
>> Specifically, each sync end as so:
>>
>> ldm@pildm:~ $ mbsync -aV
>> Reading configuration file /home/ldm/.mbsyncrc
>> Channel work-inbox
>> Opening far side store work-remote...
>> Resolving outlook.office365.com...
>> Opening near side store work-local...
>> Connecting to outlook.office365.com (52.96.97.162:993)...
>> Connection is now encrypted
>> Logging in...
>> Authenticating with SASL mechanism XOAUTH2...
>> Opening far side box INBOX...
>> Opening near side box INBOX...
>> Loading far side box...
>> Loading near side box...
>> near side: 0 messages, 0 recent
>> far side: 1971 messages, 0 recent
>> Error: channel work-inbox, near side box INBOX: Unable to recover 
>> from UIDVALIDITY change.
>> Processed 1 box(es) in 1 channel(s),
>> pulled 0 new message(s) and 0 flag update(s),
>> expunged 0 message(s) from near side,
>> pushed 0 new message(s) and 0 flag update(s),
>> expunged 0 message(s) from far side.
>>
>> As you can see, it’s not an authentication issue but an UIDVALIDITY 
>> issue. I
>> looked on the web and tried pretty much every single thing I found, 
>> including
>> deleting my local copy in its entirety:
>>
>> rm -rf ~/Mail
>>
>> And let it redownload everything. The .uidvalidity file(s) are held 
>> within the
>> ~/Mail
>>
>> and rerunning the command above after atomizing the folder yields 
>> this content
>>
>> ldm@pildm:~ $ cd Mail/
>> ldm@pildm:~/Mail $ tree
>> .
>> └── work
>>     └── In
>>         ├── cur
>>         ├── new
>>         └── tmp
>>
>> 6 directories, 0 files
>> ldm@pildm:~/Mail $ ls -al
>> total 12
>> drwx------  3 ldm ldm   4096 Aug 14 14:44 .
>> drwxr-xr-x 57 ldm users 4096 Aug 14 14:44 ..
>> drwx------  3 ldm ldm   4096 Aug 14 14:44 work
>> ldm@pildm:~/Mail $ ls -alR
>> .:
>> total 12
>> drwx------  3 ldm ldm   4096 Aug 14 14:44 .
>> drwxr-xr-x 57 ldm users 4096 Aug 14 14:44 ..
>> drwx------  3 ldm ldm   4096 Aug 14 14:44 work
>>
>> ./work:
>> total 12
>> drwx------ 3 ldm ldm 4096 Aug 14 14:44 .
>> drwx------ 3 ldm ldm 4096 Aug 14 14:44 ..
>> drwx------ 5 ldm ldm 4096 Aug 14 14:44 In
>>
>> ./work/In:
>> total 24
>> drwx------ 5 ldm ldm 4096 Aug 14 14:44 .
>> drwx------ 3 ldm ldm 4096 Aug 14 14:44 ..
>> drwx------ 2 ldm ldm 4096 Aug 14 14:44 cur
>> drwx------ 2 ldm ldm 4096 Aug 14 14:44 new
>> drwx------ 2 ldm ldm 4096 Aug 14 14:44 tmp
>> -rw------- 1 ldm ldm   13 Aug 14 14:44 .uidvalidity
>>
>> ./work/In/cur:
>> total 8
>> drwx------ 2 ldm ldm 4096 Aug 14 14:44 .
>> drwx------ 5 ldm ldm 4096 Aug 14 14:44 ..
>>
>> ./work/In/new:
>> total 8
>> drwx------ 2 ldm ldm 4096 Aug 14 14:44 .
>> drwx------ 5 ldm ldm 4096 Aug 14 14:44 ..
>>
>> ./work/In/tmp:
>> total 8
>> drwx------ 2 ldm ldm 4096 Aug 14 14:44 .
>> drwx------ 5 ldm ldm 4096 Aug 14 14:44 ..
>>
>> I even have renamed my inbox from INBOX to “In” in fear that a 
>> hidden file was
>> lurking somewhere. But no dice.
>>
>> running
>>
>> mbsync -aVD
>>
>> produces a huge output. Only showing the very end below
>>
>> uid=1051965 flags=S    size=0      tuid=?
>>   uid=1052014 flags=S    size=0      tuid=?
>>   uid=1052020 flags=S    size=0      tuid=?
>>   uid=1052026 flags=S    size=0      tuid=?
>>   uid=1052032 flags=RS   size=0      tuid=?
>>   uid=1052050 flags=RS   size=0      tuid=?
>>   uid=1052062 flags=S    size=0      tuid=?
>>   uid=1052073 flags=S    size=0      tuid=?
>>   uid=1052079 flags=S    size=0      tuid=?
>>   uid=1052085 flags=S    size=0      tuid=?
>>   uid=1052099 flags=S    size=0      tuid=?
>>   uid=1052104 flags=S    size=0      tuid=?
>>   uid=1052112 flags=S    size=0      tuid=?
>> far side: 1971 messages, 0 recent
>> matching messages on far side against sync records
>> trying to re-approve uid validity of near side
>> Error: channel work-inbox, near side box INBOX: Unable to recover 
>> from UIDVALIDITY change.
>> F: [ 7] Enter cancel_cmds
>> F: [ 7] Callback enter cancel_cmds
>> F: [ 7] Callback leave cancel_cmds
>> F: [ 7] Leave cancel_cmds
>> N: [ 8] Enter cancel_cmds
>> N: [ 8] Callback enter cancel_cmds
>> F: Enter free_store
>> F: Leave free_store
>> N: Enter free_store
>> N: Leave free_store
>> F: >>> 8 LOGOUT
>> N: [ 8] Callback leave cancel_cmds
>> N: [ 8] Leave cancel_cmds
>> F: [ 5] Callback leave load_box
>> F: * BYE Microsoft Exchange Server IMAP4 server signing off.
>> F: 8 OK LOGOUT completed.
>> Processed 1 box(es) in 1 channel(s),
>> pulled 0 new message(s) and 0 flag update(s),
>> expunged 0 message(s) from near side,
>> pushed 0 new message(s) and 0 flag update(s),
>> expunged 0 message(s) from far side.
>> ldm@pildm:~/Mail $
>>
>> There are 1971 messages in my inbox, so that value we see above is 
>> correct. I
>> have no idea why this is failing. Any help would be much appreciated.
>>
>> My .mbsyncrc is pretty vanilla (just removing my address)
>>
>> ldm@pildm:~ $ cat .mbsyncrc
>> #################################
>> ######## Account work ###########
>> #################################
>>
>> IMAPAccount work
>> Host outlook.office365.com
>> User xx...@yy...
>> PassCmd /home/ldm/bin/oauth2ms
>> AuthMechs XOAUTH2
>> TLSType IMAPS
>> PipelineDepth 1
>>
>> # Remote storage
>> IMAPStore work-remote
>> Account work
>>
>> # Local storage
>> MaildirStore work-local
>> SubFolders Maildir++
>> Inbox ~/Mail/work/In
>>
>> Channel work-inbox
>> Far  :work-remote:"INBOX"
>> Near :work-local:INBOX
>> Create Both
>> Sync Pull Push New Flags
>> Expunge Both
>>
>> Group work
>> Channel work-inbox
>>
>> The only .uidvalidity file I have is created in that tree and 
>> contains the
>> following
>>
>> ldm@pildm:~ $ cat Mail/work/In/.uidvalidity
>> 1723661072
>> 0
>>
>> Help?
>>
>> —
>> Laurent
>>
>
>
>> _______________________________________________
>> isync-devel mailing list
>> isy...@li...
>> https://lists.sourceforge.net/lists/listinfo/isync-devel

Re: Issue with Office365

[Laurent M. <ld...@th...>]

You are likely going to be a life savior for me. I added the SyncState 
stanza
and it started synchronizing! Fingers crossed that it will do the whole 
bit, but that is amazing.

A .mbsyncstate(.journal | .lock | .new) set of files appeared too.

Thank you so much, it seems like I’m out of the woods!

—
  Laurent

On 14 Aug 2024, at 18:12, Marton Balazs wrote:

> ooops, forgot to cc the List :-)
>
> ----- Forwarded message from Marton Balazs <ba...@gm...> -----
>
> Date: Wed, 14 Aug 2024 23:10:57 +0100
> From: Marton Balazs <ba...@gm...>
> To: Laurent Michel <ld...@th...>
> Subject: Re: Issue with Office365
>
> Do you have
> ~/.mbsync/ ?
>
> My .mbsyncrc starts with
> SyncState *
>
> meaning the sync state files .mbsyncstate are in the local maildir 
> folders. Without this it seems you'll have them in ~/.mbsync/ , so you 
> may need to find and edit the UidValidity parameters there somewhere?
>
>
> "
> SyncState {*|path}
>
> Set the location of this Channel’s synchronization state files. * 
> means that the state should be saved in a file named .mbsyncstate in 
> the near side mailbox itself; this has the advantage that you do not 
> need to handle the state file separately if you delete the mailbox, 
> but it works only with Maildir mailboxes, obviously. Otherwise this is 
> interpreted as a string to prepend to the near side mailbox name to 
> make up a complete path.
> This option can be used outside any section for a global effect. In 
> this case the appended string is made up according to the pattern 
> :far-store:far-box_:near-store:near-box (see also FieldDelimiter 
> below).
> (Global default: ~/.mbsync/).
> "
> https://isync.sourceforge.io/mbsync.html
>
> Best wishes,
> Marton
>
> On Wed, Aug 14, 2024 at 04:11:03PM GMT, Laurent Michel wrote:
>> Thanks.
>>
>> I tried a few things based on your email.
>>
>> First I do not have any file anywhere going by the name 
>> “.mbsyncstate”
>>
>> ldm@pildm:~ $ find . -name ".mbsyncstate"
>> ldm@pildm:~ $
>>
>> I changed my mbsyncrc as you suggested to
>>
>> IMAPAccount work
>> Host outlook.office365.com
>> User …..
>> PassCmd /home/ldm/bin/oauth2ms
>> AuthMechs XOAUTH2
>> TLSType IMAPS
>> PipelineDepth 1
>>
>>
>>
>> # Remote storage
>> IMAPStore work-remote
>> Account work
>>
>> # Local storage
>> MaildirStore work-local
>> SubFolders Maildir++
>> Inbox ~/FOO/work/Inbox
>>
>> Channel work-inbox
>> Far  :work-remote:"INBOX"
>> Near :work-local:INBOX
>> Patterns *
>> Create Near
>> Sync None
>> Expunge None
>> Remove None
>>
>> But it ended with the same UIDVALIDITY complain:
>>
>> ldm@pildm:~ $ mbsync -aV
>> Reading configuration file /home/ldm/.mbsyncrc
>> Channel work-inbox
>> Opening far side store work-remote...
>> Resolving outlook.office365.com...
>> Opening near side store work-local...
>> Connecting to outlook.office365.com (52.96.109.130:993)...
>> Connection is now encrypted
>> Logging in...
>> Authenticating with SASL mechanism XOAUTH2...
>> Opening far side box INBOX...
>> Opening near side box INBOX...
>> Loading far side box...
>> Loading near side box...
>> near side: 0 messages, 0 recent
>> far side: 1973 messages, 0 recent
>> Error: channel work-inbox, near side box INBOX: Unable to recover 
>> from UIDVALIDITY change.
>> Processed 1 box(es) in 1 channel(s).
>>
>> I’m puzzled. This is what the foo folder contains
>>
>> ldm@pildm:~ $ cd FOO/
>> ldm@pildm:~/FOO $ ls
>> work
>> ldm@pildm:~/FOO $ cd work/Inbox/
>> cur/ new/ tmp/
>> ldm@pildm:~/FOO $ cd work/Inbox/
>> ldm@pildm:~/FOO/work/Inbox $ ls -al
>> total 24
>> drwx------ 5 ldm ldm 4096 Aug 14 15:55 .
>> drwx------ 3 ldm ldm 4096 Aug 14 15:55 ..
>> drwx------ 2 ldm ldm 4096 Aug 14 15:55 cur
>> drwx------ 2 ldm ldm 4096 Aug 14 15:55 new
>> drwx------ 2 ldm ldm 4096 Aug 14 15:55 tmp
>> -rw------- 1 ldm ldm   13 Aug 14 15:55 .uidvalidity
>> ldm@pildm:~/FOO/work/Inbox $ cat .uidvalidity
>> 1723665330
>> 0
>>
>> On 14 Aug 2024, at 15:45, Marton Balazs wrote:
>>
>>     As far as I understand, every now and then MS screws up 
>> UidValidity, which
>>     it really shouldn't do -- but it's MS. On these occasions Mbsync 
>> usually
>>     recovers most of my folders from this but on a few I get the 
>> error you
>>     mentioned. I then create an empty directory and pull all of my 
>> folders from
>>     MS to there using
>>
>>     Patterns *
>>     Create Near
>>     Sync None
>>     Expunge None
>>     Remove None
>>
>>     This creates the folders and the
>>     .mbsyncstate
>>     .uidvalidity
>>
>>     files in them but won't download the actual emails so it runs 
>> fast. I don't
>>     know why you never mentioned .mbsyncstate? This is the one I need 
>> to look
>>     at: if its first line like
>>
>>     FarUidValidity 14
>>
>>     differs between the pulled empty folders and my existing local 
>> email
>>     folder, that's when I get the errors. So, I manually edit this 
>> parameter in
>>     my local emails folder to match the one I just pulled from MS and 
>> magically
>>     everything works ok again.
>>
>>     I'm not sure you have the same problem but it would be worth 
>> looking at the
>>     .mbsyncstate files and the FarUidValidity parameter in them.
>>
>>     Best wishes,
>>     Marton
>>
>>     On Wed, Aug 14, 2024 at 02:50:19PM GMT, Laurent Michel wrote:
>>
>>         Dear All,
>>
>>         I had isync running flawlessly for a few years with OAUTH2. 
>> Recently,
>>         it seems
>>         like my organization must have done an “upgrade” to 
>> Office365. And now,
>>         everything is broken on my end.
>>
>>         Specifically, each sync end as so:
>>
>>         ldm@pildm:~ $ mbsync -aV
>>         Reading configuration file /home/ldm/.mbsyncrc
>>         Channel work-inbox
>>         Opening far side store work-remote...
>>         Resolving outlook.office365.com...
>>         Opening near side store work-local...
>>         Connecting to outlook.office365.com (52.96.97.162:993)...
>>         Connection is now encrypted
>>         Logging in...
>>         Authenticating with SASL mechanism XOAUTH2...
>>         Opening far side box INBOX...
>>         Opening near side box INBOX...
>>         Loading far side box...
>>         Loading near side box...
>>         near side: 0 messages, 0 recent
>>         far side: 1971 messages, 0 recent
>>         Error: channel work-inbox, near side box INBOX: Unable to 
>> recover from
>>         UIDVALIDITY change.
>>         Processed 1 box(es) in 1 channel(s),
>>         pulled 0 new message(s) and 0 flag update(s),
>>         expunged 0 message(s) from near side,
>>         pushed 0 new message(s) and 0 flag update(s),
>>         expunged 0 message(s) from far side.
>>
>>         As you can see, it’s not an authentication issue but an 
>> UIDVALIDITY
>>         issue. I
>>         looked on the web and tried pretty much every single thing I 
>> found,
>>         including
>>         deleting my local copy in its entirety:
>>
>>         rm -rf ~/Mail
>>
>>         And let it redownload everything. The .uidvalidity file(s) 
>> are held
>>         within the
>>         ~/Mail
>>
>>         and rerunning the command above after atomizing the folder 
>> yields this
>>         content
>>
>>         ldm@pildm:~ $ cd Mail/
>>         ldm@pildm:~/Mail $ tree
>>         .
>>         └── work
>>         └── In
>>         ├── cur
>>         ├── new
>>         └── tmp
>>
>>         6 directories, 0 files
>>         ldm@pildm:~/Mail $ ls -al
>>         total 12
>>         drwx------ 3 ldm ldm 4096 Aug 14 14:44 .
>>         drwxr-xr-x 57 ldm users 4096 Aug 14 14:44 ..
>>         drwx------ 3 ldm ldm 4096 Aug 14 14:44 work
>>         ldm@pildm:~/Mail $ ls -alR
>>         .:
>>         total 12
>>         drwx------ 3 ldm ldm 4096 Aug 14 14:44 .
>>         drwxr-xr-x 57 ldm users 4096 Aug 14 14:44 ..
>>         drwx------ 3 ldm ldm 4096 Aug 14 14:44 work
>>
>>         ./work:
>>         total 12
>>         drwx------ 3 ldm ldm 4096 Aug 14 14:44 .
>>         drwx------ 3 ldm ldm 4096 Aug 14 14:44 ..
>>         drwx------ 5 ldm ldm 4096 Aug 14 14:44 In
>>
>>         ./work/In:
>>         total 24
>>         drwx------ 5 ldm ldm 4096 Aug 14 14:44 .
>>         drwx------ 3 ldm ldm 4096 Aug 14 14:44 ..
>>         drwx------ 2 ldm ldm 4096 Aug 14 14:44 cur
>>         drwx------ 2 ldm ldm 4096 Aug 14 14:44 new
>>         drwx------ 2 ldm ldm 4096 Aug 14 14:44 tmp
>>         -rw------- 1 ldm ldm 13 Aug 14 14:44 .uidvalidity
>>
>>         ./work/In/cur:
>>         total 8
>>         drwx------ 2 ldm ldm 4096 Aug 14 14:44 .
>>         drwx------ 5 ldm ldm 4096 Aug 14 14:44 ..
>>
>>         ./work/In/new:
>>         total 8
>>         drwx------ 2 ldm ldm 4096 Aug 14 14:44 .
>>         drwx------ 5 ldm ldm 4096 Aug 14 14:44 ..
>>
>>         ./work/In/tmp:
>>         total 8
>>         drwx------ 2 ldm ldm 4096 Aug 14 14:44 .
>>         drwx------ 5 ldm ldm 4096 Aug 14 14:44 ..
>>
>>         I even have renamed my inbox from INBOX to “In” in fear 
>> that a hidden
>>         file was
>>         lurking somewhere. But no dice.
>>
>>         running
>>
>>         mbsync -aVD
>>
>>         produces a huge output. Only showing the very end below
>>
>>         uid=1051965 flags=S size=0 tuid=?
>>         uid=1052014 flags=S size=0 tuid=?
>>         uid=1052020 flags=S size=0 tuid=?
>>         uid=1052026 flags=S size=0 tuid=?
>>         uid=1052032 flags=RS size=0 tuid=?
>>         uid=1052050 flags=RS size=0 tuid=?
>>         uid=1052062 flags=S size=0 tuid=?
>>         uid=1052073 flags=S size=0 tuid=?
>>         uid=1052079 flags=S size=0 tuid=?
>>         uid=1052085 flags=S size=0 tuid=?
>>         uid=1052099 flags=S size=0 tuid=?
>>         uid=1052104 flags=S size=0 tuid=?
>>         uid=1052112 flags=S size=0 tuid=?
>>         far side: 1971 messages, 0 recent
>>         matching messages on far side against sync records
>>         trying to re-approve uid validity of near side
>>         Error: channel work-inbox, near side box INBOX: Unable to 
>> recover from
>>         UIDVALIDITY change.
>>         F: [ 7] Enter cancel_cmds
>>         F: [ 7] Callback enter cancel_cmds
>>         F: [ 7] Callback leave cancel_cmds
>>         F: [ 7] Leave cancel_cmds
>>         N: [ 8] Enter cancel_cmds
>>         N: [ 8] Callback enter cancel_cmds
>>         F: Enter free_store
>>         F: Leave free_store
>>         N: Enter free_store
>>         N: Leave free_store
>>         F: >>> 8 LOGOUT
>>         N: [ 8] Callback leave cancel_cmds
>>         N: [ 8] Leave cancel_cmds
>>         F: [ 5] Callback leave load_box
>>         F: * BYE Microsoft Exchange Server IMAP4 server signing off.
>>         F: 8 OK LOGOUT completed.
>>         Processed 1 box(es) in 1 channel(s),
>>         pulled 0 new message(s) and 0 flag update(s),
>>         expunged 0 message(s) from near side,
>>         pushed 0 new message(s) and 0 flag update(s),
>>         expunged 0 message(s) from far side.
>>         ldm@pildm:~/Mail $
>>
>>         There are 1971 messages in my inbox, so that value we see 
>> above is
>>         correct. I
>>         have no idea why this is failing. Any help would be much 
>> appreciated.
>>
>>         My .mbsyncrc is pretty vanilla (just removing my address)
>>
>>         ldm@pildm:~ $ cat .mbsyncrc
>>         #################################
>>         ######## Account work ###########
>>         #################################
>>
>>         IMAPAccount work
>>         Host outlook.office365.com
>>         User xx...@yy...
>>         PassCmd /home/ldm/bin/oauth2ms
>>         AuthMechs XOAUTH2
>>         TLSType IMAPS
>>         PipelineDepth 1
>>
>>         Remote storage
>>
>>         IMAPStore work-remote
>>         Account work
>>
>>         Local storage
>>
>>         MaildirStore work-local
>>         SubFolders Maildir++
>>         Inbox ~/Mail/work/In
>>
>>         Channel work-inbox
>>         Far :work-remote:"INBOX"
>>         Near :work-local:INBOX
>>         Create Both
>>         Sync Pull Push New Flags
>>         Expunge Both
>>
>>         Group work
>>         Channel work-inbox
>>
>>         The only .uidvalidity file I have is created in that tree and 
>> contains
>>         the
>>         following
>>
>>         ldm@pildm:~ $ cat Mail/work/In/.uidvalidity
>>         1723661072
>>         0
>>
>>         Help?
>>
>>         —
>>         Laurent
>>
>>         ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
>>
>>         isync-devel mailing list
>>         isy...@li...
>>       https://lists.sourceforge.net/lists/listinfo/isync-devel
>>
>
> ----- End forwarded message -----
>
>
> _______________________________________________
> isync-devel mailing list
> isy...@li...
> https://lists.sourceforge.net/lists/listinfo/isync-devel

Fwd: Re: Issue with Office365

[Marton B. <ba...@gm...>]

ooops, forgot to cc the List :-)

----- Forwarded message from Marton Balazs <ba...@gm...> -----

Date: Wed, 14 Aug 2024 23:10:57 +0100
From: Marton Balazs <ba...@gm...>
To: Laurent Michel <ld...@th...>
Subject: Re: Issue with Office365

Do you have
~/.mbsync/ ?

My .mbsyncrc starts with
SyncState * 

meaning the sync state files .mbsyncstate are in the local maildir folders. Without this it seems you'll have them in ~/.mbsync/ , so you may need to find and edit the UidValidity parameters there somewhere?


"
SyncState {*|path}

Set the location of this Channel’s synchronization state files. * means that the state should be saved in a file named .mbsyncstate in the near side mailbox itself; this has the advantage that you do not need to handle the state file separately if you delete the mailbox, but it works only with Maildir mailboxes, obviously. Otherwise this is interpreted as a string to prepend to the near side mailbox name to make up a complete path.
This option can be used outside any section for a global effect. In this case the appended string is made up according to the pattern :far-store:far-box_:near-store:near-box (see also FieldDelimiter below).
(Global default: ~/.mbsync/).
"
https://isync.sourceforge.io/mbsync.html

Best wishes,
Marton

On Wed, Aug 14, 2024 at 04:11:03PM GMT, Laurent Michel wrote:
> Thanks.
> 
> I tried a few things based on your email.
> 
> First I do not have any file anywhere going by the name “.mbsyncstate”
> 
> ldm@pildm:~ $ find . -name ".mbsyncstate"
> ldm@pildm:~ $
> 
> I changed my mbsyncrc as you suggested to
> 
> IMAPAccount work
> Host outlook.office365.com
> User …..
> PassCmd /home/ldm/bin/oauth2ms
> AuthMechs XOAUTH2
> TLSType IMAPS
> PipelineDepth 1
> 
> 
> 
> # Remote storage
> IMAPStore work-remote
> Account work
> 
> # Local storage
> MaildirStore work-local
> SubFolders Maildir++
> Inbox ~/FOO/work/Inbox
> 
> Channel work-inbox
> Far  :work-remote:"INBOX"
> Near :work-local:INBOX
> Patterns *
> Create Near
> Sync None
> Expunge None
> Remove None
> 
> But it ended with the same UIDVALIDITY complain:
> 
> ldm@pildm:~ $ mbsync -aV
> Reading configuration file /home/ldm/.mbsyncrc
> Channel work-inbox
> Opening far side store work-remote...
> Resolving outlook.office365.com...
> Opening near side store work-local...
> Connecting to outlook.office365.com (52.96.109.130:993)...
> Connection is now encrypted
> Logging in...
> Authenticating with SASL mechanism XOAUTH2...
> Opening far side box INBOX...
> Opening near side box INBOX...
> Loading far side box...
> Loading near side box...
> near side: 0 messages, 0 recent
> far side: 1973 messages, 0 recent
> Error: channel work-inbox, near side box INBOX: Unable to recover from UIDVALIDITY change.
> Processed 1 box(es) in 1 channel(s).
> 
> I’m puzzled. This is what the foo folder contains
> 
> ldm@pildm:~ $ cd FOO/
> ldm@pildm:~/FOO $ ls
> work
> ldm@pildm:~/FOO $ cd work/Inbox/
> cur/ new/ tmp/
> ldm@pildm:~/FOO $ cd work/Inbox/
> ldm@pildm:~/FOO/work/Inbox $ ls -al
> total 24
> drwx------ 5 ldm ldm 4096 Aug 14 15:55 .
> drwx------ 3 ldm ldm 4096 Aug 14 15:55 ..
> drwx------ 2 ldm ldm 4096 Aug 14 15:55 cur
> drwx------ 2 ldm ldm 4096 Aug 14 15:55 new
> drwx------ 2 ldm ldm 4096 Aug 14 15:55 tmp
> -rw------- 1 ldm ldm   13 Aug 14 15:55 .uidvalidity
> ldm@pildm:~/FOO/work/Inbox $ cat .uidvalidity
> 1723665330
> 0
> 
> On 14 Aug 2024, at 15:45, Marton Balazs wrote:
> 
>     As far as I understand, every now and then MS screws up UidValidity, which
>     it really shouldn't do -- but it's MS. On these occasions Mbsync usually
>     recovers most of my folders from this but on a few I get the error you
>     mentioned. I then create an empty directory and pull all of my folders from
>     MS to there using
> 
>     Patterns *
>     Create Near
>     Sync None
>     Expunge None
>     Remove None
> 
>     This creates the folders and the
>     .mbsyncstate
>     .uidvalidity
> 
>     files in them but won't download the actual emails so it runs fast. I don't
>     know why you never mentioned .mbsyncstate? This is the one I need to look
>     at: if its first line like
> 
>     FarUidValidity 14
> 
>     differs between the pulled empty folders and my existing local email
>     folder, that's when I get the errors. So, I manually edit this parameter in
>     my local emails folder to match the one I just pulled from MS and magically
>     everything works ok again.
> 
>     I'm not sure you have the same problem but it would be worth looking at the
>     .mbsyncstate files and the FarUidValidity parameter in them.
> 
>     Best wishes,
>     Marton
> 
>     On Wed, Aug 14, 2024 at 02:50:19PM GMT, Laurent Michel wrote:
> 
>         Dear All,
> 
>         I had isync running flawlessly for a few years with OAUTH2. Recently,
>         it seems
>         like my organization must have done an “upgrade” to Office365. And now,
>         everything is broken on my end.
> 
>         Specifically, each sync end as so:
> 
>         ldm@pildm:~ $ mbsync -aV
>         Reading configuration file /home/ldm/.mbsyncrc
>         Channel work-inbox
>         Opening far side store work-remote...
>         Resolving outlook.office365.com...
>         Opening near side store work-local...
>         Connecting to outlook.office365.com (52.96.97.162:993)...
>         Connection is now encrypted
>         Logging in...
>         Authenticating with SASL mechanism XOAUTH2...
>         Opening far side box INBOX...
>         Opening near side box INBOX...
>         Loading far side box...
>         Loading near side box...
>         near side: 0 messages, 0 recent
>         far side: 1971 messages, 0 recent
>         Error: channel work-inbox, near side box INBOX: Unable to recover from
>         UIDVALIDITY change.
>         Processed 1 box(es) in 1 channel(s),
>         pulled 0 new message(s) and 0 flag update(s),
>         expunged 0 message(s) from near side,
>         pushed 0 new message(s) and 0 flag update(s),
>         expunged 0 message(s) from far side.
> 
>         As you can see, it’s not an authentication issue but an UIDVALIDITY
>         issue. I
>         looked on the web and tried pretty much every single thing I found,
>         including
>         deleting my local copy in its entirety:
> 
>         rm -rf ~/Mail
> 
>         And let it redownload everything. The .uidvalidity file(s) are held
>         within the
>         ~/Mail
> 
>         and rerunning the command above after atomizing the folder yields this
>         content
> 
>         ldm@pildm:~ $ cd Mail/
>         ldm@pildm:~/Mail $ tree
>         .
>         └── work
>         └── In
>         ├── cur
>         ├── new
>         └── tmp
> 
>         6 directories, 0 files
>         ldm@pildm:~/Mail $ ls -al
>         total 12
>         drwx------ 3 ldm ldm 4096 Aug 14 14:44 .
>         drwxr-xr-x 57 ldm users 4096 Aug 14 14:44 ..
>         drwx------ 3 ldm ldm 4096 Aug 14 14:44 work
>         ldm@pildm:~/Mail $ ls -alR
>         .:
>         total 12
>         drwx------ 3 ldm ldm 4096 Aug 14 14:44 .
>         drwxr-xr-x 57 ldm users 4096 Aug 14 14:44 ..
>         drwx------ 3 ldm ldm 4096 Aug 14 14:44 work
> 
>         ./work:
>         total 12
>         drwx------ 3 ldm ldm 4096 Aug 14 14:44 .
>         drwx------ 3 ldm ldm 4096 Aug 14 14:44 ..
>         drwx------ 5 ldm ldm 4096 Aug 14 14:44 In
> 
>         ./work/In:
>         total 24
>         drwx------ 5 ldm ldm 4096 Aug 14 14:44 .
>         drwx------ 3 ldm ldm 4096 Aug 14 14:44 ..
>         drwx------ 2 ldm ldm 4096 Aug 14 14:44 cur
>         drwx------ 2 ldm ldm 4096 Aug 14 14:44 new
>         drwx------ 2 ldm ldm 4096 Aug 14 14:44 tmp
>         -rw------- 1 ldm ldm 13 Aug 14 14:44 .uidvalidity
> 
>         ./work/In/cur:
>         total 8
>         drwx------ 2 ldm ldm 4096 Aug 14 14:44 .
>         drwx------ 5 ldm ldm 4096 Aug 14 14:44 ..
> 
>         ./work/In/new:
>         total 8
>         drwx------ 2 ldm ldm 4096 Aug 14 14:44 .
>         drwx------ 5 ldm ldm 4096 Aug 14 14:44 ..
> 
>         ./work/In/tmp:
>         total 8
>         drwx------ 2 ldm ldm 4096 Aug 14 14:44 .
>         drwx------ 5 ldm ldm 4096 Aug 14 14:44 ..
> 
>         I even have renamed my inbox from INBOX to “In” in fear that a hidden
>         file was
>         lurking somewhere. But no dice.
> 
>         running
> 
>         mbsync -aVD
> 
>         produces a huge output. Only showing the very end below
> 
>         uid=1051965 flags=S size=0 tuid=?
>         uid=1052014 flags=S size=0 tuid=?
>         uid=1052020 flags=S size=0 tuid=?
>         uid=1052026 flags=S size=0 tuid=?
>         uid=1052032 flags=RS size=0 tuid=?
>         uid=1052050 flags=RS size=0 tuid=?
>         uid=1052062 flags=S size=0 tuid=?
>         uid=1052073 flags=S size=0 tuid=?
>         uid=1052079 flags=S size=0 tuid=?
>         uid=1052085 flags=S size=0 tuid=?
>         uid=1052099 flags=S size=0 tuid=?
>         uid=1052104 flags=S size=0 tuid=?
>         uid=1052112 flags=S size=0 tuid=?
>         far side: 1971 messages, 0 recent
>         matching messages on far side against sync records
>         trying to re-approve uid validity of near side
>         Error: channel work-inbox, near side box INBOX: Unable to recover from
>         UIDVALIDITY change.
>         F: [ 7] Enter cancel_cmds
>         F: [ 7] Callback enter cancel_cmds
>         F: [ 7] Callback leave cancel_cmds
>         F: [ 7] Leave cancel_cmds
>         N: [ 8] Enter cancel_cmds
>         N: [ 8] Callback enter cancel_cmds
>         F: Enter free_store
>         F: Leave free_store
>         N: Enter free_store
>         N: Leave free_store
>         F: >>> 8 LOGOUT
>         N: [ 8] Callback leave cancel_cmds
>         N: [ 8] Leave cancel_cmds
>         F: [ 5] Callback leave load_box
>         F: * BYE Microsoft Exchange Server IMAP4 server signing off.
>         F: 8 OK LOGOUT completed.
>         Processed 1 box(es) in 1 channel(s),
>         pulled 0 new message(s) and 0 flag update(s),
>         expunged 0 message(s) from near side,
>         pushed 0 new message(s) and 0 flag update(s),
>         expunged 0 message(s) from far side.
>         ldm@pildm:~/Mail $
> 
>         There are 1971 messages in my inbox, so that value we see above is
>         correct. I
>         have no idea why this is failing. Any help would be much appreciated.
> 
>         My .mbsyncrc is pretty vanilla (just removing my address)
> 
>         ldm@pildm:~ $ cat .mbsyncrc
>         #################################
>         ######## Account work ###########
>         #################################
> 
>         IMAPAccount work
>         Host outlook.office365.com
>         User xx...@yy...
>         PassCmd /home/ldm/bin/oauth2ms
>         AuthMechs XOAUTH2
>         TLSType IMAPS
>         PipelineDepth 1
> 
>         Remote storage
> 
>         IMAPStore work-remote
>         Account work
> 
>         Local storage
> 
>         MaildirStore work-local
>         SubFolders Maildir++
>         Inbox ~/Mail/work/In
> 
>         Channel work-inbox
>         Far :work-remote:"INBOX"
>         Near :work-local:INBOX
>         Create Both
>         Sync Pull Push New Flags
>         Expunge Both
> 
>         Group work
>         Channel work-inbox
> 
>         The only .uidvalidity file I have is created in that tree and contains
>         the
>         following
> 
>         ldm@pildm:~ $ cat Mail/work/In/.uidvalidity
>         1723661072
>         0
> 
>         Help?
> 
>         —
>         Laurent
> 
>         ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
>        
>         isync-devel mailing list
>         isy...@li...
>         https://lists.sourceforge.net/lists/listinfo/isync-devel
> 

----- End forwarded message -----

Re: Issue with Office365

[Marton B. <ba...@gm...>]

As far as I understand, every now and then MS screws up UidValidity, which it really shouldn't do -- but it's MS. On these occasions Mbsync usually recovers most of my folders from this but on a few I get the error you mentioned. I then create an empty directory and pull all of my folders from MS to there using

Patterns *
Create Near
Sync None
Expunge None
Remove None

This creates the folders and the
.mbsyncstate
.uidvalidity

files in them but won't download the actual emails so it runs fast. I don't know why you never mentioned .mbsyncstate? This is the one I need to look at: if its first line like

FarUidValidity 14

differs between the pulled empty folders and my existing local email folder, that's when I get the errors. So, I manually edit this parameter in my local emails folder to match the one I just pulled from MS and magically everything works ok again.

I'm not sure you have the same problem but it would be worth looking at the .mbsyncstate files and the FarUidValidity parameter in them.

Best wishes,
Marton

On Wed, Aug 14, 2024 at 02:50:19PM GMT, Laurent Michel wrote:
> Dear All,
> 
> I had isync running flawlessly for a few years with OAUTH2. Recently, it seems
> like my organization must have done an “upgrade” to Office365. And now,
> everything is broken on my end.
> 
> Specifically, each sync end as so:
> 
> ldm@pildm:~ $ mbsync -aV
> Reading configuration file /home/ldm/.mbsyncrc
> Channel work-inbox
> Opening far side store work-remote...
> Resolving outlook.office365.com...
> Opening near side store work-local...
> Connecting to outlook.office365.com (52.96.97.162:993)...
> Connection is now encrypted
> Logging in...
> Authenticating with SASL mechanism XOAUTH2...
> Opening far side box INBOX...
> Opening near side box INBOX...
> Loading far side box...
> Loading near side box...
> near side: 0 messages, 0 recent
> far side: 1971 messages, 0 recent
> Error: channel work-inbox, near side box INBOX: Unable to recover from UIDVALIDITY change.
> Processed 1 box(es) in 1 channel(s),
> pulled 0 new message(s) and 0 flag update(s),
> expunged 0 message(s) from near side,
> pushed 0 new message(s) and 0 flag update(s),
> expunged 0 message(s) from far side.
> 
> As you can see, it’s not an authentication issue but an UIDVALIDITY issue. I
> looked on the web and tried pretty much every single thing I found, including
> deleting my local copy in its entirety:
> 
> rm -rf ~/Mail
> 
> And let it redownload everything. The .uidvalidity file(s) are held within the
> ~/Mail
> 
> and rerunning the command above after atomizing the folder yields this content
> 
> ldm@pildm:~ $ cd Mail/
> ldm@pildm:~/Mail $ tree
> .
> └── work
>     └── In
>         ├── cur
>         ├── new
>         └── tmp
> 
> 6 directories, 0 files
> ldm@pildm:~/Mail $ ls -al
> total 12
> drwx------  3 ldm ldm   4096 Aug 14 14:44 .
> drwxr-xr-x 57 ldm users 4096 Aug 14 14:44 ..
> drwx------  3 ldm ldm   4096 Aug 14 14:44 work
> ldm@pildm:~/Mail $ ls -alR
> .:
> total 12
> drwx------  3 ldm ldm   4096 Aug 14 14:44 .
> drwxr-xr-x 57 ldm users 4096 Aug 14 14:44 ..
> drwx------  3 ldm ldm   4096 Aug 14 14:44 work
> 
> ./work:
> total 12
> drwx------ 3 ldm ldm 4096 Aug 14 14:44 .
> drwx------ 3 ldm ldm 4096 Aug 14 14:44 ..
> drwx------ 5 ldm ldm 4096 Aug 14 14:44 In
> 
> ./work/In:
> total 24
> drwx------ 5 ldm ldm 4096 Aug 14 14:44 .
> drwx------ 3 ldm ldm 4096 Aug 14 14:44 ..
> drwx------ 2 ldm ldm 4096 Aug 14 14:44 cur
> drwx------ 2 ldm ldm 4096 Aug 14 14:44 new
> drwx------ 2 ldm ldm 4096 Aug 14 14:44 tmp
> -rw------- 1 ldm ldm   13 Aug 14 14:44 .uidvalidity
> 
> ./work/In/cur:
> total 8
> drwx------ 2 ldm ldm 4096 Aug 14 14:44 .
> drwx------ 5 ldm ldm 4096 Aug 14 14:44 ..
> 
> ./work/In/new:
> total 8
> drwx------ 2 ldm ldm 4096 Aug 14 14:44 .
> drwx------ 5 ldm ldm 4096 Aug 14 14:44 ..
> 
> ./work/In/tmp:
> total 8
> drwx------ 2 ldm ldm 4096 Aug 14 14:44 .
> drwx------ 5 ldm ldm 4096 Aug 14 14:44 ..
> 
> I even have renamed my inbox from INBOX to “In” in fear that a hidden file was
> lurking somewhere. But no dice.
> 
> running
> 
> mbsync -aVD
> 
> produces a huge output. Only showing the very end below
> 
> uid=1051965 flags=S    size=0      tuid=?
>   uid=1052014 flags=S    size=0      tuid=?
>   uid=1052020 flags=S    size=0      tuid=?
>   uid=1052026 flags=S    size=0      tuid=?
>   uid=1052032 flags=RS   size=0      tuid=?
>   uid=1052050 flags=RS   size=0      tuid=?
>   uid=1052062 flags=S    size=0      tuid=?
>   uid=1052073 flags=S    size=0      tuid=?
>   uid=1052079 flags=S    size=0      tuid=?
>   uid=1052085 flags=S    size=0      tuid=?
>   uid=1052099 flags=S    size=0      tuid=?
>   uid=1052104 flags=S    size=0      tuid=?
>   uid=1052112 flags=S    size=0      tuid=?
> far side: 1971 messages, 0 recent
> matching messages on far side against sync records
> trying to re-approve uid validity of near side
> Error: channel work-inbox, near side box INBOX: Unable to recover from UIDVALIDITY change.
> F: [ 7] Enter cancel_cmds
> F: [ 7] Callback enter cancel_cmds
> F: [ 7] Callback leave cancel_cmds
> F: [ 7] Leave cancel_cmds
> N: [ 8] Enter cancel_cmds
> N: [ 8] Callback enter cancel_cmds
> F: Enter free_store
> F: Leave free_store
> N: Enter free_store
> N: Leave free_store
> F: >>> 8 LOGOUT
> N: [ 8] Callback leave cancel_cmds
> N: [ 8] Leave cancel_cmds
> F: [ 5] Callback leave load_box
> F: * BYE Microsoft Exchange Server IMAP4 server signing off.
> F: 8 OK LOGOUT completed.
> Processed 1 box(es) in 1 channel(s),
> pulled 0 new message(s) and 0 flag update(s),
> expunged 0 message(s) from near side,
> pushed 0 new message(s) and 0 flag update(s),
> expunged 0 message(s) from far side.
> ldm@pildm:~/Mail $
> 
> There are 1971 messages in my inbox, so that value we see above is correct. I
> have no idea why this is failing. Any help would be much appreciated.
> 
> My .mbsyncrc is pretty vanilla (just removing my address)
> 
> ldm@pildm:~ $ cat .mbsyncrc
> #################################
> ######## Account work ###########
> #################################
> 
> IMAPAccount work
> Host outlook.office365.com
> User xx...@yy...
> PassCmd /home/ldm/bin/oauth2ms
> AuthMechs XOAUTH2
> TLSType IMAPS
> PipelineDepth 1
> 
> # Remote storage
> IMAPStore work-remote
> Account work
> 
> # Local storage
> MaildirStore work-local
> SubFolders Maildir++
> Inbox ~/Mail/work/In
> 
> Channel work-inbox
> Far  :work-remote:"INBOX"
> Near :work-local:INBOX
> Create Both
> Sync Pull Push New Flags
> Expunge Both
> 
> Group work
> Channel work-inbox
> 
> The only .uidvalidity file I have is created in that tree and contains the
> following
> 
> ldm@pildm:~ $ cat Mail/work/In/.uidvalidity
> 1723661072
> 0
> 
> Help?
> 
> —
> Laurent
> 


> _______________________________________________
> isync-devel mailing list
> isy...@li...
> https://lists.sourceforge.net/lists/listinfo/isync-devel

Issue with Office365

[Laurent M. <ld...@th...>]

Dear All,

I had isync running flawlessly for a few years with OAUTH2. Recently, it 
seems like my organization must have done an “upgrade” to Office365. 
And now, everything is broken on my end.

Specifically, each sync end as so:

```
ldm@pildm:~ $ mbsync -aV
Reading configuration file /home/ldm/.mbsyncrc
Channel work-inbox
Opening far side store work-remote...
Resolving outlook.office365.com...
Opening near side store work-local...
Connecting to outlook.office365.com (52.96.97.162:993)...
Connection is now encrypted
Logging in...
Authenticating with SASL mechanism XOAUTH2...
Opening far side box INBOX...
Opening near side box INBOX...
Loading far side box...
Loading near side box...
near side: 0 messages, 0 recent
far side: 1971 messages, 0 recent
Error: channel work-inbox, near side box INBOX: Unable to recover from 
UIDVALIDITY change.
Processed 1 box(es) in 1 channel(s),
pulled 0 new message(s) and 0 flag update(s),
expunged 0 message(s) from near side,
pushed 0 new message(s) and 0 flag update(s),
expunged 0 message(s) from far side.
```
As you can see, it’s not an authentication issue but an UIDVALIDITY 
issue. I looked on the web and tried pretty much every single thing I 
found, including deleting my local copy in its entirety:

```
rm -rf ~/Mail
```

And let it redownload everything. The `.uidvalidity` file(s) are held 
within the `~/Mail`

and rerunning the command above after atomizing the folder yields this 
content

```
ldm@pildm:~ $ cd Mail/
ldm@pildm:~/Mail $ tree
.
└── work
     └── In
         ├── cur
         ├── new
         └── tmp

6 directories, 0 files
ldm@pildm:~/Mail $ ls -al
total 12
drwx------  3 ldm ldm   4096 Aug 14 14:44 .
drwxr-xr-x 57 ldm users 4096 Aug 14 14:44 ..
drwx------  3 ldm ldm   4096 Aug 14 14:44 work
ldm@pildm:~/Mail $ ls -alR
.:
total 12
drwx------  3 ldm ldm   4096 Aug 14 14:44 .
drwxr-xr-x 57 ldm users 4096 Aug 14 14:44 ..
drwx------  3 ldm ldm   4096 Aug 14 14:44 work

./work:
total 12
drwx------ 3 ldm ldm 4096 Aug 14 14:44 .
drwx------ 3 ldm ldm 4096 Aug 14 14:44 ..
drwx------ 5 ldm ldm 4096 Aug 14 14:44 In

./work/In:
total 24
drwx------ 5 ldm ldm 4096 Aug 14 14:44 .
drwx------ 3 ldm ldm 4096 Aug 14 14:44 ..
drwx------ 2 ldm ldm 4096 Aug 14 14:44 cur
drwx------ 2 ldm ldm 4096 Aug 14 14:44 new
drwx------ 2 ldm ldm 4096 Aug 14 14:44 tmp
-rw------- 1 ldm ldm   13 Aug 14 14:44 .uidvalidity

./work/In/cur:
total 8
drwx------ 2 ldm ldm 4096 Aug 14 14:44 .
drwx------ 5 ldm ldm 4096 Aug 14 14:44 ..

./work/In/new:
total 8
drwx------ 2 ldm ldm 4096 Aug 14 14:44 .
drwx------ 5 ldm ldm 4096 Aug 14 14:44 ..

./work/In/tmp:
total 8
drwx------ 2 ldm ldm 4096 Aug 14 14:44 .
drwx------ 5 ldm ldm 4096 Aug 14 14:44 ..
```

I even have renamed my inbox from INBOX to “In” in fear that a 
hidden file was lurking somewhere.  But no dice.

running

```
mbsync -aVD
```

produces a huge output. Only showing the very end below

```
uid=1051965 flags=S    size=0      tuid=?
   uid=1052014 flags=S    size=0      tuid=?
   uid=1052020 flags=S    size=0      tuid=?
   uid=1052026 flags=S    size=0      tuid=?
   uid=1052032 flags=RS   size=0      tuid=?
   uid=1052050 flags=RS   size=0      tuid=?
   uid=1052062 flags=S    size=0      tuid=?
   uid=1052073 flags=S    size=0      tuid=?
   uid=1052079 flags=S    size=0      tuid=?
   uid=1052085 flags=S    size=0      tuid=?
   uid=1052099 flags=S    size=0      tuid=?
   uid=1052104 flags=S    size=0      tuid=?
   uid=1052112 flags=S    size=0      tuid=?
far side: 1971 messages, 0 recent
matching messages on far side against sync records
trying to re-approve uid validity of near side
Error: channel work-inbox, near side box INBOX: Unable to recover from 
UIDVALIDITY change.
F: [ 7] Enter cancel_cmds
F: [ 7] Callback enter cancel_cmds
F: [ 7] Callback leave cancel_cmds
F: [ 7] Leave cancel_cmds
N: [ 8] Enter cancel_cmds
N: [ 8] Callback enter cancel_cmds
F: Enter free_store
F: Leave free_store
N: Enter free_store
N: Leave free_store
F: >>> 8 LOGOUT
N: [ 8] Callback leave cancel_cmds
N: [ 8] Leave cancel_cmds
F: [ 5] Callback leave load_box
F: * BYE Microsoft Exchange Server IMAP4 server signing off.
F: 8 OK LOGOUT completed.
Processed 1 box(es) in 1 channel(s),
pulled 0 new message(s) and 0 flag update(s),
expunged 0 message(s) from near side,
pushed 0 new message(s) and 0 flag update(s),
expunged 0 message(s) from far side.
ldm@pildm:~/Mail $
```

There are 1971 messages in my inbox, so that value we see above is 
correct. I have no idea why this is failing. Any help would be much 
appreciated.

My .mbsyncrc is pretty vanilla (just removing my address)

```
ldm@pildm:~ $ cat .mbsyncrc
#################################
######## Account work ###########
#################################

IMAPAccount work
Host outlook.office365.com
User xx...@yy...
PassCmd /home/ldm/bin/oauth2ms
AuthMechs XOAUTH2
TLSType IMAPS
PipelineDepth 1

# Remote storage
IMAPStore work-remote
Account work

# Local storage
MaildirStore work-local
SubFolders Maildir++
Inbox ~/Mail/work/In

Channel work-inbox
Far  :work-remote:"INBOX"
Near :work-local:INBOX
Create Both
Sync Pull Push New Flags
Expunge Both

Group work
Channel work-inbox
```

The only .uidvalidity file I have is created in that tree and contains 
the following

```
ldm@pildm:~ $ cat Mail/work/In/.uidvalidity
1723661072
0
```


Help?

—
  Laurent

Re: isync 1.5 breaks configuration

[Norbert P. <no...@pr...>]

Hi Oswald,

On Thu, 08 Aug 2024, Oswald Buddenhagen via isync-devel wrote:
> hmm, yeah, Inbox' default is ~/Maildir, so your Path collides with the
> restriction (cf. commit 3bfc3c50).
> in principle, i could delay the complaint until INBOX is actually
> referenced, but this would add quite some complexity.
> a simpler but much more drastic measure would be removing the default
> altogether - i don't suppose that too many people would be actually
> affected by this (everyone who needs it configures it explicitly, no?),
> so it may be still acceptable. disagreements?

As I wrote, I have fixed it with adding explicit inbox lines to each
stanza. 
But I really don't know what the statement is changing (besides the
detection).

For me as of now it is fixed, but getting rid of the (to me) useless
Inbox statements wouldn't be bad, either.

Thanks for your work on this, it is such a great tool!

Best regards

Norbert

--
PREINING Norbert                              https://www.preining.info
arXiv / Cornell University   +   IFMGA Guide   +   TU Wien  +  TeX Live
GPG: 0x860CDC13   fp: F7D8 A928 26E3 16A1 9FA0 ACF0 6CAC A448 860C DC13

Re: isync 1.5 breaks configuration

[Oswald B. <osw...@gm...>]

On Thu, Aug 08, 2024 at 11:07:38AM +0900, Norbert Preining wrote:
>On Thu, 08 Aug 2024, Norbert Preining wrote:
>> 	Maildir store 'xxxx-local': Path cannot be nested under Inbox
>>
>> The respective entry in .mbsyncrc is:
>>
>> 	MaildirStore xxxx-local
>> 	Path ~/Maildir/xxxx/
>
>Adding
>
>	Inbox ~/Maildir/xxxx/xxxx
>
>there [...] makes all work again.
>
hmm, yeah, Inbox' default is ~/Maildir, so your Path collides with the
restriction (cf. commit 3bfc3c50).
in principle, i could delay the complaint until INBOX is actually
referenced, but this would add quite some complexity.
a simpler but much more drastic measure would be removing the default
altogether - i don't suppose that too many people would be actually
affected by this (everyone who needs it configures it explicitly, no?),
so it may be still acceptable. disagreements?

Re: isync 1.5 breaks configuration

[Norbert P. <no...@pr...>]

On Thu, 08 Aug 2024, Norbert Preining wrote:
> 	Maildir store 'xxxx-local': Path cannot be nested under Inbox
> 
> The respective entry in .mbsyncrc is:
> 
> 	MaildirStore xxxx-local
> 	Path ~/Maildir/xxxx/

Adding
	Inbox ~/Maildir/xxxx/xxxx
there, since I have

	Channel xxxx-INBOX
	Far :xxxx-remote:INBOX
	Near :xxxx-local:xxxx

set, makes all work again.

Best regards

Norbert

--
PREINING Norbert                              https://www.preining.info
arXiv / Cornell University   +   IFMGA Guide   +   TU Wien  +  TeX Live
GPG: 0x860CDC13   fp: F7D8 A928 26E3 16A1 9FA0 ACF0 6CAC A448 860C DC13

isync 1.5 breaks configuration

[Norbert P. <no...@pr...>]

Hi all

Since upgrade from 1.4.N to 1.5 now my .mbsyncrc does not work anymore,
I get warnings:

	Maildir store 'xxxx-local': Path cannot be nested under Inbox

The respective entry in .mbsyncrc is:

	MaildirStore xxxx-local
	Path ~/Maildir/xxxx/

This used to work without any problems at all. And my local mail readers
are also very happy with that (mutt).

Is this an intentional change? And if, what is the best way to remedy
this on the local side?

Thanks

Norbert

--
PREINING Norbert                              https://www.preining.info
arXiv / Cornell University   +   IFMGA Guide   +   TU Wien  +  TeX Live
GPG: 0x860CDC13   fp: F7D8 A928 26E3 16A1 9FA0 ACF0 6CAC A448 860C DC13

Re: .mbsyncrc correct?

[Oswald B. <osw...@gm...>]

On Wed, Aug 07, 2024 at 06:27:35AM +0700, Louis A. Turk via isync-devel wrote:
>Error: Cannot resolve server 'mail.firmanelohim.org/': Name or service not known.
>
>Is my .mbsyncrc correct?
>
no:

>Host mail.fm.org/
>
that trailing slash makes no sense at all.
maybe you got confused by the requirement for Path?

.mbsyncrc correct?

[Louis A. T. <la...@fi...>]

Hello everyone,

My first post. I'm having trouble getting mbsync to download my email. mbsync -a gives: Error: Cannot resolve server 'mail.firmanelohim.org/': Name or service not known.

Is my .mbsyncrc correct? Here it is:


IMAPAccount laturk
Host mail.fm.org/
Port 993
Use...@fm...
Pass MyPassword
SSLVersion TLSv1.2
CertificateFile /etc/ssl/certs/ca-certificates.crt

IMAPStore laturk-remote
Account laturk

MaildirStore laturk-local
Path ~/.mail/laturk/
Inbox ~/.mail/laturk/INBOX
Trash ~/.mail/laturk/Trash
SubFolders Verbatim

Channel laturk
Far :laturk-remote:
Near :laturk-local:
Patterns *
Create Both
Expunge None
SyncState *

Group laturk
Channel laturk-inbox
Channel laturk-sent
Channel laturk-trash
Channel laturk-all
Channel laturk-starred

================================

Thanks in advance,

Louis

[commit] master: add missing trailing newlines in error() calls

[ossi <os...@us...>]

commit 76e5f223ee975ede474333247d43888d18d77ece
Author: Oswald Buddenhagen <os...@us...>
Date:   Tue Aug 6 00:43:42 2024 +0200

    add missing trailing newlines in error() calls

 src/drv_imap.c    | 2 +-
 src/drv_maildir.c | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/src/drv_imap.c b/src/drv_imap.c
index b4c05fd..1280e3c 100644
--- a/src/drv_imap.c
+++ b/src/drv_imap.c
@@ -1682,7 +1682,7 @@ prepare_box( char **buf, const imap_store_t *ctx )
 		if (!memcmp( name, "INBOX", 5 )) {
 			pfx = "";
 		} else if (!*pfx) {
-			error( "IMAP error: cannot use unqualified '%s'. Did you mean INBOX?", name );
+			error( "IMAP error: cannot use unqualified '%s'. Did you mean INBOX?\n", name );
 			return -1;
 		}
 	}
diff --git a/src/drv_maildir.c b/src/drv_maildir.c
index 847bc06..d15cf46 100644
--- a/src/drv_maildir.c
+++ b/src/drv_maildir.c
@@ -1533,7 +1533,7 @@ maildir_fetch_msg( store_t *gctx, message_t *gmsg, msg_data_t *data, int minimal
 	}
 	fstat( fd, &st );
 	if (st.st_size > INT_MAX) {
-		error( "Maildir error: %s is too big", buf );
+		error( "Maildir error: %s is too big\n", buf );
 		goto mbad;
 	}
 	data->len = st.st_size;

[commit] master: fix IMAP INBOX case normalization

[ossi <os...@us...>]

commit d54c22d20e5d3173c5398c2e9fc5e87837a26963
Author: Oswald Buddenhagen <os...@us...>
Date:   Tue Aug 6 15:16:27 2024 +0200

    fix IMAP INBOX case normalization
    
    while is_INBOX() was adjusted to work with not null-terminated strings,
    is_inbox() wasn't.
    
    amends 3aead33.

 src/drv_imap.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/src/drv_imap.c b/src/drv_imap.c
index cbf5804..b4c05fd 100644
--- a/src/drv_imap.c
+++ b/src/drv_imap.c
@@ -1530,7 +1530,7 @@ is_inbox( imap_store_t *ctx, const char *arg, int argl )
 {
 	if (!starts_with_upper( arg, argl, "INBOX", 5 ))
 		return 0;
-	if (arg[5] && arg[5] != ctx->delimiter[0])
+	if (argl > 5 && arg[5] != ctx->delimiter[0])
 		return 0;
 	return 1;
 }
@@ -3602,7 +3602,7 @@ imap_list_store( store_t *gctx, int flags,
 	// path      | P [i] | i [P] | P
 	//
 	int pfx_is_empty = !*ctx->prefix;
-	int pfx_is_inbox = !pfx_is_empty && is_inbox( ctx, ctx->prefix, -1 );
+	int pfx_is_inbox = !pfx_is_empty && is_inbox( ctx, ctx->prefix, strlen( ctx->prefix ) );
 	if (((flags & (LIST_PATH | LIST_PATH_MAYBE)) || pfx_is_empty) && !pfx_is_inbox && !(ctx->listed & LIST_PATH)) {
 		ctx->listed |= LIST_PATH;
 		if (pfx_is_empty)

[commit] master: fix initial build from git

[ossi <os...@us...>]

commit 4c2031d616f92bd7437fdb34cf4eafa0840c84bd
Author: Oswald Buddenhagen <os...@us...>
Date:   Tue Aug 6 09:33:31 2024 +0200

    fix initial build from git
    
    we need to ignore the absence of VERSION, as aclocal executes the
    include() while ignoring the prior m4_syscmd().

 configure.ac | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/configure.ac b/configure.ac
index 2555ce5..7f1fc6e 100644
--- a/configure.ac
+++ b/configure.ac
@@ -3,7 +3,7 @@
 dnl SPDX-License-Identifier: GPL-2.0-or-later
 
 m4_syscmd([./version.sh])
-AC_INIT([isync], m4_include([VERSION]))
+AC_INIT([isync], m4_sinclude([VERSION]))
 AC_CONFIG_HEADERS([autodefs.h])
 
 AC_CANONICAL_TARGET

[commit] tag 'v1.5.0' created

[ossi <os...@us...>]

The tag 'v1.5.0' has been created at 4279aea.

[commit] master: substitute version and date in man pages

[ossi <os...@us...>]

commit 6fbbcbb2c76a3b56e07e2ed4240506cb94973d45
Author: Oswald Buddenhagen <os...@us...>
Date:   Fri Aug 2 10:14:26 2024 +0200

    substitute version and date in man pages
    
    this shortens the release checklist and reduces commit churn.
    
    for the date we use configure's timestamp. this should reflect the
    package's creation time and be consistent with the version.

 configure.ac                        | 5 ++++-
 src/.gitignore                      | 2 ++
 src/Makefile.am                     | 5 ++++-
 src/{mbsync.1 => mbsync.1.in}       | 2 +-
 src/{mdconvert.1 => mdconvert.1.in} | 2 +-
 5 files changed, 12 insertions(+), 4 deletions(-)

diff --git a/configure.ac b/configure.ac
index 41670ca..2555ce5 100644
--- a/configure.ac
+++ b/configure.ac
@@ -251,7 +251,10 @@ if test "x$have_macos_keychain" != xno; then
     AC_SUBST(KEYCHAIN_LIBS, ["-Wl,-framework,Security,-framework,CoreFoundation"])
 fi
 
-AC_CONFIG_FILES([Makefile src/Makefile isync.spec])
+RELEASE_DATE=`date -r $0 +%F`
+AC_SUBST(RELEASE_DATE)
+
+AC_CONFIG_FILES([Makefile src/Makefile src/mbsync.1 src/mdconvert.1 isync.spec])
 AC_OUTPUT
 
 AC_MSG_RESULT()
diff --git a/src/.gitignore b/src/.gitignore
index dea9ee7..a80a371 100644
--- a/src/.gitignore
+++ b/src/.gitignore
@@ -1,6 +1,8 @@
 /drv_proxy.inc
 /mbsync
+/mbsync.1
 /mdconvert
+/mdconvert.1
 /tst_imap_msgs
 /tst_imap_utf7
 /tst_msg_cvt
diff --git a/src/Makefile.am b/src/Makefile.am
index 42d14f4..d6e0407 100644
--- a/src/Makefile.am
+++ b/src/Makefile.am
@@ -27,7 +27,10 @@ mdconvert_prog = mdconvert
 mdconvert_man = mdconvert.1
 endif
 
+in_man = mbsync.1.in mdconvert.1.in
+
 bin_PROGRAMS = mbsync $(mdconvert_prog)
+# don't forget to update AC_CONFIG_FILES in configure.ac!
 man_MANS = mbsync.1 $(mdconvert_man)
 
 tst_imap_msgs_SOURCES = tst_imap_msgs.c imap_msgs.c util.c
@@ -47,6 +50,6 @@ EXTRA_PROGRAMS = tst_timers
 exampledir = $(docdir)/examples
 example_DATA = mbsyncrc.sample
 
-EXTRA_DIST = drv_proxy_gen.pl run-tests.pl $(example_DATA) $(man_MANS)
+EXTRA_DIST = drv_proxy_gen.pl run-tests.pl $(example_DATA) $(in_man)
 
 CLEANFILES = drv_proxy.inc
diff --git a/src/mbsync.1 b/src/mbsync.1.in
similarity index 99%
rename from src/mbsync.1
rename to src/mbsync.1.in
index 939c8c5..89c7a4a 100644
--- a/src/mbsync.1
+++ b/src/mbsync.1.in
@@ -4,7 +4,7 @@
 .\"
 .\" mbsync - mailbox synchronizer
 .
-.TH mbsync 1 "2022 Jun 16"
+.TH mbsync 1 @RELEASE_DATE@ "@PACKAGE_STRING@" "User Commands"
 .
 .SH NAME
 mbsync - synchronize IMAP4 and Maildir mailboxes
diff --git a/src/mdconvert.1 b/src/mdconvert.1.in
similarity index 93%
rename from src/mdconvert.1
rename to src/mdconvert.1.in
index b616841..e8800d9 100644
--- a/src/mdconvert.1
+++ b/src/mdconvert.1.in
@@ -3,7 +3,7 @@
 .\"
 .\" mdconvert - Maildir mailbox UID storage scheme converter
 .
-.TH mdconvert 1 "2004 Mar 27"
+.TH mdconvert 1 @RELEASE_DATE@ "@PACKAGE_STRING@" "User Commands"
 .
 .SH NAME
 mdconvert - Maildir mailbox UID storage scheme converter

[commit] master: generalize AUTHORS section of man page

[ossi <os...@us...>]

commit 4279aea6a0bb27c6998dadfa77e0c3435a09407e
Author: Oswald Buddenhagen <os...@us...>
Date:   Fri Aug 2 10:03:11 2024 +0200

    generalize AUTHORS section of man page

 src/mbsync.1.in | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/src/mbsync.1.in b/src/mbsync.1.in
index 89c7a4a..759dbf1 100644
--- a/src/mbsync.1.in
+++ b/src/mbsync.1.in
@@ -884,4 +884,4 @@ http://www.iana.org/assignments/sasl-mechanisms/sasl-mechanisms.xhtml
 .SH AUTHORS
 Originally written by Michael R. Elkins,
 rewritten and currently maintained by Oswald Buddenhagen,
-contributions by Theodore Y. Ts'o.
+contributions by many others; see the AUTHORS file for details.

[commit] master: automate setting package version

[ossi <os...@us...>]

commit 8421b3cb223eced0e07ae65eaf21486d22beb573
Author: Oswald Buddenhagen <os...@us...>
Date:   Fri Aug 2 09:56:02 2024 +0200

    automate setting package version
    
    this avoids the need for bumping the version, which is particularly
    helpful if one doesn't know yet whether the next release will be a
    patch, minor, or major.
    
    we cache the version extracted from git, which also provides a fallback
    for the case of somebody rebuilding configure from a tar-ball.
    
    note that it's impossible to determine the version at configure time, so
    after git-tagging you need to remember to run version.sh (or autoconf)
    prior to rolling a tar-ball.

 .gitignore   |  1 +
 Makefile.am  |  2 +-
 configure.ac |  3 ++-
 version.sh   | 36 ++++++++++++++++++++++++++++++++++++
 4 files changed, 40 insertions(+), 2 deletions(-)

diff --git a/.gitignore b/.gitignore
index 6b9c837..66a12ec 100644
--- a/.gitignore
+++ b/.gitignore
@@ -1,6 +1,7 @@
 /.autoconf_trace
 /ChangeLog
 /INSTALL
+/VERSION
 /autom4te.cache/
 /aclocal.m4
 /autodefs.h
diff --git a/Makefile.am b/Makefile.am
index 901b4a5..79067cb 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -4,7 +4,7 @@
 
 SUBDIRS = src
 bin_SCRIPTS = mbsync-get-cert
-EXTRA_DIST = LICENSES debian isync.spec $(bin_SCRIPTS)
+EXTRA_DIST = LICENSES VERSION debian isync.spec $(bin_SCRIPTS)
 
 LOG_PL = \
     use POSIX qw(strftime); \
diff --git a/configure.ac b/configure.ac
index ee74d96..41670ca 100644
--- a/configure.ac
+++ b/configure.ac
@@ -2,7 +2,8 @@
 # SPDX-FileCopyrightText: 2002-2022 Oswald Buddenhagen <os...@us...>
 dnl SPDX-License-Identifier: GPL-2.0-or-later
 
-AC_INIT([isync], [1.5.0])
+m4_syscmd([./version.sh])
+AC_INIT([isync], m4_include([VERSION]))
 AC_CONFIG_HEADERS([autodefs.h])
 
 AC_CANONICAL_TARGET
diff --git a/version.sh b/version.sh
new file mode 100755
index 0000000..fc1ed7d
--- /dev/null
+++ b/version.sh
@@ -0,0 +1,36 @@
+#!/bin/sh
+# SPDX-FileCopyrightText: (C) 2024 Oswald Buddenhagen <os...@us...>
+# SPDX-License-Identifier: GPL-2.0-or-later
+
+cd $(dirname $0)
+
+test -e .git || exit
+
+mb=$(git merge-base HEAD "@{upstream}" 2> /dev/null)
+if test -z "$mb"; then
+	# we presume that a failure to find a merge base means no upstream.
+	# and no upstream may mean detached head in the middle of a rebase
+	br=$(git branch | sed -n -e 's/^\* (no branch, rebasing \([^\)]*\))$/\1/p')
+	if test -n "$br"; then
+		mb=$(git merge-base HEAD "$br@{upstream}" 2> /dev/null)
+	fi
+fi
+if test -z "$mb"; then
+	# still no upstream, so just describe HEAD as-is.
+	gver=$(git describe --tags HEAD)
+else
+	# find out whether we have local work, and if so, collapse it into
+	# a single suffix. otherwise, we'd cause pointless rebuilds during
+	# development.
+	gver=$(git describe --tags $mb)
+	lcl=$(git rev-list -n 1 $mb..HEAD)
+	if test -n "$lcl"; then
+		gver="$gver-plus"
+	fi
+fi
+gver=${gver#v}
+pgver=$(cat VERSION 2> /dev/null)
+if [ "x$gver" != "x$pgver" ]; then
+	echo "$gver" > VERSION
+fi
+

[commit] branch 'wip/gpl-exception' deleted

[ossi <os...@us...>]

The branch 'wip/gpl-exception', previously at 28a4636, has been
deleted.

Re: lost track of X messages / socket error

[Edd B. <ed...@th...>]

On Wed, Jul 31, 2024 at 07:20:38PM +0200, Oswald Buddenhagen wrote:
> so, after not even two years after i received the logs (:'-D), i finally
> analyzed them. posting for posterity.

Thanks for following this up!

For what it's worth, I haven't seen these problems again since our original
exchange.

I did change my router for one that does QoS to avoid buffer bloat, so perhaps
that helped...

-- 
Best Regards
Edd Barrett

https://www.theunixzoo.co.uk

[commit] master: generalize GPL exception

[ossi <os...@us...>]

commit f467b57a957672a02cd7d923603a8a9da7f0f9fc
Author: Oswald Buddenhagen <os...@us...>
Date:   Sun Jun 26 12:59:44 2022 +0200

    generalize GPL exception
    
    we have explicit approval from:
    - Anton Khirnov <an...@kh...>
    - Jeremy Katz <ka...@fe...>
    - Jesse Weaver <pia...@gm...>
    - Marc Hoersken <in...@ma...>
    - Michael J Gruber <mi...@gr...>
    - Noa Resare <no...@re...> (formerly Daniel)
    - Oliver Runge <oli...@gm...>
    - Patrick Steinhardt <ps...@pk...>
    - Theodore Ts'o <ty...@mi...>
    
    notably missing approval from:
    - Michael Elkins <me...@si...>
    
    further missing approval from:
    - Eivind Eklund <ei...@Fr...>
    - Georgy Kibardin <ge...@ki...>
    - Jack Stone <jwj...@fa...>
    - Jan Synacek <jsy...@re...>
    
    still, because
    - isync already contains an exception for OpenSSL,
    - every contributor implicitly agreed to that exception, and
    - that exception exists specifically because of the advertising clause
      in OpenSSL < v3's license,
    i'm assuming that the MIA contributors are actually fine with the
    proposed change.

 LICENSES/LicenseRef-isync-GPL-exception.txt | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/LICENSES/LicenseRef-isync-GPL-exception.txt b/LICENSES/LicenseRef-isync-GPL-exception.txt
index 894d89c..741e349 100644
--- a/LICENSES/LicenseRef-isync-GPL-exception.txt
+++ b/LICENSES/LicenseRef-isync-GPL-exception.txt
@@ -8,5 +8,5 @@ Usage-Guide:
     SPDX-License-Identifier: <SPDX-License> WITH LicenseRef-isync-GPL-exception
 License-Text:
 
-As a special exception, mbsync may be linked with the OpenSSL library,
-despite that library's more restrictive license.
+As a special exception, mbsync may be linked with libraries with a more
+restrictive license if the only incompatibility are advertising clauses.