Wildcard cert not accepted
Failing to fetch complete mailbox from Yahoo!
the culprit is the server returning partial fetch responses, which are tagged with [LIMIT] . this obviously completely derails isync. yahoo is attempting to standardize (a slightly more advanced version of) this behavior in rfc9738. to cite from this awesome document: If the server has a MESSAGELIMIT of 1000, the client will only be able to download 1000 of the most recent messages; the client will not understand why, will not be prepared to recover from the situation, and will act as if it is interacting...
I've been unable to reproduce it using Dry Run. After several attempts, it always finishes with: Channels: 1 Boxes: 20 Far: +0 *0 #0 -0 Near: +122976 *0 #0 -0 Which looks like the correct grand total for the entire remote store (i.e inclusive of other folders, besides INBOX). So I haven't been able to reproduce the weird cap at 2000, or the message deletions. Some more context: I was originally using version 1.4.4-5. Over the course of a dozen or so attempts to synchronise the entire Yahoo! store...
this sounds bad. it seems that it thinks that the downloads were completed, so it treats the missing mails as deletions. and it somehow fails to see 70k+ messages. to debug this, i need the full log (with -D) of both the initial sync attempt and the destructive followup. that means that you need to start from scratch locally. do the second run with --dry-run, which should be good enough without causing actual damage. you can mail me the logs in private.
Failing to fetch complete mailbox from Yahoo!
Failing to fetch complete mailbox from Yahoo!
Zed editor with a deepseek chat model to be fair, it did a lot of capability and domain checks, i asked it to remove those
which coding agent exactly was that? the patch looks about correct, except that the actual ID string is mildly stupid. i'll have to check the spec for the correct fields and command order.
I Asked Coding Agent to fix this for me. here's what I got: it a/src/drv_imap.c b/src/drv_imap.c index 9b170ab..7b7f607 100644 --- a/src/drv_imap.c +++ b/src/drv_imap.c @@ -9,6 +9,7 @@ #include "imap_p.h" #include "socket.h" +#include "config.h" #include <ctype.h> #include <sys/wait.h> @@ -2088,6 +2089,8 @@ static void imap_open_store_authenticate_p3( imap_store_t *, imap_cmd_t *, int ) static void imap_open_store_authenticate2( imap_store_t * ); static void imap_open_store_authenticate2_p2( imap_store_t...
I Asked Coding Agent to fix this for me. here's what I got: it a/src/drv_imap.c b/src/drv_imap.c index 9b170ab..7b7f607 100644 --- a/src/drv_imap.c +++ b/src/drv_imap.c @@ -9,6 +9,7 @@ #include "imap_p.h" #include "socket.h" +#include "config.h" #include <ctype.h> #include <sys/wait.h> @@ -2088,6 +2089,8 @@ static void imap_open_store_authenticate_p3( imap_store_t *, imap_cmd_t *, int ) static void imap_open_store_authenticate2( imap_store_t * ); static void imap_open_store_authenticate2_p2( imap_store_t...
they only offer this, as @shanyi posted in the link(https://help.mail.163.com/faqDetail.do?code=d7a5dc8471cd0c0e8b4b8f4f8e49998b374173cfe9171305fa1ce630d7f67ac2eda07326646e6eb0): ********************代码块区域开始******************** Properties props = new Properties(); props.setProperty("mail.store.protocol", "imap"); props.setProperty("mail.imap.host", "imap.163.com"); props.setProperty("mail.imap.port", "143"); HashMap IAM = new HashMap(); //带上IMAP ID信息,由key和value组成,例如name,version,vendor,support-email等。...
126.com, 163.com and possibly 188.com. from the response I get, I don't think they have any plan to fix this though. I couldn't even get to the tech team
add support for imap id extension
so it appears that the various netease domains weren't fixed ....
I have written an email to kefu@188.com on this issue, In the mean time, would you consider implement this?
make imap_msgs test actually return non-zero on failure
clarify that maildir message moves should preserve the flag field
mention that plain --debug excludes driver-all as well
fix conditional on undefined value on some config syntax errors
fix crash in mkdir_p() with invalid path
please attach (or mail me) your config file and the logs of both runs, but with the -D option added. make sure that each run would actually have something to upgrade.
Upgrades do not work when synchronizing whole channel
Update spec file
Use GNOME's secret-tool rather gnome-keyring-query
you may want to give the wip/includecmd branch a shot.
something that has been requested exactly once so far I was looking for this too, and i'm sure many others were as well
Add IncludeCmd directive to config parser
Refactor printing configuration errors
add support for single-quoting to our printf implementation
add unit test for our printf implementation
make imap_msgs test actually return non-zero on failure
Add IncludeCmd directive to config parser
Add IncludeCmd directive to config parser
Refactor printing configuration errors
fix conditional on undefined value on some config syntax errors
That was it. For some unknown reason I had an INBOX and an Inbox in the local machine, both empty. There was no counterpart in the mail server. I removed them and the warnings are gone. Furthermore, the summary line now says the expected number of boxes. Thanks!
That was it. For some unknown reason I had an INBOX and an Inbox in the local machine, both empty. There was no counterpart in the mail server. I removed them and the warnings are gone. Furthermore, the summary line now says the expected number of boxes.
the warning suggests that you have a literal "INBOX" folder in your maildir, next to the "Inbox" you actually want. you should delete it after making sure there is nothing worthwhile in it. the error is more mystifying, as this shouldn't be even possible with MapInbox. that might be a genuine bug. however, your summary line says 4 boxes, while your pattern specifies only 3, so something is inconsistent here.
This is my current mbsyncrc: Expunge Both Create Both MaildirStore emlocal Path ~/Maildir/ Inbox ~/Maildir/ MapInbox Inbox ImapStore em Host em User ... Pass ... CertificateFile /etc/ssl/certs/ca-certificates.crt TLSVersions +1.2 MapInbox Inbox Timeout 80 Channel emPriority Far :em: Near :emlocal: Pattern Inbox mbox Sent When I use it I get the message: $ mbsync emPriority Maildir warning: ignoring INBOX in /home/mochan/Maildir/ Error: channel emPriority: far side box INBOX cannot be opened anymore....
you should cut it down to what is actually needed. if some weirdness persists at this point, we can investigate.
possible test failures (linux)
Thanks a lot. If I cd src first, then all tests pass indeed.
make check does indeed run only the unit tests, while the integration test needs to be run manually. this has historical reasons, but given how slow the latter is, i am hesitant to change things. the test expects to be run from within the src directory with ./run-tests.pl.
possible test failures (linux) (v1.5.1)
Many thanks.
I'm sorry; kind of short of time, so I returned to the configuration I showed above on 2025-01-26. The system has continued working but for the warnings Maildir warning: ignoring INBOX in /home/mochan/Maildir/ Error: channel emPriority: far side box INBOX cannot be opened. I haven't understood yet the situation, but my inboxes in the remote and local machine and my other mailboxes are being correctly synchronized.
I'm sorry; kind of short of time, so I returned to the configuration I showed above on 2025-01-26. The system has continued working but for the warnings Maildir warning: ignoring INBOX in /home/mochan/Maildir/ Error: channel emPriority: far side box INBOX cannot be opened.
release preparation
expand NEWS
stub out ChangeLog
--quiet is not suppressing some notices
Dry run results in duplicate emails
Path cannot be nested under Inbox
as there were no further followups, i'm assuming that the configuration issue was resolved. i had a quick look at the code checking that Path is configured, and found that it should be triggered only if Path is actually referenced somehow, f.ex. via a Pattern. so i'm assuming that this issue was part of the bigger misconfiguration. please post a minimal reproducer if that is incorrect.
the results of mbsync -ls -D before updating my .mbsyncrc: isync 1.5.0 called with: '-ls' '-D' Reading configuration file /home/mochan/.mbsyncrc merge ops (in Channel 'icf'): common: OP_NEW,OP_OLD,OP_UPGRADE,OP_GONE,OP_FLAGS,OP_EXPUNGE far: XOP_HAVE_TYPE,XOP_HAVE_EXPUNGE,XOP_HAVE_CREATE near: OP_CREATE => far: OP_NEW,OP_OLD,OP_UPGRADE,OP_GONE,OP_FLAGS,OP_EXPUNGE,XOP_HAVE_TYPE,XOP_HAVE_EXPUNGE,XOP_HAVE_CREATE => near: OP_NEW,OP_OLD,OP_UPGRADE,OP_GONE,OP_FLAGS,OP_EXPUNGE,OP_CREATE merge ops (in global...
Hi, Thanks for your answer. I configured my mbsync more than a decade ago, so you're right that I don't actually understand what Inbox and INBOX are or ought to be. In my local machine I have a ~/Maildir/ directory with its usual cur/ new/ tmp/ subdirectories, and that is my actual inbox, where I receive nuew mail. Furthermore, I have many subdirectories, such as mbox, mbox2023, mbox2023... where I write emails I have already processed. In my configuration I have a couple of channels to either synchronice...
prefix deprecation notices with file name
presumably, commit 1e0b661d in master fixes this.
Dry run results in duplicate emails
the mapping looks just fine, no? that it wants to both push and pull is a classic indication that no sync state is present (or has been found, in your case). testing this with dummy accounts seems rather pointless, as that will just sync from scratch anyway. then you can just throw away your maildirs and sync state, and start afresh. so just move the sync state files in place and try with -y (preferably after you install git master). cp -al the local store somewhere, just in case.
I guess I will make a couple of dummy accounts before testing that. However, I found I made an erroneous interpretation: The 6K+ messages that mbsync would fetch are from mbox, not from INBOX. The outputs of mbsync -l are $ mbsync -c mbsyncrc_new -l emCon INBOX <=> INBOX $ mbsync -c mbsyncrc_new -l emSin mbox <=> INBOX/mbox $ mbsync -c mbsyncrc_new -l ems emSin: mbox <=> INBOX/mbox emCon: INBOX <=> INBOX I also tried to change Path to a link to Maildir, but it didn't work so I reverted my new configuration...
whoops, yeah, it obviously should be Near :emlocal:INBOX/. yes, you can (and should) delete the bogus folders. as you're not using SyncState *, you should also clean out corresponding state files from the sync state directory. ah, the SyncState thing is also why it wants to fetch the inbox from scratch - the logical folder names changed, so the state file names don't match anymore. you can simply rename them to match (considering that / is translated to !), but i would recommend that you actually...
$ mbsync -c mbsyncrc_new -l ems emSin: mbox <=> INBOX.mbox emCon: INBOX <=> INBOX I don't know if it is correct. Locally I expected INBOX/mbox (see below) $ mbsync -c mbsyncrc_new -ls ===== emlocal: INBOX/mbox2000 INBOX/mbox2022 INBOX/Drafts INBOX/convocatorias ... INBOX/INBOX ... INBOX/Inbox INBOX ===== em: INBOX mbox2016 coloquio mbox2010 Firmas ... mboINBOX ... Seems correct, except for the funny INBOX/INBOX, mboINBOX, etc, which must have been created while doing tests but are empty. (I wonder...
well, the dots on the server are just directories as far as isync is concerned, provided the server tells it so according to spec. just try the new config with -l. if it's still wrong, you need to look at your paths and -ls output again. if you can't make sense of it, attach the outputs and the updated config.
Great! I still wonder about my configuration. I guess part of the problem is due to using different formats in my mail server (with dots) and in the client (with subdirectories). I haven't attempted a non-dry run with the configuration you suggested, but I don't want to saturate my laptop's disk with copies under 'Path' of emails I already have elsewhere. My current (incoherent) configuration seems to work, except for the error messages about non-existing Inboxes in the client and the server. Any...
i believe the -ls hang and the ineffective --dry-run to be now fixed in git master.
fix -ls hanging after synchronous error
fix --dry-run without --debug-driver
oh, right, now that you mention it, it's totally obvious: the "drying" is implemented in the proxy driver which implements -Dd. fix upcoming ...
The output from mbsync -c mbsyncrc_new -y -Dd -V ems >rem.txt is attached below
It did. If I run with -Dd and -y it doesn't, but if run with -y it does: $ rm -r mbsyncDummyPath/ $ mkdir mbsyncDummyPath/ $ du -s mbsyncDummyPath/ 4 mbsyncDummyPath/ $ mbsync -c mbsyncrc_new -y -Dd -V ems ... Processed 2 box(es) in 2 channel(s), would pull 6284 new message(s) and 0 flag update(s), would expunge 0 message(s) from near side, would push 0 new message(s) and 0 flag update(s), would expunge 0 message(s) from far side. $ du -s mbsyncDummyPath/ 4 mbsyncDummyPath/ $ mbsync -c mbsyncrc_new...
but did it actually copy anything? the dry run is intentionally very thorough in pretending that it does something. adding -Dd will show what is actually happening.
Hi. Before trying your full suggestion I made one of the new channels Channel emSin Far :em: Near :emlocal:INBOX. Patterns mbox using the emlocal store you suggested before but specifying just one Pattern to test MaildirStore emlocal Inbox ~/Maildir SubFolders Verbatim Path ~/mbsyncDummyPath/ adding an empty directory as Path, and made a dry run of mbsync with the new configuration for just this channel $ mbsync -c mbsyncrc_new -y -V emSin Reading configuration file ./mbsyncrc_new Channel emSin Opening...
you probably need something like Channel em Far :em: Near :emlocal:INBOX. Patterns mbox% ... Channel emInbox Far :em: Near :emlocal: Group ems Channels em emInbox hmm, indeed, Path cannot be omitted with SubFolders Verbatim. that's actually kinda a bug. just set it to something useless like ~/tmp.
By the way, with the new config mbsync -ls doesn't hang. mbsync -l -a complains that my local store has no Path
Question: After following your suggestions above for the local store, mbsync -ls shows my local store has boxes such as INBOX, INBOX/mbox, INBOX/..., while in the remote store I get INBOX, mbox, ... (without INBOX/ preceeding the names of the other boxes). If I remove MapInbox from the remote store config the results look similar (i.e., I get INBOX/mbox in the local store but just mbox in the remote store). Thus, what should I put in patterns? Something Pattern INBOX INBOX/mbox% ... or Pattern INBOX...
ok, thanks. looks like i just failed to wire up the error cleanup path callbacks properly.
I attach below the results of mbsync -ls -D before updating my .mbsyncrc
I attach below the results of mbsync -ls -D before updating my .mbsyncrc
yeah, that config won't work anymore, because it's incoherent. however, you can adjust the config to actually match the physical layout. you only need to make it more similar to the other store: unset Path and MapInbox, set Inbox ~/Maildir, and SubFolders Verbatim. then adjust the Patterns to the resulting new layout. make sure to test with -ls and -l before you run an actual sync, as otherwise you'll get duplicate boxes if something doesn't match. the hang is definitely a bug, so keep a copy of...
Thanks, I reply in the attached text file
first you should make sure that you actually understand what Inbox and INBOX are. then you can assess how this relates to your actual setup, and from there determine how to fix or simply suppress the problem. you probably also shouldn't be using MapInbox. useful diagnostic steps are running mbsync -ls and later mbsync -l -a, but that might require at least suppressing the issue at first. if you need further assistance, attach your config and the output of the first command.
Path cannot be nested under Inbox
explicitly strip newline from contents of VERSION
Update https://isync.sourceforge.io/mbsync.html
Update https://isync.sourceforge.io/mbsync.html
Good idea. Did not cross my mind somehow! Next time! :) December 9, 2024 at 7:26 AM, "Huy" <huyz@users.sourceforge.net mailto:huyz@users.sourceforge.net?to=%22Huy%22%20%3Chuyz%40users.sourceforge.net%3E > wrote: @Githubbie, out of curiosity, since docker-mailserver's Dovecot is backed by Maildir by default, why not just do a FreeFileSync or Resilio Sync (or rsync or even straight cp/scp if this is a one-off migration) to do a Maildir-to-Maildir transfer? I imagine it'd be faster and possibly have...
@Githubbie, out of curiosity, since docker-mailserver's Dovecot is backed by Maildir by default, why not just do a Syncthing or Resilio Sync (or rsync/rclone/unison or even straight cp/scp if this is a one-off migration) to do a Maildir-to-Maildir transfer? I imagine it'd be faster and possibly have higher fidelity
@Githubbie, out of curiosity, since docker-mailserver's Dovecot is backed by Maildir by default, why not just do a Syncthing or Resilio Sync (or rsync or even straight cp/scp if this is a one-off migration) to do a Maildir-to-Maildir transfer? I imagine it'd be faster and possibly have higher fidelity
@Githubbie, out of curiosity, since docker-mailserver's Dovecot is backed by Maildir by default, why not just do a FreeFileSync or Resilio Sync (or rsync or even straight cp/scp if this is a one-off migration) to do a Maildir-to-Maildir transfer? I imagine it'd be faster and possibly have higher fidelity
build: also consider builds off of git with `git clone --depth 1`
build: also consider builds off of git with `git clone --depth 1`
Large Maildir Synced fast for first 78% of ~600k message, now at snails pace one per email for rest
ok, thanks for the update.
What I found from using docker mailserver is that I can get past these issues by setting a mailbox quota and then using PipelineDepth 1 Somehow that all worked. BUT, I think this is a good converstaino for anyone who might have similar issues. Feel free to close.
mbsync keeps crashing