isync-devel — All-purpose list, actually - but RTFM first. Moderated for non-subscribers for SPAM protection.

Re: [commit] master: build: also consider builds off of git with `git clone --depth 1`

[Oswald B. <os...@us...>]

On Wed, Dec 04, 2024 at 11:15:29AM -0500, Paymon wrote:
>spoke to early; it's git-branch that returns something like this:
>* (HEAD detached at refs/git-r3/HEAD)
>
>that escapes our regex.
>
>i would pass `--always` unconditionally to be defensive about it; what do you
>think?
>
i did that, with a forced push, because i'm evil like that.

but maybe this could be still handled better? i guess not, as the branch 
name above is meaningless, and presumably no other refs are present?  
does gitk --all show anything sensible in this repo?

Re: [commit] master: build: also consider builds off of git with `git clone --depth 1`

[Paymon <dar...@gm...>]

you beat me to it; please discard the other patch.

-- 

               Paymon

build: take count of detached HEAD

[Paymon <dar...@gm...>]

-- 

               Paymon

[commit] master: build: also consider builds off of git with `git clone --depth 1`

[ossi <os...@us...>]

commit 884413b48871361cf6c0cfa41d01c22ed1ec8d30
Author:     Paymon MARANDI <dar...@gm...>
AuthorDate: Tue Dec 3 15:25:09 2024 -0500
Commit:     Oswald Buddenhagen <os...@us...>
CommitDate: Wed Dec 4 17:20:49 2024 +0100

    build: also consider builds off of git with `git clone --depth 1`
    
    one case where this could happen is a shallow clone, purely for build
    purposes. in source based distros, like gentoo, setting depth of the
    clone to 1, globally, is a common configuration. this patch avoids that
    those builds fail.
    
    Signed-off-by: Paymon MARANDI <dar...@gm...>

 version.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/version.sh b/version.sh
index fc1ed7d..669f1b6 100755
--- a/version.sh
+++ b/version.sh
@@ -17,12 +17,12 @@ if test -z "$mb"; then
 fi
 if test -z "$mb"; then
 	# still no upstream, so just describe HEAD as-is.
-	gver=$(git describe --tags HEAD)
+	gver=$(git describe --always --tags HEAD)
 else
 	# find out whether we have local work, and if so, collapse it into
 	# a single suffix. otherwise, we'd cause pointless rebuilds during
 	# development.
-	gver=$(git describe --tags $mb)
+	gver=$(git describe --always --tags $mb)
 	lcl=$(git rev-list -n 1 $mb..HEAD)
 	if test -n "$lcl"; then
 		gver="$gver-plus"

[commit] branch 'master' rewound

[ossi <os...@us...>]

The branch 'master', previously at 8920b6a, has been rewound by 1
revision(s) to 15c7e02.

Re: [commit] master: build: also consider builds off of git with `git clone --depth 1`

[Paymon <dar...@gm...>]

On Wed, Dec 04, 2024 at 10:50:43AM -0500, Paymon wrote:
> On Wed, Dec 04, 2024 at 08:41:44AM +0000, ossi via isync-devel wrote:
> > commit 8920b6a1db443aac00783f7754a05852bb1bc71c
> > Author:     Paymon MARANDI <dar...@gm...>
> > AuthorDate: Tue Dec 3 15:25:09 2024 -0500
> > Commit:     Oswald Buddenhagen <os...@us...>
> > CommitDate: Wed Dec 4 09:36:28 2024 +0100
> > 
> >     build: also consider builds off of git with `git clone --depth 1`
> >     
> >     one case where this could happen is a shallow clone, purely for build
> >     purposes. in source based distros, like gentoo, setting depth of the
> >     clone to 1, globally, is a common configuration. this patch avoids that
> >     those builds fail.
> >     
> >     Signed-off-by: Paymon MARANDI <dar...@gm...>
> > 
> >  version.sh | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/version.sh b/version.sh
> > index fc1ed7d..5dfc61b 100755
> > --- a/version.sh
> > +++ b/version.sh
> > @@ -22,7 +22,7 @@ else
> >  	# find out whether we have local work, and if so, collapse it into
> >  	# a single suffix. otherwise, we'd cause pointless rebuilds during
> >  	# development.
> > -	gver=$(git describe --tags $mb)
> > +	gver=$(git describe --always --tags $mb)
> >  	lcl=$(git rev-list -n 1 $mb..HEAD)
> >  	if test -n "$lcl"; then
> >  		gver="$gver-plus"
> > 
> > 
> 
> it turns out that this doesn't cover the gentoo case. $mb is always going to
> come up empty! i should have tested... i will submit a revised version.
> 
> -- 
> 
>                Paymon

spoke to early; it's git-branch that returns something like this:
* (HEAD detached at refs/git-r3/HEAD)

that escapes our regex.

i would pass `--always` unconditionally to be defensive about it; what do you
think?

-- 

               Paymon

Re: [commit] master: build: also consider builds off of git with `git clone --depth 1`

[Oswald B. <os...@us...>]

On Wed, Dec 04, 2024 at 10:50:43AM -0500, Paymon wrote:
>> +++ b/version.sh
>> @@ -22,7 +22,7 @@ else
>>  	# find out whether we have local work, and if so, collapse it into
>>  	# a single suffix. otherwise, we'd cause pointless rebuilds during
>>  	# development.
>> -	gver=$(git describe --tags $mb)
>> +	gver=$(git describe --always --tags $mb)
>>  	lcl=$(git rev-list -n 1 $mb..HEAD)
>>  	if test -n "$lcl"; then
>>  		gver="$gver-plus"
>> 
>> 
>
>it turns out that this doesn't cover the gentoo case. $mb is always going to
>come up empty! i should have tested... i will submit a revised version.
>
interesting.
i originally thought that the other branch should be covered, but then i 
tested it with a local shallow clone, and the merge base was always the 
commit itself.
what's the difference?

Re: [commit] master: build: also consider builds off of git with `git clone --depth 1`

[Paymon <dar...@gm...>]

On Wed, Dec 04, 2024 at 08:41:44AM +0000, ossi via isync-devel wrote:
> commit 8920b6a1db443aac00783f7754a05852bb1bc71c
> Author:     Paymon MARANDI <dar...@gm...>
> AuthorDate: Tue Dec 3 15:25:09 2024 -0500
> Commit:     Oswald Buddenhagen <os...@us...>
> CommitDate: Wed Dec 4 09:36:28 2024 +0100
> 
>     build: also consider builds off of git with `git clone --depth 1`
>     
>     one case where this could happen is a shallow clone, purely for build
>     purposes. in source based distros, like gentoo, setting depth of the
>     clone to 1, globally, is a common configuration. this patch avoids that
>     those builds fail.
>     
>     Signed-off-by: Paymon MARANDI <dar...@gm...>
> 
>  version.sh | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/version.sh b/version.sh
> index fc1ed7d..5dfc61b 100755
> --- a/version.sh
> +++ b/version.sh
> @@ -22,7 +22,7 @@ else
>  	# find out whether we have local work, and if so, collapse it into
>  	# a single suffix. otherwise, we'd cause pointless rebuilds during
>  	# development.
> -	gver=$(git describe --tags $mb)
> +	gver=$(git describe --always --tags $mb)
>  	lcl=$(git rev-list -n 1 $mb..HEAD)
>  	if test -n "$lcl"; then
>  		gver="$gver-plus"
> 
> 

it turns out that this doesn't cover the gentoo case. $mb is always going to
come up empty! i should have tested... i will submit a revised version.

-- 

               Paymon

Re: IMAP Timeouts During Local Operations—Correct Behaviour?

[Oswald B. <osw...@gm...>]

On Tue, Dec 03, 2024 at 04:12:09PM +0000, Sabahattin Gucukoglu via isync-devel wrote:
>So my question is this: is it expected behaviour that timeouts occur
>because of long local operations,
>
yes, sort of.

>and is it *correct* or desirable behaviour?
>
certainly not desirable.

>Isn’t the “right thing” here to have a dedicated idle timeout, which
>would take effect while isync is doing purely local computation?
>
is that timeout reported by isync or the server? what's the end of the
log with -Dn?

if it's the server, then it's quite some work to fix - the maildir
driver would have to be made asynchronous (possibly threaded).

>I am successfully working around this situation by using a separate
>config file for the duration of the initial upload [...]
>
how is it different?

my first idea would be to limit the operations to the minimum, in your
case that should be --push-new if i understand your case correctly.

secondly, if the problem is local i/o and you're pushing, then copying
the box to a ram drive might be a solution. (make sure to move the state
file into place when switching back to the proper box.)

did you try to identify what the time is actually spent on? i'd first
try htop and iotop, and then for finer results valgrind/cachegrind and
strace -tt, resp.

[commit] master: build: also consider builds off of git with `git clone --depth 1`

[ossi <os...@us...>]

commit 8920b6a1db443aac00783f7754a05852bb1bc71c
Author:     Paymon MARANDI <dar...@gm...>
AuthorDate: Tue Dec 3 15:25:09 2024 -0500
Commit:     Oswald Buddenhagen <os...@us...>
CommitDate: Wed Dec 4 09:36:28 2024 +0100

    build: also consider builds off of git with `git clone --depth 1`
    
    one case where this could happen is a shallow clone, purely for build
    purposes. in source based distros, like gentoo, setting depth of the
    clone to 1, globally, is a common configuration. this patch avoids that
    those builds fail.
    
    Signed-off-by: Paymon MARANDI <dar...@gm...>

 version.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/version.sh b/version.sh
index fc1ed7d..5dfc61b 100755
--- a/version.sh
+++ b/version.sh
@@ -22,7 +22,7 @@ else
 	# find out whether we have local work, and if so, collapse it into
 	# a single suffix. otherwise, we'd cause pointless rebuilds during
 	# development.
-	gver=$(git describe --tags $mb)
+	gver=$(git describe --always --tags $mb)
 	lcl=$(git rev-list -n 1 $mb..HEAD)
 	if test -n "$lcl"; then
 		gver="$gver-plus"

Re: Suddenly errors regarding "near side box INBOX/XXX cannot be opened."

[Johannes K. <ma...@oj...>]

Hi Oswald,

On 02.12.24 11:28 Oswald Buddenhagen via isync-devel wrote:
> On Sun, Dec 01, 2024 at 08:47:48PM +0100, Johannes Kastl wrote:
>>> Error: channel YYY: near side box INBOX/XXX cannot be opened.
>>
>>Funnily enough, in one case this happened after the folder was created
>>locally and the sync succeeded. On the next run the error above was shown.
>>
> weird. try 'mbsync -ls' for starters.
> if that looks good, try 'mbsync -l'.
> if that still looks good, it's presumably not a user error.
> in that case try syncing only one failing mailbox (YYY:INBOX/XXX).
> add -D to the above command and the normal YYY run, und mail me the
> logs.

Will try to do some more debugging.

>>I also noticed that my GMX mailboxes suddenly stop using Entw&rfe
>>locally, but use the proper German "Entwürfe". I am not sure if this
>>is related or something that has changed on the provider's side?
>>
> v1.5 gained proper support for UTF-7 & UTF-8, so your pre-existing local
> folders need to be renamed.
> it's possible that this is in fact the root cause of the above problem,
> but that depends on your exact config.

Only the name of the drafts folder changed, the folder that cannot be
synced have not changed and still have the same name.

Kind Regards,
Johannes

Re: [PATCH] build: also consider builds off of git with `git clone --depth 1`

[Paymon <dar...@gm...>]

On Sat, Nov 30, 2024 at 11:55:33AM +0100, Oswald Buddenhagen via isync-devel wrote:
> On Fri, Nov 29, 2024 at 06:21:45PM -0500, Paymon MARANDI wrote:
> > any feedback?
> > 
> yeah, sorry, i've been procrastinating and then it went out of view ...
> 
> i don't like it, because it makes the version info useless, which would
> make worse log files. adding --always to the describe command seems less
> disruptive.

oh, didn't know about that one.

> the use case also feels a bit silly - given that the worktree is over
> half a meg and the build tree several times that much, shrinking the
> pack from just 2 megs to 200k seems mostly pointless.

i updated the log, hopefully will add proper justification.

-- 

               Paymon

IMAP Timeouts During Local Operations—Correct Behaviour?

[Sabahattin G. <lis...@me...>]

Hi.

I’m trying to stuff a new maildir folder containing 1m+ messages (gatewayed from news) onto my IMAP server. Notwithstanding the usual *slowness* of that server (Apple IMAP, so maybe not the best choice), it’s usual for a timeout of 30 seconds to work in normal operation, getting neatly around stalls that occur for seemingly random reasons despite the fact that I’m well-connected from a fixed connection. (Insert rant about NAT and firewalls and other evil middleboxes here.) But while initially synchronising the mailbox, even before the synchronising stage has identified all the far-side messages to be appended, and before the first append has taken place, the 30 sec timeout repeatedly occurs with every subsequent invocation of mbsync from launchd.

So my question is this: is it expected behaviour that timeouts occur because of long local operations, and is it *correct* or desirable behaviour? Isn’t the “right thing” here to have a dedicated idle timeout, which would take effect while isync is doing purely local computation?

I am successfully working around this situation by using a separate config file for the duration of the initial upload which I am running interactively (and yes, the server really *is* slow). Once that’s done, I should be able to revert to my usual configuration, and if my understanding of the loading/sync process is right, subsequent incremental additions should be relatively fast because isync only needs information from the directory listing, won’t need to rename large numbers of files, and won’t spend a long time calculating the pending operations to be done.

This will all probably turn out to be a terrible idea and I really should just find other ways to examine the mailbox and slim it down. But I’d love to know what you think.

Cheers,
Sabahattin

Re: Suddenly errors regarding "near side box INBOX/XXX cannot be opened."

[Oswald B. <osw...@gm...>]

On Sun, Dec 01, 2024 at 08:47:48PM +0100, Johannes Kastl wrote:
>> Error: channel YYY: near side box INBOX/XXX cannot be opened.
>
>Funnily enough, in one case this happened after the folder was created
>locally and the sync succeeded. On the next run the error above was shown.
>
weird. try 'mbsync -ls' for starters.
if that looks good, try 'mbsync -l'.
if that still looks good, it's presumably not a user error.
in that case try syncing only one failing mailbox (YYY:INBOX/XXX).
add -D to the above command and the normal YYY run, und mail me the
logs.

>I also noticed that my GMX mailboxes suddenly stop using Entw&rfe
>locally, but use the proper German "Entwürfe". I am not sure if this
>is related or something that has changed on the provider's side?
>
v1.5 gained proper support for UTF-7 & UTF-8, so your pre-existing local
folders need to be renamed.
it's possible that this is in fact the root cause of the above problem,
but that depends on your exact config.

Suddenly errors regarding "near side box INBOX/XXX cannot be opened."

[Johannes K. <ma...@oj...>]

Dear all,

today I noticed that some of my mbsync calls are returning errors.

> Error: channel YYY: near side box INBOX/XXX cannot be opened.

Funnily enough, in one case this happened after the folder was created
locally and the sync succeeded. On the next run the error above was shown.

The command I am using is:

mbsync --pull -c ~/.config/mbsync/YYY.conf YYY

The configuration was only changed recently to replace SSLType with
TLSType and remove the TLSVersions I had previously. But after that it
was already working properly.

I do not see any obvious permission issues in the file system.

Side note:
I also noticed that my GMX mailboxes suddenly stop using Entw&rfe
locally, but use the proper German "Entwürfe". I am not sure if this
is related or something that has changed on the provider's side?

Did anyone experience something similar? Is this a known issue that I
overlooked?

I'll do some more digging and debugging in the next couple of days,
once I find time.

Have a nice day!

Kind Regards,
Johannes

Re: [PATCH] build: also consider builds off of git with `git clone --depth 1`

[Oswald B. <osw...@gm...>]

On Fri, Nov 29, 2024 at 06:21:45PM -0500, Paymon MARANDI wrote:
>any feedback?
>
yeah, sorry, i've been procrastinating and then it went out of view ...

i don't like it, because it makes the version info useless, which would
make worse log files. adding --always to the describe command seems less
disruptive.

the use case also feels a bit silly - given that the worktree is over
half a meg and the build tree several times that much, shrinking the
pack from just 2 megs to 200k seems mostly pointless.

Re: [PATCH] build: also consider builds off of git with `git clone --depth 1`

[Paymon M. <dar...@gm...>]

resending as attachment
-- 

               Paymon

Re: [PATCH] build: also consider builds off of git with `git clone --depth 1`

[Paymon M. <dar...@gm...>]

any feedback?

-- 

               Paymon

[commit] master: accept zero-sized messages from IMAP

[ossi <os...@us...>]

commit 15c7e02e4a5fe127bd55bce2b9aa63a5da8eb228
Author: Oswald Buddenhagen <os...@us...>
Date:   Sun Nov 24 13:11:32 2024 +0100

    accept zero-sized messages from IMAP
    
    while such a thing is rather pointless, completely empty messages are
    not forbidden. consequently, we should be able to deal with them, and
    above all not crash.

 src/drv_imap.c | 5 ++++-
 src/socket.c   | 1 -
 src/util.c     | 5 +++++
 3 files changed, 9 insertions(+), 2 deletions(-)

diff --git a/src/drv_imap.c b/src/drv_imap.c
index 1280e3c..b0be1fd 100644
--- a/src/drv_imap.c
+++ b/src/drv_imap.c
@@ -780,6 +780,8 @@ parse_imap_list( imap_store_t *ctx, char **sp, parse_list_state_t *sts )
 				if (sts->callback->atom( ctx, NULL, bytes, AtomChunkedLiteral ) != LIST_OK)
 					goto bail;
 				sts->in_literal = ChunkedLiteral;
+				if (!bytes)
+					goto nobytes;
 				sts->big_literal = 1;
 			  get_chunked:
 				n = 1;
@@ -806,7 +808,7 @@ parse_imap_list( imap_store_t *ctx, char **sp, parse_list_state_t *sts )
 				} else if (DFlags & DEBUG_NET_ALL) {
 					printf( "%s=========\n", ctx->label );
 					fwrite( p, n, 1, stdout );
-					if (p[n - 1] != '\n')
+					if (n && p[n - 1] != '\n')
 						fputs( "\n(no-nl) ", stdout );
 					printf( "%s=========\n", ctx->label );
 				} else {
@@ -819,6 +821,7 @@ parse_imap_list( imap_store_t *ctx, char **sp, parse_list_state_t *sts )
 			bytes -= n;
 			if (bytes > 0)
 				goto postpone;
+		  nobytes:
 			if (sts->in_literal == ChunkedLiteral && sts->callback->atom( ctx, NULL, 0, AtomLiteral ) != LIST_OK)
 				goto bail;
 		  getline:
diff --git a/src/socket.c b/src/socket.c
index afd3f18..1ef0e95 100644
--- a/src/socket.c
+++ b/src/socket.c
@@ -952,7 +952,6 @@ socket_expect_bytes( conn_t *conn, uint len )
 char *
 socket_read( conn_t *conn, uint min_len, uint max_len, uint *out_len )
 {
-	assert( min_len > 0 );
 	assert( min_len <= sizeof(conn->buf) );
 	assert( min_len <= max_len );
 
diff --git a/src/util.c b/src/util.c
index e697cfb..121ab0a 100644
--- a/src/util.c
+++ b/src/util.c
@@ -637,6 +637,11 @@ nfmalloc( size_t sz )
 {
 	void *ret;
 
+#ifndef __GLIBC__
+	// The C standard allows NULL returns for zero-sized allocations.
+	if (!sz)
+		sz = 1;
+#endif
 	if (!(ret = malloc( sz )))
 		oom();
 	return ret;

[commit] master: mbsync-get-cert: add support for STARTTLS

[ossi <os...@us...>]

commit d7305e12d9f348975eb0f1a29be3b9f9999d76d3
Author:     Behnam Lal <de...@be...>
AuthorDate: Sun Sep 29 14:35:11 2024 +0200
Commit:     Oswald Buddenhagen <os...@us...>
CommitDate: Sun Nov 24 11:22:34 2024 +0100

    mbsync-get-cert: add support for STARTTLS
    
    nowadays, many servers offer STARTTLS on the default IMAP port 143
    instead of (or in addition to) the traditional IMAP over SSL/TLS (IMAPS)
    on port 993.
    
    this patch has been fixed up somewhat by the maintainer.

 mbsync-get-cert | 30 +++++++++++++++++++++++++++---
 1 file changed, 27 insertions(+), 3 deletions(-)

diff --git a/mbsync-get-cert b/mbsync-get-cert
index 19e1485..d8f194a 100755
--- a/mbsync-get-cert
+++ b/mbsync-get-cert
@@ -9,9 +9,25 @@
 # from a trusted source.
 #
 
-if [ $# != 1 ]; then
-	echo "Usage: $0 <host>" >&2
+usage() {
+	echo "Usage: $0 [-s] <host>" >&2
+	echo "  -s        Use IMAP+STARTTLS (port 143) instead of IMAPS (port 993)" >&2
 	exit 1
+}
+
+STARTTLS=false
+
+while getopts "s" opt; do
+	case $opt in
+	s) STARTTLS=true ;;
+	*) usage ;;
+	esac
+done
+
+shift `expr $OPTIND - 1`
+
+if [ $# -ne 1 ]; then
+	usage
 fi
 
 HOST=$1
@@ -33,7 +49,15 @@ TMPFILE=$TMPDIR/get-cert
 ERRFILE=$TMPDIR/get-cert-err
 CERTFILE=$TMPDIR/cert
 
-echo QUIT | openssl s_client -connect $HOST:993 -showcerts \
+if $STARTTLS; then
+	FLAGS="-starttls imap"
+	PORT=143
+else
+	FLAGS=
+	PORT=993
+fi
+
+echo QUIT | openssl s_client $FLAGS -connect $HOST:$PORT -showcerts \
 	> $TMPFILE 2> $ERRFILE
 sed -e '1,/^-----BEGIN CERTIFICATE-----/d' \
 	-e '/^-----END CERTIFICATE-----/,$d' < $TMPFILE > $CERTFILE

[commit] master: make summary more concise

[ossi <os...@us...>]

commit a1be7e9a369c1d038428a9d0b56ad0eb5d407441
Author: Oswald Buddenhagen <os...@us...>
Date:   Tue Nov 19 13:05:00 2024 +0100

    make summary more concise
    
    the verbose summary was actually hard to read due to the numbers getting
    lost between the words.
    
    i considered highlighting the numbers using ansi escapes, but the
    irregular structure would be still hard to parse, and the escapes would
    be unsuitable for log files.
    
    also considered was clustering the numbers at the beginnings of the
    lines, but that would result in a messy sentence structure.
    
    a proper tabular format would introduce a lot more spacing, and would be
    a lot harder to implement for little tangible benefit.
    
    i tried just using the progress counter format, but with plain numbers
    instead of the "x/y", but it looked kinda stupid.
    
    so instead use a slightly expanded, semi-tabular version of that, as
    suggested by Akshay Hegde on the list. this format bears a risk of
    exceeding 80 columns, and in log files the internal spacing looks kinda
    out of place, but these should be minor issues in practice.
    
    amends a1a3313e.
    
    REF: <Zzw...@ak...>

 src/main_sync.c | 32 ++++----------------------------
 src/mbsync.1.in |  2 +-
 2 files changed, 5 insertions(+), 29 deletions(-)

diff --git a/src/main_sync.c b/src/main_sync.c
index afd136e..2ea4a92 100644
--- a/src/main_sync.c
+++ b/src/main_sync.c
@@ -9,7 +9,6 @@
 
 #define nz(a, b) ((a) ? (a) : (b))
 
-static int ops_any[2], trash_any[2], expunge_any[2];
 static int chans_total, chans_done;
 static int boxes_total, boxes_done;
 
@@ -84,25 +83,10 @@ summary( void )
 	if (!boxes_done)
 		return;  // Shut up if we errored out early.
 
-	printf( "Processed %d box(es) in %d channel(s)", boxes_done, chans_done );
-	for (int t = 2; --t >= 0; ) {
-		if (ops_any[t])
-			printf( (DFlags & DRYRUN) ?
-			            ",\nwould %s %d new message(s) and %d flag update(s)" :
-			            ",\n%sed %d new message(s) and %d flag update(s)",
-			        str_hl[t], new_done[t], flags_done[t] );
-		if (trash_any[t])
-			printf( (DFlags & DRYRUN) ?
-			            ",\nwould move %d %s message(s) to trash" :
-			            ",\nmoved %d %s message(s) to trash",
-			        trash_done[t], str_fn[t] );
-		if (expunge_any[t])
-			printf( (DFlags & DRYRUN) ?
-			            ",\nwould expunge %d message(s) from %s" :
-			            ",\nexpunged %d message(s) from %s",
-			        expunge_done[t], str_fn[t] );
-	}
-	puts( "." );
+	printf( "Channels: %d    Boxes: %d    Far: +%d *%d #%d -%d    Near: +%d *%d #%d -%d\n",
+	        chans_done, boxes_done,
+	        new_done[F], flags_done[F], trash_done[F], expunge_done[F],
+	        new_done[N], flags_done[N], trash_done[N], expunge_done[N] );
 }
 
 static int
@@ -244,14 +228,6 @@ add_channel( chan_ent_t ***chanapp, channel_conf_t *chan, int ops[] )
 			free( ce );
 			return NULL;
 		}
-		if (chan->ops[t] & OP_MASK_TYPE)
-			ops_any[t] = 1;
-		if (chan->ops[t] & (OP_EXPUNGE | OP_EXPUNGE_SOLO)) {
-			expunge_any[t] = 1;
-			if (chan->stores[t]->trash ||
-			    (chan->stores[t^1]->trash && chan->stores[t^1]->trash_remote_new))
-				trash_any[t] = 1;
-		}
 	}
 
 	**chanapp = ce;
diff --git a/src/mbsync.1.in b/src/mbsync.1.in
index fba3669..032918b 100644
--- a/src/mbsync.1.in
+++ b/src/mbsync.1.in
@@ -802,7 +802,7 @@ No attempt is made to calculate the totals in advance, so they grow over
 time as more information is gathered.
 .P
 Irrespective of output redirection, \fBmbsync\fR will print a summary
-of the above in plain language upon completion, except in quiet mode.
+of the above upon completion, except in quiet mode.
 .
 .SH RECOMMENDATIONS
 Make sure your IMAP server does not auto-expunge deleted messages - it is

[commit] master: fix crash when resuming message propagation with MaxMessages

[ossi <os...@us...>]

commit 1e7a75095b82b43c60de55890d6728f68e3e9e0e
Author: Oswald Buddenhagen <os...@us...>
Date:   Wed Nov 20 09:08:26 2024 +0100

    fix crash when resuming message propagation with MaxMessages
    
    the problem is triggered by the source-side message disappearing
    after a transaction to propagate it was started and then interrupted.
    
    it seems tempting to centralize the null-check, but some of the other
    branches are taken in situations where the relevant messages from the
    source store have not been requested.
    
    no autotest, as our test suite does not support injecting changes before
    resuming.
    
    amends 0089f49.

 src/sync.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/src/sync.c b/src/sync.c
index 1937d65..8cf3c65 100644
--- a/src/sync.c
+++ b/src/sync.c
@@ -1229,6 +1229,8 @@ box_loaded( int sts, message_t *msgs, int total_msgs, int recent_msgs, void *aux
 					// but we may be pulling in the real ones.
 					nflags = (srec->pflags | srec->aflags[xt]) & ~srec->dflags[xt];
 				} else {
+					if (!srec->msg[xt^1])
+						continue;
 					nflags = srec->msg[xt^1]->flags;
 				}
 			}

[commit] master: fix omissions in making expiration target side configurable

[ossi <os...@us...>]

commit 5f953c5162cef64c26b78edf7feb37499367c371
Author: Oswald Buddenhagen <os...@us...>
Date:   Mon Nov 18 15:13:14 2024 +0100

    fix omissions in making expiration target side configurable
    
    amends 8566283c.

 src/sync.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/src/sync.c b/src/sync.c
index 38dbf06..1937d65 100644
--- a/src/sync.c
+++ b/src/sync.c
@@ -1013,7 +1013,7 @@ box_loaded( int sts, message_t *msgs, int total_msgs, int recent_msgs, void *aux
 						sflags = sanitize_flags( sflags, svars, t );
 						if ((t != xt) && (srec->status & (S_EXPIRE | S_EXPIRED))) {
 							/* Don't propagate deletion resulting from expiration. */
-							debug( "  near side expiring\n" );
+							debug( "  %s side expiring\n", str_fn[xt] );
 							sflags &= ~F_DELETED;
 						}
 						if (srec->status & S_DUMMY(t^1)) {
@@ -1204,7 +1204,7 @@ box_loaded( int sts, message_t *msgs, int total_msgs, int recent_msgs, void *aux
 			if (!srec->uid[xt^1])
 				continue;
 			if (!(srec->status & S_PENDING)) {
-				// We ignore unpaired far-side messages, as there is obviously nothing
+				// We ignore unpaired keep-side messages, as there is obviously nothing
 				// to expire in the first place.
 				if (!srec->msg[xt])
 					continue;

[commit] master: do not let both-sided uidvalidity change deter us

[ossi <os...@us...>]

commit bf34d9fd29b7e2bd82ddb6b892ea0e671fc19926
Author: Oswald Buddenhagen <os...@us...>
Date:   Sat Nov 23 12:12:14 2024 +0100

    do not let both-sided uidvalidity change deter us
    
    the algorithm is symmetrical, comparing the msgids that belong to the
    paired uids. so it doesn't matter for recovering one side if the other
    side's uidvalidity also changed. it does however impact our ability to
    say on which side the change was genuine.
    
    the pointless limitation was presumably a vestige from an earlier
    iteration.
    
    amends 77acc26 and 594e60b.

 src/sync.c | 31 +++++++++++++++----------------
 1 file changed, 15 insertions(+), 16 deletions(-)

diff --git a/src/sync.c b/src/sync.c
index 405d98f..38dbf06 100644
--- a/src/sync.c
+++ b/src/sync.c
@@ -511,7 +511,7 @@ box_opened2( sync_vars_t *svars, int t )
 	channel_conf_t *chan;
 	sync_rec_t *srec;
 	uint_array_alloc_t mexcs;
-	uint opts[2], fails, minwuid;
+	uint opts[2], minwuid;
 
 	svars->state[t] |= ST_SELECTED;
 	if (!(svars->state[t^1] & ST_SELECTED))
@@ -520,23 +520,13 @@ box_opened2( sync_vars_t *svars, int t )
 	ctx[1] = svars->ctx[1];
 	chan = svars->chan;
 
-	fails = 0;
-	for (t = 0; t < 2; t++)
-		if (svars->uidval[t] != UIDVAL_BAD && svars->uidval[t] != svars->newuidval[t])
-			fails++;
-	// If only one side changed UIDVALIDITY, we will try to re-approve it further down.
-	if (fails == 2) {
-		error( "Error: channel %s: UIDVALIDITY of both far side %s and near side %s changed.\n",
-		       svars->chan->name, svars->orig_name[F], svars->orig_name[N]);
+	if (!lock_state( svars )) {
 	  bail:
 		svars->ret = SYNC_FAIL;
 		sync_bail( svars );
 		return;
 	}
 
-	if (!lock_state( svars ))
-		goto bail;
-
 	int any_dummies[2] = { 0, 0 };
 	int any_purges[2] = { 0, 0 };
 	int any_upgrades[2] = { 0, 0 };
@@ -574,9 +564,11 @@ box_opened2( sync_vars_t *svars, int t )
 	}
 
 	opts[F] = opts[N] = 0;
-	if (fails)
-		opts[F] = opts[N] = OPEN_PAIRED | OPEN_PAIRED_IDS;
 	for (t = 0; t < 2; t++) {
+		if (svars->uidval[t] != UIDVAL_BAD && svars->uidval[t] != svars->newuidval[t]) {
+			opts[F] |= OPEN_PAIRED | OPEN_PAIRED_IDS;
+			opts[N] |= OPEN_PAIRED | OPEN_PAIRED_IDS;
+		}
 		if (any_purges[t]) {
 			debug( "resuming %d %s purge(s)\n", any_purges[t], str_fn[t] );
 			opts[t] |= OPEN_SETFLAGS;
@@ -859,8 +851,15 @@ box_loaded( int sts, message_t *msgs, int total_msgs, int recent_msgs, void *aux
 				if (!srec->msg[t^1])
 					continue;  // Partner disappeared.
 				if (!srec->msg[t^1]->msgid || strcmp( srec->msg[F]->msgid, srec->msg[N]->msgid )) {
-					error( "Error: channel %s, %s box %s: UIDVALIDITY genuinely changed (at UID %u).\n",
-					       svars->chan->name, str_fn[t], svars->orig_name[t], srec->uid[t] );
+					if (svars->uidval[t^1] != svars->newuidval[t^1]) {
+						error( "Error: channel %s, %s box %s (at UID %u):"
+						       " Unable to recover from both-sided UIDVALIDITY change,"
+						       " as it is genuine on at least one side.\n",
+						       svars->chan->name, str_fn[t], svars->orig_name[t], srec->uid[t] );
+					} else {
+						error( "Error: channel %s, %s box %s (at UID %u): UIDVALIDITY genuinely changed.\n",
+						       svars->chan->name, str_fn[t], svars->orig_name[t], srec->uid[t] );
+					}
 				  uvchg:
 					svars->ret |= SYNC_FAIL;
 					cancel_sync( svars );

Re: Option for Concise Output

[Akshay H. <lis...@ak...>]

On 2024-11-23 15:09 +0100, Oswald Buddenhagen via isync-devel wrote:
> On Tue, Nov 19, 2024 at 06:26:06PM -0800, Akshay Hegde wrote:
> > Like most new UIs, my initial reaction was 'oh I don't like that', but
> > perhaps I just need to get used to it.
> > 
> i thought that myself, but i'm at the point where i've concluded that it
> was simply a stupid idea.
> 
> On Wed, Nov 20, 2024 at 04:05:26PM +0100, Daniel Tameling wrote:
> > I think it would help to put all numbers at the start of the line.
> > [...]
> > It's not very elegant, but I find it more readable.
> > 
> yes, exactly. better than the verbose version, but not really better
> than the short version, imo.
> 
> On Wed, Nov 20, 2024 at 06:56:59PM -0800, Akshay Hegde wrote:
> > My preference for a concise output would be something like this:
> > 
> >     Channels: 2    Boxes: 4    Far: +13 *42 #0 -0    Near: +7 *0 #0 -0
> > 
> so the short version, but with words instead of letters. that makes it
> half-self-explanatory ...
> 
> i thought i'd just go with the short version, but it looks kinda stupid.
> 
> so i'll go with your idea.
> i tried less spacing, but that was significantly harder to read, which
> you probably noticed yourself. on the downside, now there is a somewhat
> higher risk of exceeding 80 columns, and in a log file internal spacing
> looks kinda out of place (and i don't want to use a different format
> there). i also tried comma-space, but that looks too noisy again.
> 
> > > you can paypal me on this address.
> > 
> > Sent, thank you ;)
> > 
> thanks!
> 
> 
> _______________________________________________
> isync-devel mailing list
> isy...@li...
> https://lists.sourceforge.net/lists/listinfo/isync-devel

See, I didn't even realize there was a risk of exceeding 80 chars. :)
I was just trying to be as clear as possible. You are of course, right.

The entire reason why I even thought to post is because I like the
progress output so much; it's already so clear. So for me, there's
really nothing much to improve on, except to print out the final
statistics without the dup'd numbers like you said.

After using the verbose output for a week or so, I don't mind it _that_
much now, which just goes to show how quickly I can change my mind-and
to not put too much weight into what I may say I prefer... :)

I know you will come up with something that has a nice balance between
readability and conciseness and I'll look forward to trying that out!

Cheers,
-- 
	Akshay