#144 conf setting "--exclude-other-filesystems" corrupts generated cmd line

[David Epping]

Hi duply team,

after upgrading from Debian 12 to Debian 13 duply fails to correctly use duplicity for some commands with my config (or duplicity handles them differently than before).
Duply version is 2.5.5 and duplicity version is 3.0.4.
The problem combination is the MAX_FULLBKP_AGE or VOLSIZE feature of duply with for example the status command of duplicity. It seems the parameter, which has no meaning for those commands, is no longer ignored but treated as additional and disallowed positional argument:

$ duply rootbkp status
Start duply v2.5.5, time is 2025-08-21 12:05:48.
Using profile '/etc/duply/rootbkp'.
Using installed duplicity version 3.0.4, python 3.13.5 (/usr/bin/python3) 'PYTHONPATH=:/usr/lib/python313.zip:/usr/lib/python3.13:/usr/lib/python3.13/lib-dynload:/usr/local/lib/python3.13/dist-packages:/usr/lib/python3/dist-packages', gpg 2.4.7 (Home: /root/.gnupg), awk 'mawk 1.3.4 20250131', grep 'grep (GNU grep) 3.11', bash '5.2.37(1)-release (x86_64-pc-linux-gnu)'.
Signing disabled. No GPG_KEY entries in config.
Checking TEMP_DIR '/tmp' is a folder and writable (OK)
Test - Encryption with passphrase (OK)
Test - Decryption with passphrase (OK)
Test - Compare (OK)
Cleanup - Delete '/tmp/duply.2137.1755770748_*'(OK)

--- Start running command STATUS at 2025-08-21 12:05:49.927 ---
CommandLineError: Wrong number of positional args for 'collection-status', got 2
Expected 1 positionals from ['1W', 'ftp://user@backup.server/template-debian13-root'].
Enter 'duplicity --help' for help screen.
2025-08-21 12:05:50.064 Task 'STATUS' failed with exit code '23'.
--- Finished state FAILED 'code 23' at 2025-08-21 12:05:50.064 - Runtime 00:00:00.136 ---

Commenting the two maximum full backup age lines makes the command succeed.
I'll let cron decide when to perform full backups as a workaroung, but this used to be a nice feature.
VOLSIZE has a similar parameter scheme and fails the same way.

Thanks for the good work on duply,
David

[ede]

hey David,

will try to reproduce and fix. just wanted to let you know that i've seen your report and appreciate it.

THANKS! ..ede

[David Epping]

Hi ede,
thanks a lot. Just let me know if you need additional information from my side.
Best regards, David

[ede]

The problem combination is the MAX_FULLBKP_AGE or VOLSIZE feature of duply with for example the status command of duplicity.

are you sure VOLSIZE raises an issue? afaics it is stripped for all non-backup commands.

[David Epping]

My initial config had only MAX_FULLBKP_AGE uncommented and this was what triggered the problem for me. When creating the bug report, because it looked similar, I also tried with VOLSIZE briefly and saw it fail.
I just re-tested and the error message is as follows, with default VOLSIZE of 50:

2025-08-26T22:05:50.147354+02:00 share duply-backup: --- Start running command PURGE at 2025-08-26 22:05:50.140 ---
2025-08-26T22:05:50.246601+02:00 share duply-backup: CommandLineError: Wrong number of positional args for 'remove-older-than', got 3
2025-08-26T22:05:50.246697+02:00 share duply-backup: Expected 2 positionals from ['2M', '50', 'ftp://user@ubackup.com/share-srv_slow'].
2025-08-26T22:05:50.246761+02:00 share duply-backup: Enter 'duplicity --help' for help screen.
2025-08-26T22:05:50.273214+02:00 share duply-backup: 2025-08-26 22:05:50.267 Task 'PURGE' failed with exit code '23'.
2025-08-26T22:05:50.277927+02:00 share duply-backup: --- Finished state FAILED 'code 23' at 2025-08-26 22:05:50.267 - Runtime 00:00:00.126 ---

The 2M is from my MAX_AGE=2M parameter.

So yes, it seems to fail for me. I upgraded from Debian 12 to 13 and the config in etc was unchanged. Maybe the VOLSIZE handling changed and my config should have changed as well. I'll compare to current upstream tomorrow.

[ede]

i see. thanks! will have a deeper look.

[ede]

just tested the VOLSIZE issue

VOLSIZE=50
DUPL_PARAMS="$DUPL_PARAMS --volsize $VOLSIZE "

with duply test status (test is a profile) and it works fine. verified it with the duply --preview parameter which does not execute but merely prints the generated cmd lines.
NOTE the missing --volsize 50 from the status cmd line.

:~$ ./duply_dev/duply_wrap test status --preview
Start duply.sh v2.5.6dev, time is 2025-08-28 10:54:23.
Using profile '/home/dup/.duply/test'.
SNIP
--- Start running command STATUS at 2025-08-28 10:54:24.791 ---
TMPDIR='/tmp' PASSPHRASE='duplicity&'   duplicity  collection-status --archive-dir /tmp/foo --name test --encrypt-key 87ABE172E0AAD23BA4BC91EE671B39931C1B14DE --sign-key 87ABE172E0AAD23BA4BC91EE671B39931C1B14DE  --gpg-options='--pinentry-mode=loopback'     'file:///tmp/t e s t 12 noenc'
--- Finished state OK at 2025-08-28 10:54:24.946 - Runtime 00:00:00.155 ---

:~$ ./duply_dev/duply_wrap test bkp --preview
Start duply.sh v2.5.6dev, time is 2025-08-28 10:54:42.
Using profile '/home/dup/.duply/test'.
SNIP
--- Start running command BKP at 2025-08-28 10:54:44.298 ---
TMPDIR='/tmp' PASSPHRASE='duplicity&'   duplicity  backup --archive-dir /tmp/foo --name test --encrypt-key 87ABE172E0AAD23BA4BC91EE671B39931C1B14DE --sign-key 87ABE172E0AAD23BA4BC91EE671B39931C1B14DE  --gpg-options='--pinentry-mode=loopback'  --volsize 50     --exclude-filelist /home/dup/.duply/test/exclude duply_dev/ 'file:///tmp/t e s t 12 noenc'
--- Finished state OK at 2025-08-28 10:54:44.435 - Runtime 00:00:00.136 ---

1.
can you please provide me with a copy of your profile/conf file? obfuscate data you deem private in there (e.g. passphrases).

2.
please run the preview commands for me to see what is generated and post the full shell output. again obfuscate if needed.

duply <profile> status --preview
duply <profile> bkp --preview

[ede]

also can't replicate the issue with

MAX_FULLBKP_AGE=1M
DUPL_PARAMS="$DUPL_PARAMS --full-if-older-than $MAX_FULLBKP_AGE "

with duplicity 3.0.4 . maybe a python 3.13 issue or something else on Debian 13 . will setup a vm and check.

[ede]

• created a fresh Debian 13 container
• installed duplicity/duply
• created test profile, enabled VOLSIZE and MAX_FULLBKP_AGE

works for me. please send me the outputs requested above https://sourceforge.net/p/ftplicity/bugs/144/#f9d6

user@debian13-lxc:~$ duply test status
Start duply v2.5.5, time is 2025-08-28 12:51:28.
Using profile '/home/user/.duply/test'.
Using installed duplicity version 3.0.4, python 3.13.5 (/usr/bin/python3) 'PYTHONPATH=:/usr/lib/python313.zip:/usr/lib/python3.13:/usr/lib/python3.13/lib-dynload:/usr/local/lib/python3.13/dist-packages:/usr/lib/python3/dist-packages', gpg 2.4.7 (Home: /home/user/.gnupg), awk 'mawk 1.3.4 20250131', grep 'grep (GNU grep) 3.11', bash '5.2.37(1)-release (x86_64-pc-linux-gnu)'.
Checking TEMP_DIR '/tmp' is a folder and writable (OK)
Test - En/Decryption skipped. (GPG='disabled')

--- Start running command STATUS at 2025-08-28 12:51:28.511 ---
Last full backup date: none
Collection Status
-----------------
Connecting with backend: BackendWrapper
Archive dir: /home/user/.cache/duplicity/duply_test

Found 0 secondary backup chain(s).
No backup chains with active signatures found
No orphaned or incomplete backup sets found.
--- Finished state OK at 2025-08-28 12:51:28.847 - Runtime 00:00:00.336 ---


user@debian13-lxc:~$ duply test status --preview
Start duply v2.5.5, time is 2025-08-28 12:52:20.
Using profile '/home/user/.duply/test'.
Using installed duplicity version 3.0.4, python 3.13.5 (/usr/bin/python3) 'PYTHONPATH=:/usr/lib/python313.zip:/usr/lib/python3.13:/usr/lib/python3.13/lib-dynload:/usr/local/lib/python3.13/dist-packages:/usr/lib/python3/dist-packages', gpg 2.4.7 (Home: /home/user/.gnupg), awk 'mawk 1.3.4 20250131', grep 'grep (GNU grep) 3.11', bash '5.2.37(1)-release (x86_64-pc-linux-gnu)'.
-- Run cmd -- Checking TEMP_DIR '/tmp' is a folder and writable --
test -d /tmp && test -w /tmp 2>&1
Test - En/Decryption skipped. (GPG='disabled')

--- Start running command STATUS at 2025-08-28 12:52:20.925 ---
TMPDIR='/tmp'    duplicity  collection-status --name duply_test --no-encryption     --full-if-older-than 1M    'file:///tmp/testbkp'
--- Finished state OK at 2025-08-28 12:52:20.988 - Runtime 00:00:00.062 ---

[David Epping]

Hi ede, thanks for putting in all that effort. While stripping down the config to something minimal that still shows the issue, I found that I can toggle between working and not working by commenting or having active my exclude-other-filesystems line. I also found that only the MAX_FULLBKP_AGE content xor the VOLSIZE content shows up in the commandline. If both are active VOLSIZE does not show up.

1. This is the config showing the error and below outputs:

BACKUP_BASE_DIR='/srv/duply'

DUPL_PARAMS="$DUPL_PARAMS --exclude-other-filesystems "

GPG_PW='sourceforge.net public password'

TARGET="file:///absolute_path"

MAX_AGE=2M

MAX_FULLBKP_AGE=1W
DUPL_PARAMS="$DUPL_PARAMS --full-if-older-than $MAX_FULLBKP_AGE "

VOLSIZE=50
DUPL_PARAMS="$DUPL_PARAMS --volsize $VOLSIZE "

SOURCE='/'

2.

$ duply profile status --preview
Start duply v2.5.5, time is 2025-08-28 15:08:01.
Using profile '/etc/duply/profile'.
Using installed duplicity version 3.0.4, python 3.13.5 (/usr/bin/python3) 'PYTHONPATH=:/usr/lib/python313.zip:/usr/lib/python3.13:/usr/lib/python3.13/lib-dynload:/usr/local/lib/python3.13/dist-packages:/usr/lib/python3/dist-packages', gpg 2.4.7 (Home: /root/.gnupg), awk 'mawk 1.3.4 20250131', grep 'grep (GNU grep) 3.11', bash '5.2.37(1)-release (x86_64-pc-linux-gnu)'.
Signing disabled. No GPG_KEY entries in config.
-- Run cmd -- Checking TEMP_DIR '/tmp' is a folder and writable --
test -d /tmp && test -w /tmp 2>&1
-- Run cmd -- Test - Encryption with passphrase --
echo 'sourceforge.net public password' | gpg --pinentry-mode=loopback --passphrase-fd 0 -o /tmp/duply.10365.1756386481_ENC --batch -c /usr/bin/duply 2>&1
-- Run cmd -- Test - Decryption with passphrase --
echo 'sourceforge.net public password' | gpg --pinentry-mode=loopback --passphrase-fd 0 -o /tmp/duply.10365.1756386481_DEC --batch -d /tmp/duply.10365.1756386481_ENC 2>&1
-- Run cmd -- Test - Compare --
test "$(cat '/usr/bin/duply')" = "$(cat '/tmp/duply.10365.1756386481_DEC')" 2>&1
Cleanup - Delete '/tmp/duply.10365.1756386481_*'(FAILED)

--- Start running command STATUS at 2025-08-28 15:08:01.983 --- 
TMPDIR='/tmp' PASSPHRASE='sourceforge.net public password'   duplicity  collection-status --name duply_profile    --gpg-options='--pinentry-mode=loopback'  1W    'file:///absolute_path'
--- Finished state OK at 2025-08-28 15:08:02.042 - Runtime 00:00:00.058 --- 


$duply profile bkp --preview
Start duply v2.5.5, time is 2025-08-28 15:08:15.
Using profile '/etc/duply/profile'.
Using installed duplicity version 3.0.4, python 3.13.5 (/usr/bin/python3) 'PYTHONPATH=:/usr/lib/python313.zip:/usr/lib/python3.13:/usr/lib/python3.13/lib-dynload:/usr/local/lib/python3.13/dist-packages:/usr/lib/python3/dist-packages', gpg 2.4.7 (Home: /root/.gnupg), awk 'mawk 1.3.4 20250131', grep 'grep (GNU grep) 3.11', bash '5.2.37(1)-release (x86_64-pc-linux-gnu)'.
Signing disabled. No GPG_KEY entries in config.
-- Run cmd -- Checking TEMP_DIR '/tmp' is a folder and writable --
test -d /tmp && test -w /tmp 2>&1
-- Run cmd -- Test - Encryption with passphrase --
echo 'sourceforge.net public password' | gpg --pinentry-mode=loopback --passphrase-fd 0 -o /tmp/duply.10675.1756386496_ENC --batch -c /usr/bin/duply 2>&1
-- Run cmd -- Test - Decryption with passphrase --
echo 'sourceforge.net public password' | gpg --pinentry-mode=loopback --passphrase-fd 0 -o /tmp/duply.10675.1756386496_DEC --batch -d /tmp/duply.10675.1756386496_ENC 2>&1
-- Run cmd -- Test - Compare --
test "$(cat '/usr/bin/duply')" = "$(cat '/tmp/duply.10675.1756386496_DEC')" 2>&1
Cleanup - Delete '/tmp/duply.10675.1756386496_*'(FAILED)

--- Start running command BKP at 2025-08-28 15:08:16.179 --- 
TMPDIR='/tmp' PASSPHRASE='sourceforge.net public password'   duplicity  backup --name duply_profile    --gpg-options='--pinentry-mode=loopback'  --exclude-other-filesystems  --full-if-older-than 1W  --volsize 50     --exclude-filelist /etc/duply/profile/exclude / 'file:///absolute_path'
--- Finished state OK at 2025-08-28 15:08:16.215 - Runtime 00:00:00.036 --- 

[ede]

ok, that helps. at least i got a hickup now to debug :)

user@debian13-lxc:~$ duply david status
Start duply v2.5.5, time is 2025-08-28 13:25:50.
Using profile '/home/user/.duply/david'.
Using installed duplicity version 3.0.4, python 3.13.5 (/usr/bin/python3) 'PYTHONPATH=:/usr/lib/python313.zip:/usr/lib/python3.13:/usr/lib/python3.13/lib-dynload:/usr/local/lib/python3.13/dist-packages:/usr/lib/python3/dist-packages', gpg 2.4.7 (Home: /home/user/.gnupg), awk 'mawk 1.3.4 20250131', grep 'grep (GNU grep) 3.11', bash '5.2.37(1)-release (x86_64-pc-linux-gnu)'.
Signing disabled. No GPG_KEY entries in config.
Checking TEMP_DIR '/tmp' is a folder and writable (OK)
Test - Encryption with passphrase (OK)
Test - Decryption with passphrase (OK)
Test - Compare (OK)
Cleanup - Delete '/tmp/duply.5754.1756387551_*'(OK)

--- Start running command STATUS at 2025-08-28 13:25:51.720 ---
CommandLineError: Wrong number of positional args for 'collection-status', got 2
Expected 1 positionals from ['1W', 'file:///tmp/absolute_path'].
Enter 'duplicity --help' for help screen.
2025-08-28 13:25:51.938 Task 'STATUS' failed with exit code '23'.
--- Finished state FAILED 'code 23' at 2025-08-28 13:25:51.938 - Runtime 00:00:00.217 ---

[David Epping]

I hope this is a supported use case after all. It seems like the exclude-other-filesytems parameter is not in duply.sh and it must have been added to the config many years ago ...

[ede]

all good. i think i found it. please try this snapshot

https://duply.net/tmp/duply.sh

[David Epping]

Thanks so much! Yes, it works correctly in my setup as well.

[ede]

released as v2.5.6 . THANKS again David!