#138 over-quoting duplicity_params_conf causes commands to fail

[Eric]

My duply config file has the following additional param setup at the end:

DUPL_PARAMS="$DUPL_PARAMS --ssh-options=-oIdentityFile=/root/.ssh/ed25519_duplicity"

The fully $DUPLY_PARAMS after sourcing the config looks like:

--full-if-older-than 2W --volsize 1024 --ssh-options=-oIdentityFile=/root/.ssh/ed25519_duplicity

When running the PURGE command duply fails with the following error:

--- Start running command PURGE at 2023-09-30 09:36:15.749 ---
CommandLineError: Target should be url, not directory.  Got ''--ssh-options=-oIdentityFile=/root/.ssh/ed25519_duplicity'' instead.
Enter 'duplicity --help' for help screen.
Enter 'duplicity <action> --help' for help specific to the given action.
2023-09-30 09:36:16.193 Task 'PURGE' failed with exit code '23'.
--- Finished state FAILED 'code 23' at 2023-09-30 09:36:16.193 - Runtime 00:00:00.443 ---

I've added an echo to dump the output of duplicity_params_conf and I see it doing additional quoting for each subsequent command being run:

media:/etc/duply/default_conf# /usr/local/bin/duply default_conf backup_verify_purge --force
Start duply v2.5.0, time is 2023-09-30 09:34:18.
Using profile '/etc/duply/default_conf'.
Using installed duplicity version 2.1.2, python 3.8.10 (/usr/bin/python3) 'PYTHONPATH=:/usr/lib/python38.zip:/usr/lib/python3.8:/usr/lib/python3.8/lib-dynload:/usr/local/lib/python3.8/dist-packages:/usr/lib/python3/dist-packages', gpg 2.2.19 (Home: /root/.gnupg), awk 'mawk 1.3.4 20200120', grep 'grep (GNU grep) 3.4', bash '5.0.17(1)-release (x86_64-pc-linux-gnu)'.
Checking TEMP_DIR '/tmp' is a folder and writable (OK)
Test - En/Decryption skipped. (GPG='disabled')

--- Start running command PRE at 2023-09-30 09:34:19.535 ---
Skipping n/a script '/etc/duply/default_conf/pre'.
--- Finished state OK at 2023-09-30 09:34:19.555 - Runtime 00:00:00.019 ---

--- Start running command BKP at 2023-09-30 09:34:19.589 ---
DUPL_VARS_GLOBAL: TMPDIR='/tmp'
BACKEND_PARAMS:
DUPL_PRECMD:
CMD: python3 /usr/bin/duplicity
DUPL_CMD:  backup
DUPL_PARAMS_GLOBAL: --name duply_default_conf --no-encryption  --verbosity '4'
duplicity_params_conf:  --full-if-older-than 2W  --volsize 1024  --ssh-options=-oIdentityFile=/root/.ssh/ed25519_duplicity
GPG_USEAGENT:
gpg_custom_binary:
DUPL_CMD_PARAMS:  --force --exclude-filelist /etc/duply/default_conf/exclude / 'sftp://duplicity_media@backups'
Reading globbing filelist /etc/duply/default_conf/exclude
Local and Remote metadata are synchronized, no sync needed.
Last full backup date: Sat Sep 30 08:18:36 2023
--------------[ Backup Statistics ]--------------
StartTime 1696091661.11 (Sat Sep 30 09:34:21 2023)
EndTime 1696091665.13 (Sat Sep 30 09:34:25 2023)
ElapsedTime 4.03 (4.03 seconds)
SourceFiles 15980
SourceFileSize 3830701369 (3.57 GB)
NewFiles 3
NewFileSize 20 (20 bytes)
DeletedFiles 0
ChangedFiles 2
ChangedFileSize 112369 (110 KB)
ChangedDeltaSize 0 (0 bytes)
DeltaEntries 5
RawDeltaSize 6698 (6.54 KB)
TotalDestinationSizeChange 1759 (1.72 KB)
Errors 0
-------------------------------------------------

--- Finished state OK at 2023-09-30 09:34:25.195 - Runtime 00:00:05.606 ---

--- Start running command POST at 2023-09-30 09:34:25.255 ---
Skipping n/a script '/etc/duply/default_conf/post'.
--- Finished state OK at 2023-09-30 09:34:25.285 - Runtime 00:00:00.030 ---

--- Start running command VERIFY at 2023-09-30 09:34:25.402 ---
DUPL_VARS_GLOBAL: TMPDIR='/tmp'
BACKEND_PARAMS:
DUPL_PRECMD:
CMD: python3 /usr/bin/duplicity
DUPL_CMD:  verify
DUPL_PARAMS_GLOBAL: --name duply_default_conf --no-encryption  --verbosity '4'
duplicity_params_conf:  --full-if-older-than 2W '--ssh-options=-oIdentityFile=/root/.ssh/ed25519_duplicity'
GPG_USEAGENT:
gpg_custom_binary:
DUPL_CMD_PARAMS:  --force --exclude-filelist /etc/duply/default_conf/exclude 'sftp://duplicity_media@backups' /
Reading globbing filelist /etc/duply/default_conf/exclude
Local and Remote metadata are synchronized, no sync needed.
Last full backup date: Sat Sep 30 08:18:36 2023
Verify complete: 15980 file(s) compared, 0 difference(s) found.
--- Finished state OK at 2023-09-30 09:36:15.693 - Runtime 00:01:50.291 ---

--- Start running command PURGE at 2023-09-30 09:36:15.749 ---
DUPL_VARS_GLOBAL: TMPDIR='/tmp'
BACKEND_PARAMS:
DUPL_PRECMD:
CMD: python3 /usr/bin/duplicity
DUPL_CMD:  remove-older-than 4W
DUPL_PARAMS_GLOBAL: --name duply_default_conf --no-encryption  --verbosity '4'
duplicity_params_conf:  --full-if-older-than 2W ''\''--ssh-options=-oIdentityFile=/root/.ssh/ed25519_duplicity'\'''
GPG_USEAGENT:
gpg_custom_binary:
DUPL_CMD_PARAMS:  --force 'sftp://duplicity_media@backups'
CommandLineError: Target should be url, not directory.  Got ''--ssh-options=-oIdentityFile=/root/.ssh/ed25519_duplicity'' instead.
Enter 'duplicity --help' for help screen.
Enter 'duplicity <action> --help' for help specific to the given action.
2023-09-30 09:36:16.193 Task 'PURGE' failed with exit code '23'.
--- Finished state FAILED 'code 23' at 2023-09-30 09:36:16.193 - Runtime 00:00:00.443 ---

I'm not quite sure why these accumulate over the life of the script run. A quick fix though is to change the quotewrap regex to ignore =

if echo "$param"  | awk '/[^A-Za-z0-9_\.\-\/=]/{exit 0}{exit 1}'; then

[ede]

hey Eric,

thanks for your report. please try the new devel version https://duply.net/tmp/duply.sh and check if your problem persists.

Thanks! ..ede

[Eric]

Verified 2.5.1 resolve the issue for me, thank you for the quick fix!