Menu ▾ ▴

#140 Broken GPG_OPTS in duply 2.5.2 under macOS Sonoma

duply
closed-fixed
nobody
macos (1)
6
2024-07-10
2024-01-21
Ralf Schröter
No

Hi! The macOS version 2.5.2 of duply installed via homebrew under macOS Sonoma 14.2.1 on a macBook Air (M1) IMHO does not pass the "GPG_OPTS" specified in the "conf" file correctly to duplicity.

My entry under GPG_OPTS in the conf-file "/.duply/my.repository/conf":

(...)
GPG_OPTS='--cipher-algo CAMELLIA256 --pinentry-mode loopback --compress-algo bzip2 --compress-level 9 --bzip2-compress-level 9'
(...)

results in the following error when running "duply my.repository backup" under the macOS terminal:

CommandLineError: Option '--cipher-algo CAMELLIA256 --pinentry-mode loopback --compress-algo bzip2 --compress-level 9 --bzip2-compress-level 9' is not a valid option for command 'backup'.
Options valid for command 'backup' are:
--allow-source-mismatch --archive-dir --asynchronous-upload
--azure-blob-tier --azure-max-block-size --azure-max-connections
--azure-max-single-put-size --b2-hide-files --backend-retry-delay
--cf-backend --compare-data --config-dir --copy-blocksize --copy-links
--current-time --dry-run --encrypt-key --encrypt-secret-keyring
--encrypt-sign-key --exclude --exclude-device-files --exclude-filelist
--exclude-if-present --exclude-older-than --exclude-other-filesystems
--exclude-regexp --fail-on-volume --file-changed --file-prefix
--file-prefix-archive --file-prefix-jsonstat --file-prefix-manifest
--file-prefix-signature --files-from --filter-globbing
--filter-ignorecase --filter-literal --filter-regexp
--filter-strictcase --force --ftp-passive --ftp-regular
--full-if-older-than --gpg-binary --gpg-options --hidden-encrypt-key
--idr-fakeroot --ignore-errors --imap-full-address --imap-mailbox
--include --include-filelist --include-regexp --jsonstat --log-fd
--log-file --log-timestamp --max-blocksize --metadata-sync-mode
--mf-purge --mp-segment-size --name --no-compression --no-encryption
--no-files-changed --no-print-statistics --no-restore-ownership
--null-separator --num-retries --numeric-owner --par2-options
--par2-redundancy --par2-volumes --path-to-restore --progress
--progress-rate --pydevd --rename --restore-time --rsync-options
--s3-endpoint-url --s3-kms-grant --s3-kms-key-id
--s3-multipart-chunk-size --s3-mul5tipart-max-procs --s3-region-name
--s3-unencrypted-connection --s3-use-deep-archive --s3-use-glacier
--s3-use-glacier-ir --s3-use-ia --s3-use-onezone-ia --s3-use-rrs
--s3-use-server-side-encryption --s3-use-server-side-kms-encryption
--scp-command --sftp-command --show-changes-in-set --sign-key
--skip-if-no-change --skip-volume --ssh-askpass --ssh-options
--ssl-cacert-file --ssl-cacert-path --ssl-no-check-certificate
--swift-storage-policy --tempdir --time-separator --timeout
--use-agent --verbosity --version --volsize --webdav-headers

See man page for more information.
Enter 'duplicity --help' for help screen.
2024-01-21 09:52:05.000 Task 'BKP' failed with exit code '23'.
--- Finished state FAILED 'code 23' at 2024-01-21 09:52:05.000 - Runtime 00:00:00.000 ---

Without "GPG_OPTS"-entry everything runs smooth. Therefore I think that the parser does not process the "GPG_OPTS"-line correctly and does not add the prefix "--gpg-options" when running duplicity. Can you please check/investigate or give a hint how to fix GPG_OPTS? Thank you!

Best regards!
Ralf

Discussion

  • ede

    ede - 2024-01-21

    moin Ralf,

    please run duply backup command with parameter --preview and the offending GPG_OPTS enabled. post the complete output here after. i need versions outputs as well.

    thanks.. ede

     

    Last edit: ede 2024-01-21

  • Ralf Schröter

    Ralf Schröter - 2024-01-22

    moin ede

    wow - thank you for the fast reply! Here is the requested output (I have replaced only PGP-key-id and passphrase with generics):

    Start duply v2.5.2, time is 2024-01-22 08:15:15.
Using profile '/Users/ralf/.duply/devonthink.koofr.net'.
   Using installed duplicity version 2.1.5, python 3.11.7 (/opt/homebrew/Cellar/duplicity/2.1.5/libexec/bin/python) 'PYTHONPATH=:/opt/homebrew/Cellar/python@3.11/3.11.7_1/Frameworks/Python.framework/Versions/3.11/lib/python311.zip:/opt/homebrew/Cellar/python@3.11/3.11.7_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11:/opt/homebrew/Cellar/python@3.11/3.11.7_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/lib-dynload:/opt/homebrew/Cellar/duplicity/2.1.5/libexec/lib/python3.11/site-packages:/opt/homebrew/opt/llvm/lib/python3.11/site-packages:/opt/homebrew/opt/pycparser/lib/python3.11/site-packages:/opt/homebrew/opt/cffi/lib/python3.11/site-packages:/opt/homebrew/opt/protobuf/lib/python3.11/site-packages:/opt/homebrew/opt/python-certifi/lib/python3.11/site-packages:/opt/homebrew/opt/python-typing-extensions/lib/python3.11/site-packages:/opt/homebrew/opt/python-cryptography/lib/python3.11/site-packages:/opt/homebrew/opt/python-packaging/lib/python3.11/site-packages:/opt/homebrew/opt/six/lib/python3.11/site-packages:/opt/homebrew/opt/python-dateutil/lib/python3.11/site-packages:/opt/homebrew/opt/python-lxml/lib/python3.11/site-packages:/opt/homebrew/opt/python-ply/lib/python3.11/site-packages:/opt/homebrew/opt/python-psutil/lib/python3.11/site-packages:/opt/homebrew/opt/python-pyparsing/lib/python3.11/site-packages:/opt/homebrew/opt/python-pytz/lib/python3.11/site-packages:/opt/homebrew/opt/pyyaml/lib/python3.11/site-packages:/opt/homebrew/opt/llvm/lib/python3.11/site-packages:/opt/homebrew/lib/python3.11/site-packages', gpg 2.4.3 (Home: /Users/ralf/.gnupg), awk 'awk version 20200816', grep 'grep (BSD grep, GNU compatible) 2.6.0-FreeBSD', bash '3.2.57(1)-release (arm64-apple-darwin23)'.
Public key 'MYPGPKEYID' found in keychain.
Autoset found secret key of first GPG_KEY entry 'MYPGPKEYID' for signing.
-- Run cmd -- Checking TEMP_DIR '/tmp' is a folder and writable --
test -d /tmp && test -w /tmp 2>&1
-- Run cmd -- Test - Encrypt to 'MYPGPKEYID' & Sign with 'MYPGPKEYID' --
echo MYPGPPASSWORD | gpg --sign --default-key MYPGPKEYID --passphrase-fd 0 --batch -r MYPGPKEYID --status-fd 1 --cipher-algo CAMELLIA256 --pinentry-mode loopback --compress-algo bzip2 --compress-level 9 --bzip2-compress-level 9 -o /tmp/duply.67311.1705907716_ENC -e /opt/homebrew/bin/duply 2>&1
-- Run cmd -- Test - Decrypt --
echo MYPGPPASSWORD | gpg --passphrase-fd 0 --batch --cipher-algo CAMELLIA256 --pinentry-mode loopback --compress-algo bzip2 --compress-level 9 --bzip2-compress-level 9 -o /tmp/duply.67311.1705907716_DEC -d /tmp/duply.67311.1705907716_ENC 2>&1
-- Run cmd -- Test - Compare --
test "$(cat '/opt/homebrew/bin/duply')" = "$(cat '/tmp/duply.67311.1705907716_DEC')" 2>&1
Cleanup - Delete '/tmp/duply.67311.1705907716_*'(FAILED)

--- Start running command PRE at 2024-01-22 08:15:16.000 ---
. /Users/ralf/.duply/devonthink.koofr.net/pre
--- Finished state OK at 2024-01-22 08:15:16.000 - Runtime 00:00:00.000 ---

--- Start running command BKP at 2024-01-22 08:15:16.000 ---
TMPDIR='/tmp' PASSPHRASE=MYPGPPASSWORD   duplicity  backup --name duply_devonthink.koofr.net --encrypt-key MYPGPKEYID --sign-key MYPGPKEYID --verbosity 4 --gpg-options '--cipher-algo CAMELLIA256 --pinentry-mode loopback --compress-algo bzip2 --compress-level 9 --bzip2-compress-level 9'  --volsize 100  --log-file /Users/ralf/duply_backup_devonthink.log     --exclude-filelist /Users/ralf/.duply/devonthink.koofr.net/exclude /Users/ralf/Documents/devonthink/ 'rclone://ralf.koofr.net:/backup/duply-devonthink'
--- Finished state OK at 2024-01-22 08:15:16.000 - Runtime 00:00:00.000 ---

--- Start running command POST at 2024-01-22 08:15:16.000 ---
. /Users/ralf/.duply/devonthink.koofr.net/post
--- Finished state OK at 2024-01-22 08:15:16.000 - Runtime 00:00:00.000 ---

    Hopefully it will help in your investigation. Thank you!

    Best regards
    Ralf

     

  • ede

    ede - 2024-01-23
    • status: open --> closed-invalid
     

  • ede

    ede - 2024-01-23

    hey Ralf,

    the command line generated by duply is fine. i verified that duplicity is at fault here. i opened a ticket in that regard https://gitlab.com/duplicity/duplicity/-/issues/795 .

    consider downgrading duplicity, if the option is important to you or wait for the fix :)

    will close this as invalid as duply is working as expected. ..sunny regards ede

     

  • Ralf Schröter

    Ralf Schröter - 2024-01-24

    moin ede

    thank you for your help and your investigations. I will try your suggestion with downgrading...

     

  • ede

    ede - 2024-02-29
    • status: closed-invalid --> open
     

  • Milan Zimmermann

    Milan Zimmermann - 2024-05-08

    Hi, I have the same issue on Linux:

    duply --version
    duply version 2.5.2
    (https://duply.net)

    Using installed duplicity version 2.2.3, python 3.11.9 (/usr/bin/python3.11) 'PYTHONPATH=:/usr/lib64/python311.zip:/usr/lib64/python3.11:/usr/lib64/python3.11/lib-dynload:/usr/lib64/python3.11/site-packages:/usr/lib64/python3.11/site-packages/PIL:/usr/lib64/python3.11/_import_failed:/usr/lib/python3.11/site-packages', gpg 2.4.5 (Home: /root/.gnupg), awk 'GNU Awk 5.3.0, API 4.0, PMA Avon 8-g1, (GNU MPFR 4.2.1, GNU MP 6.3.0)', grep 'grep (GNU grep) 3.11', bash '5.2.26(1)-release (x86_64-suse-linux)'.

    I triied the duplicity suggestion using the '=' sign when defining options, by changing duply conf file line as follows:

    GPG_OPTS="--pinentry-mode=loopback" # Needed to work from cron

    But running

    duply my-backup-all-profile backup --allow-source-mismatch

    Results in error which includes this line:

    CommandLineError: argument --gpg-options: expected one argument

    Is downgrading duplicity still the only option, and if so, would you be able to suggest duplicity version which should not have this issue? I reviewed duplicity issues around this, and it is not clear which version to use. Sorry for the long message, and thanks.

     

  • ede

    ede - 2024-05-08

    hey Milan,

    turns out that the new command line parser used by duplicity is at fault and even worse that it will probably not be fixed as it was designed that way intentionally.

    good news for you though. since quite some time --pinentry-mode=loopback is not needed anymore as duplicity will set it itself if needed. so no need for duply to provide it anymore. PROBLEMSOLVED :) at least for you.

    will need to patch up duply. as a permanent fix (some) option values given to duplicity will need the equal sign bound syntax e.g. --option=value instead of the classic --option value.

    WORKAROUND for now. if you really need to provide parameters to gpg disable #GPGOPTS and add them in the end by appending DUPL_PARAMS= (NOTE the --gpg-options='...') e.g.

    DUPL_PARAMS="$DUPL_PARAMS --gpg-options='--compress-algo=bzip2 --bzip2-compress-level=9'"

     

    Last edit: ede 2024-05-08

  • ede

    ede - 2024-06-29

    can you guys please test if the new devel version fixes the issue for you?

    https://duply.net/tmp/duply.sh

     

  • ede

    ede - 2024-07-10
    • status: open --> closed-fixed
     

  • ede

    ede - 2024-07-10

    should be fixed in v2.5.3 . released now.

     

Log in to post a comment.

MongoDB Logo MongoDB