Re: GPG (pub/prv keys) encrypted backups without secret key being present

On 18.05.2024 23:26, Scott Hannahs wrote:
>
>
>> On May 18, 2024, at 4:48 PM, [email protected] wrote:
>>
>> On 18.05.2024 22:03, Scott Hannahs wrote:
>>>
>>> This is something I have been following closely since I do not want to put my passphrase in scripts etc.  I use only the public keys for signing and encrypting the files.   There is a problem when a backup fails with partial upload.  My scripts will run a cleanup and then the next nights backup will work.  BUT, it seems when that happens duplicity (or the gpg calls) ask for the password and block the script until entered.  When running nightly in the background this just locks up the script.
>>>
>>> Once that happens the backup script just blocks and hangs around as a non-executing process until rebooting and subsequent runs of the script fail.  I set a “Passphrase” variable to blank but it still asks somehow.  Maybe I don’t have a gpg-option set correctly?
>>>
>>> I am fine with one night failing the update due to a partial and then running cleanup and all is well the next night.  But having everything block when it fails is not cool.
>>
>>
>> hi Scott, long time no hear, hope you are well :)
>>
>> can you please provide
>> - operating system
>> - duplicity & gpg version
>> - full command line
>> ?
>>
>> and yes, assuming that gpg does not find a matching secret key or you provide an empty or wrong PASSPHRASE and duplicity needs to decrypt it should fail but not hang indefinitely.
>>
>> sunny regards ..ede
>
> Ede,
>
> It has been awhile, but duplicity has been cranking along doing its thing without a hiccup.  I have been following your substantial help to users on the listserv.  I have updated to the latest version with fink package manager build.  But last week we had a few tornadoes run though town and the network was a bit spotty for a day or two.  My place was untouched but some of my friends had major damage.  My lab had significant damage but is still functioning.
>
> Versions:
>
>     MacOS 14.4.1
>     % duplicity --version
>     duplicity 2.2.3 March 20, 2024
>
>     % gpg --version
>     gpg (GnuPG) 2.2.42
>     libgcrypt 1.10.3
>     Copyright (C) 2023 g10 Code GmbH
>     License GNU GPL-3.0-or-later <https://gnu.org/licenses/gpl.html>
>     This is free software: you are free to change and redistribute it.
>     There is NO WARRANTY, to the extent permitted by law.
>
>     Home: /Users/sth/.gnupg
>     Supported algorithms:
>     Pubkey: RSA, ELG, DSA, ECDH, ECDSA, EDDSA
>     Cipher: IDEA, 3DES, CAST5, BLOWFISH, AES, AES192, AES256, TWOFISH,
>              CAMELLIA128, CAMELLIA192, CAMELLIA256
>     Hash: SHA1, RIPEMD160, SHA256, SHA384, SHA512, SHA224
>     Compression: Uncompressed, ZIP, ZLIB, BZIP2
>
> Command and output
>
>     nice -n19 /opt/sw/bin/duplicity backup --full-if-older-than 49D --num-retries 5 --tempdir /var/tmp/duplicity --volsize 500 --encrypt-key F5A49F68 --sign-key 71CA65FE --gpg-binary /opt/sw/bin/gpg --gpg-options=--default-key=F5A49F68 --force --verbosity Notice --asynchronous-upload --s3-multipart-chunk-size 100 --exclude **/Documents/Virtual?Machines.localized --exclude **/Library/Application?Support/FileProvider --exclude **/Library/Application?Support/Google --exclude **/Library/Application?Support/Java --exclude **/Library/Application?Support/Microsoft --exclude **/Library/Application?Support/WebEx?Folder --exclude **/Library/Arq/Cache.noindex --exclude **/Library/Caches --exclude **/Library/CloudStorage --exclude **/Library/Containers --exclude **/Library/Developer --exclude **/Library/Group?Containers --exclude **/Library/Mail --exclude **/Library/Mail?Downloads --exclude **/Library/Messages --exclude **/Library/Saved?Application?State --exclude
>     **/Library/iTunes/*Updates --s3-endpoint-url https://<remote.host>.io /Users/sth/Documents s3://sth-backup/documents
>
>     Use of the --asynchronous-upload option is experimental and not safe for production! There are reported cases of undetected data loss during upload. Be aware and periodically verify your backups to be safe.
>     Local and Remote metadata are synchronized, no sync needed.
>     Last full backup left a partial set, restarting.
>     Last full backup date: Fri May 10 13:42:17 2024
>
>     RESTART: Volumes 88 to 88 failed to upload before termination.
>               Restarting backup at volume 88.
>
> I think this is where it locked up.  When I ran the script manually the terminal showed a request for a password.  When I hit just return it went on to the next stage and did a cleanup.  Upon looking at the system today, I noticed that there were no backups since Monday and that the script was just sitting in the background and not executing.  The script cycles through several directories and backs up each one to an S3 repository.

you are using different keys `--encrypt-key F5A49F68 --sign-key 71CA65FE`.
1. do you export a PASSPHRASE env var
2. which keys (sec/pub) are available in the gpg keyring?
3. are they passphrase protected?

> Today I removed the --asynchronous-upload option since it is still deprecated, but I remember so discussion that it was working correctly awhile ago.

under rare circumstances `duplicity --asynchronous-upload` does not successfully upload all files and leaves backup chains with missing volumes.

> There is also a warning: "Error accessing possibly locked file” which I don’t understand a “possibly” locked file, either a file is readable or not.  In this case the file is readable by my account (which is also the account that runs the script).  I can exclude some of the two dozen directories but I think I want some of them backed up.  Other backup programs do not have this issue.

well, locked means read/write protected by some other program as it might write to the file at this time. can you give examples that occur regularly? to circumvent that you back up snapshots ideally, not running systems :)

sunny regards ..ede


_______________________________________________
Duplicity-talk mailing list
[email protected]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk