Re: How to connect to backblaze?
Scott Hannahs via Duplicity-talk <[email protected]> Tue, 30 Sep 2025 17:10:08 -0400
| Newsgroups | gmane.comp.sysutils.backup.duplicity.general |
|---|---|
| Message-ID | <[email protected]> |
--===============2846737979126557261==
Content-Type: multipart/alternative;
boundary="Apple-Mail=_9A744116-C8BF-405D-B518-A0424F9948DA"
--Apple-Mail=_9A744116-C8BF-405D-B518-A0424F9948DA
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
charset=utf-8
ede,
I will try to stay on the list, will try to document it here on how I =
have it working/. I thought I had. I installed duplicity using the =
fink package manager. It listed as passing all tests. Since I manage =
that package possibly there is something I missed in the install phase. =
I will check.
I see that the target is s3:///<bucket/directory with 3 slashes not the =
2 I had. But that doesn=E2=80=99t fix it. It now lists that the bucket =
that I created and see through the BB web interface is there, does not =
exist.
But I think removing the bucket from the URL is working. That seems =
counter to the AWS hint you gave me, but I am going to let it run and =
check it out. I thought that the moving to the newer S3 spec had the =
bucket in the URL for a reason. Seemed weird but there it was.
Thanks, for the pointers, I think I have it working and outline the =
details below.
Here is the process for multiple S3 accounts. Note that the S3 backend =
does not automatically create a bucket so that has to be prepared. =
However the backend will create the needed directories in the target =
bucket.
First setup a separate .boto file in your home folder for each =
host/service such as
.boto-s3.us-east.<CloudProvider>.com =
<http://boto-3.us-east.cloudprovider.com/>
containing:
[Credentials]
aws_access_key_id =3D XXXXXXXXXXXXXXXXXXXXXXXXX
aws_secret_access_key =3D YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY
in the shell script (NN) is the number of days between full backups.
BUCKET=3D=E2=80=9C<BUCKET NAME>"
REMOTE_HOST=3D"s3.us-east.<CloudProvider>.com"
export BOTO_CONFIG=3D"${HOME}/.boto-${REMOTE_HOST}"
ENDURL=3D"--s3-endpoint-url https://${REMOTE_HOST} =
<https://${remote_host}/>=E2=80=9D
TARGET=3D"s3:///${BUCKET}/ <s3:///$%7BBUCKET%7D/><folder in bucket>=E2=80=9D=
SOURCE=3D=E2=80=9C<path to source directory>"
duplicity backup --full-if-older-than <NN>D --volsize 500 =
--s3-multipart-chunk-size 500 $ENDURL =E2=80=9C$SOURCE" =E2=80=9C$TARGET"
NOTE the three (3) slashes in the TARGET variable.
Thanks for the help, seems to be working now.
-Scott
> On Sep 29, 2025, at 5:17=E2=80=AFPM, edgar.soldin--- via =
Duplicity-talk <[email protected]> wrote:
>=20
> hey Scott,
>=20
> On 9/29/2025 10:55 PM, Scott Hannahs wrote:
>> I have boto3 version 1.34.144 installed
>> I am using duplicity version 3.0.4
>=20
> sounds reasonable=20
>> I think that is the difference where I am accessing the Dreamhost =
with the old style S3 with the bucket name in the target and trying to =
change to the bucket name in the URL. But the obvious solution is not =
working?
>> That should be the first command version:
>> nice -n19 /opt/sw/bin/duplicity backup --full-if-older-than 49D =
--num-retries 5 --tempdir /var/tmp/duplicity --volsize 500 --encrypt-key =
XXXXXXXX --sign-key YYYYYYYY --gpg-binary /opt/sw/bin/gpg --force =
--verbosity Notice --s3-multipart-chunk-size 500 --s3-endpoint-url =
https://*_<BUCKETNAME>._**s3.us-east-005.backblazeb2.com* =
<http://s3.us-east-005.backblazeb2.com/> <FOLDER TO BACKUP> =
_*s3://<FOLDERNAME>*_
>=20
> as per man page it should look like
>=20
> duplicity /some/path s3:///<bucket>/<subfolder> =
<s3:///%3Cbucket%3E/%3Csubfolder%3E> --s3-endpoint-url <url> notice the =
bucket in the s3:// url. i'd try only =
'https://s3.us-east-005.backblazeb2.com =
<https://s3.us-east-005.backblazeb2.com/>' as endpoint url as the bucket =
is given independently
>=20
> =
https://www.backblaze.com/docs/cloud-storage-call-the-s3-compatible-api
>=20
>> So I put the host in the s3-endpoint-url parameter and the the target =
of _*s3://<FOLDERNAME>*_
>> I have the .boto configuration file with the =E2=80=9Caws_access_key_id=
=E2=80=9D and =E2=80=9Caws_secret_access_key=E2=80=9D set.
>> One question I have is if I use Backblaze for an S3 backup and =
Dreamhost for an S3 backup I am exporting a different BOTO_CONFIG =
variable for each with the hopefully correct Key and Secret Key.
>> I am getting a warning about another backend, but I think that =
isn=E2=80=99t the problem.
>> Import of duplicity.backends._testbackend Failed
>=20
> that shouldn#t happen. how did you install duplicity?=20
>> Is there anyway to test the boto connection and see how it is =
failing. If I move the config file I get a warning about no such config =
file so it seems to be finding it. But the =E2=80=9CHeadBucket=E2=80=9D =
operation is failing?
>> Am I not using the correct target? I am using s3://<directory> but =
s3:<directory> complains it is not a URL. Even though the bucket name =
is in the URL, should it be in the target specification as well?
>=20
> as said above. add the bucket. the erro says it misses the bucket.=20
>> That help on the Boto3 issue needs some examples?
>=20
> there are lot's of s3 providers by now. maybe the man page is not the =
place for actual examples for each of them?
>=20
> ..ede
>=20
>>> On Sep 29, 2025, at 11:03=E2=80=AFAM, edgar.soldin--- via =
Duplicity-talk <[email protected] =
<mailto:[email protected]>> wrote:
>>>=20
>>> assuming you use a recent duplicity 3.x . we retired the old s3/boto =
backend as it was deprecated long ago and switched to boto3 as default. =
that changed the parameter syntax. read
>>> https://duplicity.us/stable/duplicity.1.html#a-note-on-amazon-s3 =
<https://duplicity.us/stable/duplicity.1.html#a-note-on-amazon-s3>
>>>=20
>>> main difference boto3:// does not want a host in the url but as =
extra parameter instead.
>>>=20
>>> does that help? ..ede/duply.net <http://duply.net/> =
<http://duply.net/>
>>>=20
>>> On 9/29/2025 4:49 PM, Scott Hannahs via Duplicity-talk wrote:
>>>> I am trying to switch cloud providers from Dreamhost/Wasabi to =
BackBlaze due to the pricing structure being much better.
>>>> The Backblaze B2 storage is S3 compliant and I think Dreamhost is =
just repackaaging it for their DreamObjects and adding an extra =
surcharge.
>>>> it uses a global bucket naming scheme in the domain name but I =
can=E2=80=99t get the S3 backend to work. I gather that the B2 backend =
is no longer supported? The S3 should be successful here AFAICT.
>>>> The domain <BUCKETNAME>.s3.us-east-005.backblazeb2.com =
<http://s3.us-east-005.backblazeb2.com/> =
<http://s3.us-east-005.backblazeb2.com/><http://s3.us-east-005.backblazeb2=
.com =
<http://s3.us-east-005.backblazeb2.com/><http://s3.us-east-005.backblazeb2=
.com/>> does exist and I can ping it.
>>>> My command line (minus a bunch of exclude statements:
>>>> nice -n19 /opt/sw/bin/duplicity backup --full-if-older-than 49D =
--num-retries 5 --tempdir /var/tmp/duplicity --volsize 500 --encrypt-key =
XXXXXXXX --sign-key YYYYYYYY --gpg-binary /opt/sw/bin/gpg --force =
--verbosity Notice --s3-multipart-chunk-size 500 --s3-endpoint-url =
https://<BUCKETNAME>.s3.us-east-005.backblazeb2.com <FOLDER TO BACKUP> =
s3://<FOLDERNAME>
>>>> I have also tried the following
>>>> nice -n19 /opt/sw/bin/duplicity backup --full-if-older-than 49D =
--num-retries 5 --tempdir /var/tmp/duplicity --volsize 500 --encrypt-key =
XXXXXXXX --sign-key YYYYYYYY --gpg-binary /opt/sw/bin/gpg --force =
--verbosity Notice --s3-multipart-chunk-size 500 --s3-endpoint-url =
https://s3.us-east-005.backblazeb2.com <FOLDER TO BACKUP> =
s3://<BUCKETNAME>/<FOLDERNAME>
>>>> I get the error:
>>>> ClientError: An error occurred (403) when calling the HeadBucket =
operation: Forbidden for both.
>>>> If I use Verbosity=3DDebug
>>>> Backtrace of previous error: Traceback (innermost last):
>>>> File =
"/opt/sw/lib/python3.10/site-packages/duplicity/backend.py", line 375, =
in inner_retry
>>>> return fn(self, *args)
>>>> File =
"/opt/sw/lib/python3.10/site-packages/duplicity/backend.py", line 631, =
in list
>>>> return [tobytes(x) for x in self.backend._list()]
>>>> File =
"/opt/sw/lib/python3.10/site-packages/duplicity/backends/s3_boto3_backend.=
py", line 209, in _list
>>>> self.reset_connection()
>>>> File =
"/opt/sw/lib/python3.10/site-packages/duplicity/backends/s3_boto3_backend.=
py", line 105, in reset_connection
>>>> self.s3.meta.client.head_bucket(Bucket=3Dself.bucket_name)
>>>> File =
"/opt/sw/lib/python3.10/site-packages/botocore/client.py", line 565, in =
_api_call
>>>> return self._make_api_call(operation_name, kwargs)
>>>> File =
"/opt/sw/lib/python3.10/site-packages/botocore/client.py", line 1021, in =
_make_api_call
>>>> raise error_class(parsed_response, operation_name)
>>>> botocore.exceptions.ClientError: An error occurred (403) when =
calling the HeadBucket operation: Forbidden
>>>> This seems to work with the older Dreamhost S3 storage so there is =
something in my setup that is wrong? Any advice from someone =
successfullly using Backblaze would be greatly appreciated.
>>>> -Scott
>>>> _______________________________________________
>>>> Duplicity-talk mailing list
>>>> [email protected]
>>>> https://lists.nongnu.org/mailman/listinfo/duplicity-talk
>>>=20
>>>=20
>>> _______________________________________________
>>> Duplicity-talk mailing list
>>> [email protected] <mailto:[email protected]> =
<mailto:[email protected]>
>>> https://lists.nongnu.org/mailman/listinfo/duplicity-talk =
<https://lists.nongnu.org/mailman/listinfo/duplicity-talk>
>=20
>=20
> _______________________________________________
> Duplicity-talk mailing list
> [email protected] <mailto:[email protected]>
> https://lists.nongnu.org/mailman/listinfo/duplicity-talk
--Apple-Mail=_9A744116-C8BF-405D-B518-A0424F9948DA
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
charset=utf-8
<html><head><meta http-equiv=3D"content-type" content=3D"text/html; =
charset=3Dutf-8"></head><body style=3D"overflow-wrap: break-word; =
-webkit-nbsp-mode: space; line-break: after-white-space;"><div =
dir=3D"auto" style=3D"overflow-wrap: break-word; -webkit-nbsp-mode: =
space; line-break: after-white-space;">ede,<div><br></div><div>I will =
try to stay on the list, will try to document it here on how I have it =
working/. I thought I had. I installed duplicity using the =
fink package manager. It listed as passing all tests. Since =
I manage that package possibly there is something I missed in the =
install phase. I will check.</div><div><br></div><div>I see that =
the target is s3:///<bucket/directory with 3 slashes not the 2 I had. =
But that doesn=E2=80=99t fix it. It now lists that the =
bucket that I created and see through the BB web interface is there, =
does not exist.</div><div><br></div><div>But I think removing the bucket =
from the URL is working. That seems counter to the AWS hint you =
gave me, but I am going to let it run and check it out. I thought =
that the moving to the newer S3 spec had the bucket in the URL for a =
reason. Seemed weird but there it =
was.</div><div><br></div><div>Thanks, for the pointers, I think I =
have it working and outline the details =
below.</div><div><br></div><div>Here is the process for multiple S3 =
accounts. Note that the S3 backend does not automatically create a =
bucket so that has to be prepared. However the backend will create =
the needed directories in the target =
bucket.</div><div><br></div><div>First setup a separate .boto file in =
your home folder for each host/service such =
as</div><div><br></div><div><font face=3D"Menlo">.<a =
href=3D"http://boto-3.us-east.CloudProvider.com">boto-s<span =
style=3D"caret-color: rgb(0, 0, 0); color: rgb(0, 0, =
0);">3.us-east.<CloudProvider>.com</span></a></font></div><div><span=
style=3D"caret-color: rgb(0, 0, 0); color: rgb(0, 0, =
0);">containing:</span></div></div><blockquote style=3D"margin: 0 0 0 =
40px; border: none; padding: 0px;"><div dir=3D"auto" =
style=3D"overflow-wrap: break-word; -webkit-nbsp-mode: space; =
line-break: after-white-space;"><div><span style=3D"caret-color: rgb(0, =
0, 0); color: rgb(0, 0, 0);"><font =
face=3D"Menlo"><div>[Credentials]</div><div>aws_access_key_id =3D =
XXXXXXXXXXXXXXXXXXXXXXXXX</div><div>aws_secret_access_key =3D =
YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY</div></font></span></div></div></blockquot=
e><div dir=3D"auto" style=3D"overflow-wrap: break-word; =
-webkit-nbsp-mode: space; line-break: =
after-white-space;"><div><br></div><div><br></div><div>in the shell =
script (NN) is the number of days between full =
backups.</div></div><blockquote style=3D"margin: 0 0 0 40px; border: =
none; padding: 0px;"><div dir=3D"auto" style=3D"overflow-wrap: =
break-word; -webkit-nbsp-mode: space; line-break: =
after-white-space;"><div><font face=3D"Menlo">BUCKET=3D=E2=80=9C<BUCKET=
NAME>"</font></div></div><div dir=3D"auto" style=3D"overflow-wrap: =
break-word; -webkit-nbsp-mode: space; line-break: =
after-white-space;"><div><font =
face=3D"Menlo">REMOTE_HOST=3D"s3.us-east.<<span style=3D"caret-color: =
rgb(0, 0, 0); color: rgb(0, 0, =
0);">CloudProvider>.</span>com"</font></div></div><div dir=3D"auto" =
style=3D"overflow-wrap: break-word; -webkit-nbsp-mode: space; =
line-break: after-white-space;"><div><font face=3D"Menlo">export =
BOTO_CONFIG=3D"${HOME}/.boto-${REMOTE_HOST}"</font></div></div><div =
dir=3D"auto" style=3D"overflow-wrap: break-word; -webkit-nbsp-mode: =
space; line-break: after-white-space;"><div><font =
face=3D"Menlo">ENDURL=3D"--s3-endpoint-url <a =
href=3D"https://$%7Bremote_host%7D">https://${REMOTE_HOST}</a>=E2=80=9D</f=
ont></div></div><div dir=3D"auto" style=3D"overflow-wrap: break-word; =
-webkit-nbsp-mode: space; line-break: after-white-space;"><div><font =
face=3D"Menlo">TARGET=3D"<a =
href=3D"s3:///$%7BBUCKET%7D/">s3:///${BUCKET}/</a><folder in =
bucket>=E2=80=9D</font></div></div><div dir=3D"auto" =
style=3D"overflow-wrap: break-word; -webkit-nbsp-mode: space; =
line-break: after-white-space;"><div><font =
face=3D"Menlo">SOURCE=3D=E2=80=9C<path to source =
directory>"</font></div></div><div dir=3D"auto" style=3D"overflow-wrap:=
break-word; -webkit-nbsp-mode: space; line-break: =
after-white-space;"><div><font face=3D"Menlo"><br></font></div></div><div =
dir=3D"auto" style=3D"overflow-wrap: break-word; -webkit-nbsp-mode: =
space; line-break: after-white-space;"><div><font face=3D"Menlo">duplicity=
backup --full-if-older-than <NN>D --volsize =
500 --s3-multipart-chunk-size 500 $ENDURL =E2=80=9C$SOURCE" =
=E2=80=9C$TARGET"</font></div></div></blockquote><div dir=3D"auto" =
style=3D"overflow-wrap: break-word; -webkit-nbsp-mode: space; =
line-break: after-white-space;"><div><br></div><div>NOTE the three (3) =
slashes in the TARGET variable.</div><div><br></div><div>Thanks for the =
help, seems to be working =
now.</div><div><br></div><div>-Scott</div><div><br></div><div><div><blockq=
uote type=3D"cite"><div>On Sep 29, 2025, at 5:17=E2=80=AFPM, =
edgar.soldin--- via Duplicity-talk <[email protected]> =
wrote:</div><br class=3D"Apple-interchange-newline"><div><meta =
charset=3D"UTF-8"><span style=3D"caret-color: rgb(0, 0, 0); font-family: =
LucidaGrande; font-size: 14px; font-style: normal; font-variant-caps: =
normal; font-weight: 400; letter-spacing: normal; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; =
word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: =
none; float: none; display: inline !important;">hey Scott,</span><br =
style=3D"caret-color: rgb(0, 0, 0); font-family: LucidaGrande; =
font-size: 14px; font-style: normal; font-variant-caps: normal; =
font-weight: 400; letter-spacing: normal; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; =
word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: =
none;"><br style=3D"caret-color: rgb(0, 0, 0); font-family: =
LucidaGrande; font-size: 14px; font-style: normal; font-variant-caps: =
normal; font-weight: 400; letter-spacing: normal; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; =
word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: =
none;"><span style=3D"caret-color: rgb(0, 0, 0); font-family: =
LucidaGrande; font-size: 14px; font-style: normal; font-variant-caps: =
normal; font-weight: 400; letter-spacing: normal; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; =
word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: =
none; float: none; display: inline !important;">On 9/29/2025 10:55 PM, =
Scott Hannahs wrote:</span><br style=3D"caret-color: rgb(0, 0, 0); =
font-family: LucidaGrande; font-size: 14px; font-style: normal; =
font-variant-caps: normal; font-weight: 400; letter-spacing: normal; =
text-align: start; text-indent: 0px; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none;"><blockquote type=3D"cite" style=3D"font-family: =
LucidaGrande; font-size: 14px; font-style: normal; font-variant-caps: =
normal; font-weight: 400; letter-spacing: normal; orphans: auto; =
text-align: start; text-indent: 0px; text-transform: none; white-space: =
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none;">I have boto3 version 1.34.144 =
installed<br>I am using duplicity version 3.0.4<br></blockquote><br =
style=3D"caret-color: rgb(0, 0, 0); font-family: LucidaGrande; =
font-size: 14px; font-style: normal; font-variant-caps: normal; =
font-weight: 400; letter-spacing: normal; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; =
word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: =
none;"><span style=3D"caret-color: rgb(0, 0, 0); font-family: =
LucidaGrande; font-size: 14px; font-style: normal; font-variant-caps: =
normal; font-weight: 400; letter-spacing: normal; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; =
word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: =
none; float: none; display: inline !important;">sounds reasonable<span =
class=3D"Apple-converted-space"> </span></span><br =
style=3D"caret-color: rgb(0, 0, 0); font-family: LucidaGrande; =
font-size: 14px; font-style: normal; font-variant-caps: normal; =
font-weight: 400; letter-spacing: normal; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; =
word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: =
none;"><blockquote type=3D"cite" style=3D"font-family: LucidaGrande; =
font-size: 14px; font-style: normal; font-variant-caps: normal; =
font-weight: 400; letter-spacing: normal; orphans: auto; text-align: =
start; text-indent: 0px; text-transform: none; white-space: normal; =
widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none;">I think that is the difference where I am =
accessing the Dreamhost with the old style S3 with the bucket name in =
the target and trying to change to the bucket name in the URL. But =
the obvious solution is not working?<br>That should be the first command =
version:<br> nice -n19 /opt/sw/bin/duplicity backup =
--full-if-older-than 49D --num-retries 5 --tempdir /var/tmp/duplicity =
--volsize 500 --encrypt-key XXXXXXXX --sign-key YYYYYYYY --gpg-binary =
/opt/sw/bin/gpg --force --verbosity Notice --s3-multipart-chunk-size 500 =
--s3-endpoint-url<span class=3D"Apple-converted-space"> </span><a =
href=3D"https://*_<BUCKETNAME>._**s3.us-east-005.backblazeb2.com*">https:/=
/*_<BUCKETNAME>._**s3.us-east-005.backblazeb2.com*</a><span =
class=3D"Apple-converted-space"> </span><<a =
href=3D"http://s3.us-east-005.backblazeb2.com/">http://s3.us-east-005.back=
blazeb2.com/</a>> <FOLDER TO BACKUP> =
_*s3://<FOLDERNAME>*_<br></blockquote><br style=3D"caret-color: =
rgb(0, 0, 0); font-family: LucidaGrande; font-size: 14px; font-style: =
normal; font-variant-caps: normal; font-weight: 400; letter-spacing: =
normal; text-align: start; text-indent: 0px; text-transform: none; =
white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none;"><span style=3D"caret-color: rgb(0, 0, 0); =
font-family: LucidaGrande; font-size: 14px; font-style: normal; =
font-variant-caps: normal; font-weight: 400; letter-spacing: normal; =
text-align: start; text-indent: 0px; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none; float: none; display: inline !important;">as per =
man page it should look like</span><br style=3D"caret-color: rgb(0, 0, =
0); font-family: LucidaGrande; font-size: 14px; font-style: normal; =
font-variant-caps: normal; font-weight: 400; letter-spacing: normal; =
text-align: start; text-indent: 0px; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none;"><br style=3D"caret-color: rgb(0, 0, 0); =
font-family: LucidaGrande; font-size: 14px; font-style: normal; =
font-variant-caps: normal; font-weight: 400; letter-spacing: normal; =
text-align: start; text-indent: 0px; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none;"><span style=3D"caret-color: rgb(0, 0, 0); =
font-family: LucidaGrande; font-size: 14px; font-style: normal; =
font-variant-caps: normal; font-weight: 400; letter-spacing: normal; =
text-align: start; text-indent: 0px; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none; float: none; display: inline =
!important;"> duplicity /some/path<span =
class=3D"Apple-converted-space"> </span></span><a =
href=3D"s3:///%3Cbucket%3E/%3Csubfolder%3E" style=3D"font-family: =
LucidaGrande; font-size: 14px; font-style: normal; font-variant-caps: =
normal; font-weight: 400; letter-spacing: normal; orphans: auto; =
text-align: start; text-indent: 0px; text-transform: none; white-space: =
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: =
0px;">s3:///<bucket>/<subfolder></a><span =
style=3D"caret-color: rgb(0, 0, 0); font-family: LucidaGrande; =
font-size: 14px; font-style: normal; font-variant-caps: normal; =
font-weight: 400; letter-spacing: normal; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; =
word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: =
none; float: none; display: inline !important;"><span =
class=3D"Apple-converted-space"> </span>--s3-endpoint-url =
<url> notice the bucket in the s3:// url. i'd try only '</span><a =
href=3D"https://s3.us-east-005.backblazeb2.com/" style=3D"font-family: =
LucidaGrande; font-size: 14px; font-style: normal; font-variant-caps: =
normal; font-weight: 400; letter-spacing: normal; orphans: auto; =
text-align: start; text-indent: 0px; text-transform: none; white-space: =
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: =
0px;">https://s3.us-east-005.backblazeb2.com</a><span =
style=3D"caret-color: rgb(0, 0, 0); font-family: LucidaGrande; =
font-size: 14px; font-style: normal; font-variant-caps: normal; =
font-weight: 400; letter-spacing: normal; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; =
word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: =
none; float: none; display: inline !important;">' as endpoint url as the =
bucket is given independently</span><br style=3D"caret-color: rgb(0, 0, =
0); font-family: LucidaGrande; font-size: 14px; font-style: normal; =
font-variant-caps: normal; font-weight: 400; letter-spacing: normal; =
text-align: start; text-indent: 0px; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none;"><br style=3D"caret-color: rgb(0, 0, 0); =
font-family: LucidaGrande; font-size: 14px; font-style: normal; =
font-variant-caps: normal; font-weight: 400; letter-spacing: normal; =
text-align: start; text-indent: 0px; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none;"><a =
href=3D"https://www.backblaze.com/docs/cloud-storage-call-the-s3-compatibl=
e-api" style=3D"font-family: LucidaGrande; font-size: 14px; font-style: =
normal; font-variant-caps: normal; font-weight: 400; letter-spacing: =
normal; orphans: auto; text-align: start; text-indent: 0px; =
text-transform: none; white-space: normal; widows: auto; word-spacing: =
0px; -webkit-text-stroke-width: =
0px;">https://www.backblaze.com/docs/cloud-storage-call-the-s3-compatible-=
api</a><br style=3D"caret-color: rgb(0, 0, 0); font-family: =
LucidaGrande; font-size: 14px; font-style: normal; font-variant-caps: =
normal; font-weight: 400; letter-spacing: normal; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; =
word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: =
none;"><br style=3D"caret-color: rgb(0, 0, 0); font-family: =
LucidaGrande; font-size: 14px; font-style: normal; font-variant-caps: =
normal; font-weight: 400; letter-spacing: normal; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; =
word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: =
none;"><blockquote type=3D"cite" style=3D"font-family: LucidaGrande; =
font-size: 14px; font-style: normal; font-variant-caps: normal; =
font-weight: 400; letter-spacing: normal; orphans: auto; text-align: =
start; text-indent: 0px; text-transform: none; white-space: normal; =
widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none;">So I put the host in the s3-endpoint-url =
parameter and the the target of _*s3://<FOLDERNAME>*_<br>I have =
the .boto configuration file with the =E2=80=9Caws_access_key_id=E2=80=9D =
and =E2=80=9Caws_secret_access_key=E2=80=9D set.<br>One question I have =
is if I use Backblaze for an S3 backup and Dreamhost for an S3 backup I =
am exporting a different BOTO_CONFIG variable for each with the =
hopefully correct Key and Secret Key.<br>I am getting a warning about =
another backend, but I think that isn=E2=80=99t the problem.<br>Import =
of duplicity.backends._testbackend Failed<br></blockquote><br =
style=3D"caret-color: rgb(0, 0, 0); font-family: LucidaGrande; =
font-size: 14px; font-style: normal; font-variant-caps: normal; =
font-weight: 400; letter-spacing: normal; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; =
word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: =
none;"><span style=3D"caret-color: rgb(0, 0, 0); font-family: =
LucidaGrande; font-size: 14px; font-style: normal; font-variant-caps: =
normal; font-weight: 400; letter-spacing: normal; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; =
word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: =
none; float: none; display: inline !important;">that shouldn#t happen. =
how did you install duplicity?<span =
class=3D"Apple-converted-space"> </span></span><br =
style=3D"caret-color: rgb(0, 0, 0); font-family: LucidaGrande; =
font-size: 14px; font-style: normal; font-variant-caps: normal; =
font-weight: 400; letter-spacing: normal; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; =
word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: =
none;"><blockquote type=3D"cite" style=3D"font-family: LucidaGrande; =
font-size: 14px; font-style: normal; font-variant-caps: normal; =
font-weight: 400; letter-spacing: normal; orphans: auto; text-align: =
start; text-indent: 0px; text-transform: none; white-space: normal; =
widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none;">Is there anyway to test the boto connection and =
see how it is failing. If I move the config file I get a warning =
about no such config file so it seems to be finding it. But the =
=E2=80=9CHeadBucket=E2=80=9D operation is failing?<br>Am I not using the =
correct target? I am using s3://<directory> but =
s3:<directory> complains it is not a URL. Even though the =
bucket name is in the URL, should it be in the target specification as =
well?<br></blockquote><br style=3D"caret-color: rgb(0, 0, 0); =
font-family: LucidaGrande; font-size: 14px; font-style: normal; =
font-variant-caps: normal; font-weight: 400; letter-spacing: normal; =
text-align: start; text-indent: 0px; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none;"><span style=3D"caret-color: rgb(0, 0, 0); =
font-family: LucidaGrande; font-size: 14px; font-style: normal; =
font-variant-caps: normal; font-weight: 400; letter-spacing: normal; =
text-align: start; text-indent: 0px; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none; float: none; display: inline !important;">as said =
above. add the bucket. the erro says it misses the bucket.<span =
class=3D"Apple-converted-space"> </span></span><br =
style=3D"caret-color: rgb(0, 0, 0); font-family: LucidaGrande; =
font-size: 14px; font-style: normal; font-variant-caps: normal; =
font-weight: 400; letter-spacing: normal; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; =
word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: =
none;"><blockquote type=3D"cite" style=3D"font-family: LucidaGrande; =
font-size: 14px; font-style: normal; font-variant-caps: normal; =
font-weight: 400; letter-spacing: normal; orphans: auto; text-align: =
start; text-indent: 0px; text-transform: none; white-space: normal; =
widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none;">That help on the Boto3 issue needs some =
examples?<br></blockquote><br style=3D"caret-color: rgb(0, 0, 0); =
font-family: LucidaGrande; font-size: 14px; font-style: normal; =
font-variant-caps: normal; font-weight: 400; letter-spacing: normal; =
text-align: start; text-indent: 0px; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none;"><span style=3D"caret-color: rgb(0, 0, 0); =
font-family: LucidaGrande; font-size: 14px; font-style: normal; =
font-variant-caps: normal; font-weight: 400; letter-spacing: normal; =
text-align: start; text-indent: 0px; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none; float: none; display: inline !important;">there =
are lot's of s3 providers by now. maybe the man page is not the place =
for actual examples for each of them?</span><br style=3D"caret-color: =
rgb(0, 0, 0); font-family: LucidaGrande; font-size: 14px; font-style: =
normal; font-variant-caps: normal; font-weight: 400; letter-spacing: =
normal; text-align: start; text-indent: 0px; text-transform: none; =
white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none;"><br style=3D"caret-color: rgb(0, 0, 0); =
font-family: LucidaGrande; font-size: 14px; font-style: normal; =
font-variant-caps: normal; font-weight: 400; letter-spacing: normal; =
text-align: start; text-indent: 0px; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none;"><span style=3D"caret-color: rgb(0, 0, 0); =
font-family: LucidaGrande; font-size: 14px; font-style: normal; =
font-variant-caps: normal; font-weight: 400; letter-spacing: normal; =
text-align: start; text-indent: 0px; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none; float: none; display: inline =
!important;">..ede</span><br style=3D"caret-color: rgb(0, 0, 0); =
font-family: LucidaGrande; font-size: 14px; font-style: normal; =
font-variant-caps: normal; font-weight: 400; letter-spacing: normal; =
text-align: start; text-indent: 0px; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none;"><br style=3D"caret-color: rgb(0, 0, 0); =
font-family: LucidaGrande; font-size: 14px; font-style: normal; =
font-variant-caps: normal; font-weight: 400; letter-spacing: normal; =
text-align: start; text-indent: 0px; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none;"><blockquote type=3D"cite" style=3D"font-family: =
LucidaGrande; font-size: 14px; font-style: normal; font-variant-caps: =
normal; font-weight: 400; letter-spacing: normal; orphans: auto; =
text-align: start; text-indent: 0px; text-transform: none; white-space: =
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none;"><blockquote type=3D"cite">On Sep 29, 2025, at =
11:03=E2=80=AFAM, edgar.soldin--- via Duplicity-talk <<a =
href=3D"mailto:[email protected]">[email protected]</a>>=
; wrote:<br><br>assuming you use a recent duplicity 3.x . we retired the =
old s3/boto backend as it was deprecated long ago and switched to boto3 =
as default. that changed the parameter syntax. read<br><a =
href=3D"https://duplicity.us/stable/duplicity.1.html#a-note-on-amazon-s3">=
https://duplicity.us/stable/duplicity.1.html#a-note-on-amazon-s3</a><span =
class=3D"Apple-converted-space"> </span><<a =
href=3D"https://duplicity.us/stable/duplicity.1.html#a-note-on-amazon-s3">=
https://duplicity.us/stable/duplicity.1.html#a-note-on-amazon-s3</a>><b=
r><br>main difference boto3:// does not want a host in the url but as =
extra parameter instead.<br><br>does that help? ..ede/<a =
href=3D"http://duply.net/">duply.net</a><span =
class=3D"Apple-converted-space"> </span><<a =
href=3D"http://duply.net/">http://duply.net/</a>><br><br>On 9/29/2025 =
4:49 PM, Scott Hannahs via Duplicity-talk wrote:<br><blockquote =
type=3D"cite">I am trying to switch cloud providers from =
Dreamhost/Wasabi to BackBlaze due to the pricing structure being much =
better.<br>The Backblaze B2 storage is S3 compliant and I think =
Dreamhost is just repackaaging it for their DreamObjects and adding an =
extra surcharge.<br>it uses a global bucket naming scheme in the domain =
name but I can=E2=80=99t get the S3 backend to work. I gather that =
the B2 backend is no longer supported? The S3 should be successful =
here AFAICT.<br>The domain <BUCKETNAME>.<a =
href=3D"http://s3.us-east-005.backblazeb2.com/">s3.us-east-005.backblazeb2=
.com</a><span class=3D"Apple-converted-space"> </span><<a =
href=3D"http://s3.us-east-005.backblazeb2.com/">http://s3.us-east-005.back=
blazeb2.com/</a>><<a =
href=3D"http://s3.us-east-005.backblazeb2.com/">http://s3.us-east-005.back=
blazeb2.com</a><<a =
href=3D"http://s3.us-east-005.backblazeb2.com/">http://s3.us-east-005.back=
blazeb2.com/</a>>> does exist and I can ping it.<br>My =
command line (minus a bunch of exclude =
statements:<br> nice -n19 /opt/sw/bin/duplicity backup =
--full-if-older-than 49D --num-retries 5 --tempdir /var/tmp/duplicity =
--volsize 500 --encrypt-key XXXXXXXX --sign-key YYYYYYYY --gpg-binary =
/opt/sw/bin/gpg --force --verbosity Notice --s3-multipart-chunk-size 500 =
--s3-endpoint-url =
https://<BUCKETNAME>.s3.us-east-005.backblazeb2.com <FOLDER TO =
BACKUP> s3://<FOLDERNAME><br>I have also tried the =
following<br> nice -n19 /opt/sw/bin/duplicity backup =
--full-if-older-than 49D --num-retries 5 --tempdir /var/tmp/duplicity =
--volsize 500 --encrypt-key XXXXXXXX --sign-key YYYYYYYY --gpg-binary =
/opt/sw/bin/gpg --force --verbosity Notice --s3-multipart-chunk-size 500 =
--s3-endpoint-url https://s3.us-east-005.backblazeb2.com <FOLDER TO =
BACKUP> s3://<BUCKETNAME>/<FOLDERNAME><br>I get the =
error:<br> ClientError: An error occurred (403) when =
calling the HeadBucket operation: Forbidden for both.<br>If I use =
Verbosity=3DDebug<br> Backtrace of previous error: =
Traceback (innermost last):<br> File =
"/opt/sw/lib/python3.10/site-packages/duplicity/backend.py", line 375, =
in inner_retry<br> return =
fn(self, *args)<br> File =
"/opt/sw/lib/python3.10/site-packages/duplicity/backend.py", line 631, =
in list<br> return [tobytes(x) =
for x in =
self.backend._list()]<br> File =
"/opt/sw/lib/python3.10/site-packages/duplicity/backends/s3_boto3_backend.=
py", line 209, in _list<br> =
self.reset_connection()<br>  =
;File =
"/opt/sw/lib/python3.10/site-packages/duplicity/backends/s3_boto3_backend.=
py", line 105, in reset_connection<br> =
self.s3.meta.client.head_bucket(Bucket=3Dself.bucket_name)<br>=
File =
"/opt/sw/lib/python3.10/site-packages/botocore/client.py", line 565, in =
_api_call<br> return =
self._make_api_call(operation_name, =
kwargs)<br> File =
"/opt/sw/lib/python3.10/site-packages/botocore/client.py", line 1021, in =
_make_api_call<br> raise =
error_class(parsed_response, =
operation_name)<br> botocore.exceptions.Clien=
tError: An error occurred (403) when calling the HeadBucket operation: =
Forbidden<br>This seems to work with the older Dreamhost S3 storage so =
there is something in my setup that is wrong? Any advice from =
someone successfullly using Backblaze would be greatly =
appreciated.<br>-Scott<br>_______________________________________________<=
br>Duplicity-talk mailing =
list<br>[email protected]<br>https://lists.nongnu.org/mailman/list=
info/duplicity-talk<br></blockquote><br><br>______________________________=
_________________<br>Duplicity-talk mailing list<br><a =
href=3D"mailto:[email protected]">[email protected]</a><sp=
an class=3D"Apple-converted-space"> </span><<a =
href=3D"mailto:[email protected]">mailto:[email protected]=
</a>><br><a =
href=3D"https://lists.nongnu.org/mailman/listinfo/duplicity-talk">https://=
lists.nongnu.org/mailman/listinfo/duplicity-talk</a><span =
class=3D"Apple-converted-space"> </span><<a =
href=3D"https://lists.nongnu.org/mailman/listinfo/duplicity-talk">https://=
lists.nongnu.org/mailman/listinfo/duplicity-talk</a>><br></blockquote><=
/blockquote><br style=3D"caret-color: rgb(0, 0, 0); font-family: =
LucidaGrande; font-size: 14px; font-style: normal; font-variant-caps: =
normal; font-weight: 400; letter-spacing: normal; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; =
word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: =
none;"><br style=3D"caret-color: rgb(0, 0, 0); font-family: =
LucidaGrande; font-size: 14px; font-style: normal; font-variant-caps: =
normal; font-weight: 400; letter-spacing: normal; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; =
word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: =
none;"><span style=3D"caret-color: rgb(0, 0, 0); font-family: =
LucidaGrande; font-size: 14px; font-style: normal; font-variant-caps: =
normal; font-weight: 400; letter-spacing: normal; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; =
word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: =
none; float: none; display: inline =
!important;">_______________________________________________</span><br =
style=3D"caret-color: rgb(0, 0, 0); font-family: LucidaGrande; =
font-size: 14px; font-style: normal; font-variant-caps: normal; =
font-weight: 400; letter-spacing: normal; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; =
word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: =
none;"><span style=3D"caret-color: rgb(0, 0, 0); font-family: =
LucidaGrande; font-size: 14px; font-style: normal; font-variant-caps: =
normal; font-weight: 400; letter-spacing: normal; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; =
word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: =
none; float: none; display: inline !important;">Duplicity-talk mailing =
list</span><br style=3D"caret-color: rgb(0, 0, 0); font-family: =
LucidaGrande; font-size: 14px; font-style: normal; font-variant-caps: =
normal; font-weight: 400; letter-spacing: normal; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; =
word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: =
none;"><a href=3D"mailto:[email protected]" style=3D"font-family: =
LucidaGrande; font-size: 14px; font-style: normal; font-variant-caps: =
normal; font-weight: 400; letter-spacing: normal; orphans: auto; =
text-align: start; text-indent: 0px; text-transform: none; white-space: =
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: =
0px;">[email protected]</a><br style=3D"caret-color: rgb(0, 0, =
0); font-family: LucidaGrande; font-size: 14px; font-style: normal; =
font-variant-caps: normal; font-weight: 400; letter-spacing: normal; =
text-align: start; text-indent: 0px; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none;"><a =
href=3D"https://lists.nongnu.org/mailman/listinfo/duplicity-talk" =
style=3D"font-family: LucidaGrande; font-size: 14px; font-style: normal; =
font-variant-caps: normal; font-weight: 400; letter-spacing: normal; =
orphans: auto; text-align: start; text-indent: 0px; text-transform: =
none; white-space: normal; widows: auto; word-spacing: 0px; =
-webkit-text-stroke-width: =
0px;">https://lists.nongnu.org/mailman/listinfo/duplicity-talk</a></div></=
blockquote></div><br></div></div></body></html>=
--Apple-Mail=_9A744116-C8BF-405D-B518-A0424F9948DA--
--===============2846737979126557261==
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: base64
Content-Disposition: inline
X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KRHVwbGljaXR5
LXRhbGsgbWFpbGluZyBsaXN0CkR1cGxpY2l0eS10YWxrQG5vbmdudS5vcmcKaHR0cHM6Ly9saXN0
cy5ub25nbnUub3JnL21haWxtYW4vbGlzdGluZm8vZHVwbGljaXR5LXRhbGsK
--===============2846737979126557261==--