Re: Performance issues w/ S3

> On Jul 12, 2024, at 8:19 PM, tink via Duplicity-talk <[email protected]> wrote:
> 
> On Sat, 13 Jul 2024 at 08:02, edgar.soldin--- via Duplicity-talk <[email protected] <mailto:[email protected]>> wrote:
>> 
>> On 12.07.2024 19:57, Thomas Laubrock via Duplicity-talk wrote:
>> > Hi,
>> >
>> > Duplicity has no code logic for throttle bandwidth.
>> >
>> > Volume creation and transfer via backend are pretty well isolated.
>> >
>> > I would do two tests to get a better picture.
>> > Check speed of duplicity running the same backup to a local filesystem using file://. (You can cancel the backup if you have an idea about the speed before running out of disk space)
>> 
>> hmm, if he already did compare against aws cli and that's way faster, why do it?
>> 
>> > And run a boto3 code snippet and upload a 200MB file to your storage independently from duplicity.
>> 
>> not sure they are python ppl like that. but surely it'd decouple the issue from duplicity.
>> 
>> > Then you should see what's causing the bottleneck.
>> 
>> still smelling like boto3 has some issue there ;).. ede
> 
> Fair comments, Edgar.
> I'm wondering whether anyone else out there is using wasabi as an S3 backend.
> The funny thing is that they even have a document dedicated to using duplicity for backups to a bucket on their infra.
> 
> Cheers,
> Tink

I use Wasabi as a back end since the pricing is so much less than AWS.  They are unattended automated late night backups but I haven’t seen a noticeable of a slowdown.  When I updated my net connection from 400Mb/s down and 10Mb/sec up (ie comcast) to 1,000 Mb/Sec down and 1,000 Mb/Sec up things are noticeably faster.


Using Boto3 1.34.139. / Duplicity 3.0.0, installed on MacOS with Fink Package manager

-Scott

_______________________________________________
Duplicity-talk mailing list
[email protected]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk