Re: Performance issues w/ S3

On Fri, 12 Jul 2024 at 10:14, edgar.soldin--- via Duplicity-talk <
[email protected]> wrote:

> On 11.07.2024 23:56, tink via Duplicity-talk wrote:
> > Hi,
>
Hi Edgar,


> > I've been banging my head against this for a while. (duplicity 2.2.3)
> > I'm trying to backup a sizeable chunk of data to Wasabi S3 storage, and
> using boto3+s3 with
>
> boto+s3:// was removed a while ago. you can simply use s3:// now
>
Tried that ... no change.


>
> >     --volsize 200 \
> >     --s3-multipart-chunk-size 20 \
> >     --s3-multipart-max-procs 6 \
>
> does removing --s3-multipart-chunk-size, --s3-multipart-max-procs change
> anything?
>
As with the above change - no difference.


>
> > am seeing horribly slow upload speeds (shuffling 400GB in 24 h, so
> roughly 5 MB/s).
> >
> > Using aws cli tools on the same server to the same bucket I get 80MB/s.
> > Looking at a tcpdump during both attempts the latter looks fine and
> swift, the duplicity one is seeing the far end throttling me badly, every
> other second it tells me to back off for 1.5 seconds (setting a win-size of
> 0 for that period of time).
>
> maybe some boto3 issue? do you have the latest version?
> https://pypi.orgo/project/boto3/ <https://pypi.org/project/boto3/>

Using the boto3 that comes packaged w/ the distro, 1.34.108, so somewhat
behind but not drastically ...



>
>
> ..ede/duply.net


Cheers,
Tink


>
>
> _______________________________________________
> Duplicity-talk mailing list
> [email protected]
> https://lists.nongnu.org/mailman/listinfo/duplicity-talk
>


-- 

A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing in e-mail?

_______________________________________________
Duplicity-talk mailing list
[email protected]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk