Performance issues w/ S3

tink via Duplicity-talk <[email protected]>
Newsgroups gmane.comp.sysutils.backup.duplicity.general
Message-ID <CAAfwjx4bE7=xkLxm00w9eSTpo6gWunHG7mFk76EqWciNudLfww@mail.gmail.com>
Hi,

I've been banging my head against this for a while. (duplicity 2.2.3)
I'm trying to backup a sizeable chunk of data to Wasabi S3 storage, and
using boto3+s3 with
   --volsize 200 \
   --s3-multipart-chunk-size 20 \
   --s3-multipart-max-procs 6 \
am seeing horribly slow upload speeds (shuffling 400GB in 24 h, so roughly
5 MB/s).

Using aws cli tools on the same server to the same bucket I get 80MB/s.
Looking at a tcpdump during both attempts the latter looks fine and swift,
the duplicity one is seeing the far end throttling me badly, every other
second it tells me to back off for 1.5 seconds (setting a win-size of 0 for
that period of time).


What am I doing wrong here?

Cheers,
Tink

--
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing in e-mail?

_______________________________________________
Duplicity-talk mailing list
[email protected]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.