Performance issues w/ S3
tink via Duplicity-talk <[email protected]>
| Newsgroups | gmane.comp.sysutils.backup.duplicity.general |
|---|---|
| Message-ID | <CAAfwjx4bE7=xkLxm00w9eSTpo6gWunHG7mFk76EqWciNudLfww@mail.gmail.com> |
Hi, I've been banging my head against this for a while. (duplicity 2.2.3) I'm trying to backup a sizeable chunk of data to Wasabi S3 storage, and using boto3+s3 with --volsize 200 \ --s3-multipart-chunk-size 20 \ --s3-multipart-max-procs 6 \ am seeing horribly slow upload speeds (shuffling 400GB in 24 h, so roughly 5 MB/s). Using aws cli tools on the same server to the same bucket I get 80MB/s. Looking at a tcpdump during both attempts the latter looks fine and swift, the duplicity one is seeing the far end throttling me badly, every other second it tells me to back off for 1.5 seconds (setting a win-size of 0 for that period of time). What am I doing wrong here? Cheers, Tink -- A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing? A: Top-posting. Q: What is the most annoying thing in e-mail? _______________________________________________ Duplicity-talk mailing list [email protected] https://lists.nongnu.org/mailman/listinfo/duplicity-talk