Re: Performance issues w/ S3
"edgar.soldin--- via Duplicity-talk" <[email protected]>
| Newsgroups | gmane.comp.sysutils.backup.duplicity.general |
|---|---|
| Message-ID | <[email protected]> |
On 11.07.2024 23:56, tink via Duplicity-talk wrote: > Hi, > > I've been banging my head against this for a while. (duplicity 2.2.3) > I'm trying to backup a sizeable chunk of data to Wasabi S3 storage, and using boto3+s3 with boto+s3:// was removed a while ago. you can simply use s3:// now > --volsize 200 \ > --s3-multipart-chunk-size 20 \ > --s3-multipart-max-procs 6 \ does removing --s3-multipart-chunk-size, --s3-multipart-max-procs change anything? > am seeing horribly slow upload speeds (shuffling 400GB in 24 h, so roughly 5 MB/s). > > Using aws cli tools on the same server to the same bucket I get 80MB/s. > Looking at a tcpdump during both attempts the latter looks fine and swift, the duplicity one is seeing the far end throttling me badly, every other second it tells me to back off for 1.5 seconds (setting a win-size of 0 for that period of time). maybe some boto3 issue? do you have the latest version? https://pypi.org/project/boto3/ ..ede/duply.net _______________________________________________ Duplicity-talk mailing list [email protected] https://lists.nongnu.org/mailman/listinfo/duplicity-talk