Re: Performance issues w/ S3

So I installed Python 3.12 from the deadsnakes PPA and then followed
https://duplicity.us/stable/README.html ...

When I then try to run duplicity I get:

2024-07-22 13:46:42,781 DEBUG Backtrace of previous error: Traceback
(innermost last):
  File "/root/duplicity/lib/python3.12/site-packages/duplicity/backend.py",
line 376, in inner_retry
    return fn(self, *args)
           ^^^^^^^^^^^^^^^
  File "/root/duplicity/lib/python3.12/site-packages/duplicity/backend.py",
line 632, in list
    return [tobytes(x) for x in self.backend._list()]
                                ^^^^^^^^^^^^^^^^^^^^
  File
"/root/duplicity/lib/python3.12/site-packages/duplicity/backends/s3_boto3_backend.py",
line 209, in _list
    self.reset_connection()
  File
"/root/duplicity/lib/python3.12/site-packages/duplicity/backends/s3_boto3_backend.py",
line 92, in reset_connection
    import boto3
  File "/root/duplicity/lib/python3.12/site-packages/boto3/__init__.py",
line 17, in <module>
    from boto3.session import Session
  File "/root/duplicity/lib/python3.12/site-packages/boto3/session.py",
line 17, in <module>
    import botocore.session
  File "/root/duplicity/lib/python3.12/site-packages/botocore/session.py",
line 26, in <module>
    import botocore.client
  File "/root/duplicity/lib/python3.12/site-packages/botocore/client.py",
line 15, in <module>
    from botocore import waiter, xform_name
  File "/root/duplicity/lib/python3.12/site-packages/botocore/waiter.py",
line 18, in <module>
    from botocore.docs.docstring import WaiterDocstring
  File
"/root/duplicity/lib/python3.12/site-packages/botocore/docs/__init__.py",
line 15, in <module>
    from botocore.docs.service import ServiceDocumenter
  File
"/root/duplicity/lib/python3.12/site-packages/botocore/docs/service.py",
line 13, in <module>
    from botocore.docs.bcdoc.restdoc import DocumentStructure
  File
"/root/duplicity/lib/python3.12/site-packages/botocore/docs/bcdoc/restdoc.py",
line 17, in <module>
    from botocore.compat import OrderedDict
  File "/root/duplicity/lib/python3.12/site-packages/botocore/compat.py",
line 32, in <module>
    from urllib3 import exceptions
  File "/root/duplicity/lib/python3.12/site-packages/urllib3/__init__.py",
line 7, in <module>
    from .connectionpool import HTTPConnectionPool, HTTPSConnectionPool,
connection_from_url
  File
"/root/duplicity/lib/python3.12/site-packages/urllib3/connectionpool.py",
line 11, in <module>
    from .exceptions import (
  File
"/root/duplicity/lib/python3.12/site-packages/urllib3/exceptions.py", line
2, in <module>
    from .packages.six.moves.http_client import IncompleteRead as
httplib_IncompleteRead
 ModuleNotFoundError: No module named 'urllib3.packages.six.moves'

2024-07-22 13:46:42,781 WARNING Attempt of list Nr. 3 failed.
ModuleNotFoundError: No module named 'urllib3.packages.six.moves'

even though urllib3 and six are installed and in the proper versions (I
think) - pip list -v says:

six                                             1.16.0
 /root/duplicity/lib/python3.12/site-packages pip
stevedore                                       1.20.1
 /root/duplicity/lib/python3.12/site-packages pip
stone                                           3.3.1
/root/duplicity/lib/python3.12/site-packages pip
tlslite-ng                                      0.7.6
/root/duplicity/lib/python3.12/site-packages pip
typing_extensions                               4.12.2
 /root/duplicity/lib/python3.12/site-packages pip
uritemplate                                     4.1.1
/root/duplicity/lib/python3.12/site-packages pip
urllib3                                         1.25.11
/root/duplicity/lib/python3.12/site-packages pip
wrapt                                           1.16.0
 /root/duplicity/lib/python3.12/site-packages pip

Any ideas what I might have done wrong?




On Mon, 22 Jul 2024 at 09:21, tink <[email protected]> wrote:

> And without compression and encryption the local version takes 55 seconds.
>
> On Mon, 22 Jul 2024 at 09:16, tink <[email protected]> wrote:
>
>>
>>
>> On Mon, 22 Jul 2024 at 08:42, Thomas Laubrock via Duplicity-talk <
>> [email protected]> wrote:
>>
>>> Hi Tink,
>>>
>> Hi Thomas,
>>
>>
>>> Am 21.07.24 um 22:26 schrieb tink via Duplicity-talk:
>>> > --no-compression and --no-encryption
>>>
>>> try the switches above for the local backup. Maybe the problem is CPU
>>> bound and not I/O bound.
>>>
>> I'll give it a shot, but think that w/ the recent version of duplicity
>> and concurrency
>> I should get better performance. The machine has 16 cores (HT disabled),
>> 48GB RAM
>> and while duplicity was doing its thing w/ GPG only two cores were busy
>> ...
>>
>>
>>
>>> Also the timestamps from the log should give an indicator.
>>>
>>> But for me all looks like works as designed. I know, a horrible answer.
>>>
>> Fair enough, too ;)
>>
>>
>>>
>>> Bye
>>>
>>>   Thomas
>>
>> Bis die Nächte ;)
>> Tink
>>
>
>
> --
>
> A: Because it messes up the order in which people normally read text.
> Q: Why is top-posting such a bad thing?
> A: Top-posting.
> Q: What is the most annoying thing in e-mail?
>


-- 

A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing in e-mail?

_______________________________________________
Duplicity-talk mailing list
[email protected]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk