Re: Failure testing duplicity 3.0.0

I cannot reproduce using the current pytest.

I cannot reproduce using the older pytest.

I have to conclude it's something funky in the install of the older version
that makes this happen.

Could you please uninstall your old pytest and try "$ pip3 install
pytest==7.4.4" and try the tests again.  That should work.

...Ken

============================= test session starts
==============================
platform darwin -- Python 3.10.14, pytest-7.4.4, pluggy-1.5.0 --
/Users/ken/workspace/duplicity-main/venv310/bin/python
cachedir: .pytest_cache
rootdir: /Users/ken/workspace/duplicity-main
configfile: pyproject.toml
plugins: cov-5.0.0
collecting ... collected 8 items
run-last-failure: no previously failed tests, not deselecting items.

testing/unit/test_collections.py::CollectionTest::test_backup_chains
testing/unit/test_collections.py::CollectionTest::test_collections_status
testing/unit/test_collections.py::CollectionTest::test_get_extraneous
testing/unit/test_collections.py::CollectionTest::test_get_olderthan
testing/unit/test_collections.py::CollectionTest::test_sig_chain
testing/unit/test_collections.py::CollectionTest::test_sig_chains
testing/unit/test_collections.py::CollectionTest::test_sig_chains2
testing/unit/test_collections.py::CollectionTest::test_sigchain_fileobj

=============================== warnings summary
===============================
testing/unit/test_collections.py::CollectionTest::test_backup_chains

/Users/ken/workspace/duplicity-main/venv310/lib/python3.10/site-packages/urllib3/util/ssl_.py:273:
DeprecationWarning: ssl.PROTOCOL_TLS is deprecated
    context = SSLContext(ssl_version or PROTOCOL_TLS)

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
========================= 8 passed, 1 warning in 1.90s
=========================



On Mon, Sep 2, 2024 at 1:55 PM Scott Hannahs <[email protected]> wrote:

> Ok, poked around.   It does appear that curtimestr is the default “none”
> and throwing this assertion.   So I will move on to 3.0.2 as suggested.
>
> The errors below are from building and running the tests on Duplicity
> 3.0.2.
>
> However, it was suggested (strongly) that I update pytest.  For some
> reason my building of the current 8.3.2 version of pytest does not generate
> the executable pytest.py in the bin directory.  But using my current
> version of pytest, I get several failures in the collections section of the
> tests.  There is a test directory full of about 80 duplicity files all full
> backup of about 7 volumes and a manifest.
>
> testing/unit/test_collections.py::CollectionTest::test_sig_chains2 FAILED
> [ 90%]
> testing/unit/test_collections.py::CollectionTest::test_collections_status
> FAILED [ 90%]
> testing/unit/test_collections.py::CollectionTest::test_get_extraneous
> FAILED [ 91%]
> testing/unit/test_collections.py::CollectionTest::test_sigchain_fileobj
> FAILED [ 91%]
> testing/unit/test_collections.py::CollectionTest::test_sig_chain FAILED  [
> 91%]
> testing/unit/test_collections.py::CollectionTest::test_get_olderthan
> FAILED [ 91%]
> testing/unit/test_collections.py::CollectionTest::test_sig_chains FAILED [
> 91%]
> testing/unit/test_collections.py::CollectionTest::test_backup_chains
> FAILED [ 92%]
>
> For some reason the collection status is returning null dates and no
> lengths.
> =================================== FAILURES
> ===================================
> _______________________ CollectionTest.test_sig_chains2
> ________________________
>
> self = <testing.unit.test_collections.CollectionTest
> testMethod=test_sig_chains2>
>
>     def test_sig_chains2(self):
>         """Test making signature chains from filename list on backend"""
>         cs = dup_collections.CollectionsStatus(self.archive_dir_backend,
> config.archive_dir_path, "full")
>         chains, orphaned_paths = cs.get_signature_chains(local=None)
> >       self.sig_chains_helper(chains, orphaned_paths)
>
> chains     = []
> cs         = <duplicity.dup_collections.CollectionsStatus object at
> 0x10c932e30>
> orphaned_paths = []
> self       = <testing.unit.test_collections.CollectionTest
> testMethod=test_sig_chains2>
>
>
> /opt/sw/src/fink.build/duplicity-3.0.2-1/duplicity-rel.3.0.2/testing/unit/test_collections.py:153:
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _ _ _
>
> self = <testing.unit.test_collections.CollectionTest
> testMethod=test_sig_chains2>
> chains = [], orphaned_paths = []
>
>     def sig_chains_helper(self, chains, orphaned_paths):
>         """Test chains and orphaned_paths values for two above tests"""
>         if orphaned_paths:
>             for op in orphaned_paths:
>                 print(op)
>             assert 0
> >       assert len(chains) == 1, chains
> E       AssertionError: []
> E       assert 0 == 1
> E        +  where 0 = len([])
>
> chains     = []
> orphaned_paths = []
> self       = <testing.unit.test_collections.CollectionTest
> testMethod=test_sig_chains2>
>
> /opt/sw/src/fink.build/duplicity-3.0.2-1/duplicity-rel.3.0.2/testing/unit/test_collections.py:161:
> AssertionError
> ____________________ CollectionTest.test_collections_status
> ____________________
>
>
> -Scott
>
>
> On Sep 2, 2024, at 1:52 PM, Kenneth Loafman <[email protected]> wrote:
>
> If this were C I would say it's an uninitialized variable. It acted like
> it.
>
> I could not reproduce it on Linux.  Seems to be only on macOS.
>
> It was just a weird error.  The patch is for 3.0.2, but should work on
> 3.0.0.  I'd upgrade.  There were fixes made.
>
> ...Ken
>
>
> On Mon, Sep 2, 2024 at 12:32 PM Scott Hannahs <[email protected]> wrote:
>
>> Ken,
>>
>> Not running under Docker.  Running native on MacOS 14.6.1 (Sonoma)
>>
>> The timing error was a wild guess based on "assert dup_time.curtimestr”
>> line.  It is searching for a file name in the Manifest as far as I can tel
>> in this test.  A one second delay should be sufficient to update the
>> filename/timestamp.
>>
>> My test script is:
>>         #!/bin/sh -ev
>>         ulimit -n 8192
>>         %p/bin/pytest-3.10 --capture=fd
>>
>> (The %p expands to /opt/sw which is where my package manager sticks
>> everything instead of spewing around the file system like home-brew)
>>
>> It is odd that it works most of the time and then occasionally can not
>> find an incremental file in the manifest.  Might be my specific setup but I
>> can’t think of anything odd that I have installed that would interfere with
>> duplicity.
>>
>> I am using pytest version 7.4.4 and have had issues trying to update to
>> pytest 8.3.2.  If this is just a weird error, I will go work on updating
>> pytest and duplicity to the latest and greatest.
>>
>> -Scott
>>
>>
>> On Sep 2, 2024, at 12:28, Kenneth Loafman <[email protected]> wrote:
>>
>> Scott,
>>
>> Exactly how do you run your tests?  What command line?
>>
>> I do know that this is not a timing issue.  test_file_naming does not
>> actually create files, just filenames.  The tests that do backup, etc..
>> have a 1 second delay in between, so no timing issue.
>>
>> Are you perhaps running this under Docker?  We've had timing issues
>> there, but going to the 1 second delay solved that problem years ago.
>>
>> ...Ken
>>
>>
>>
>> On Mon, Sep 2, 2024 at 7:23 AM Scott Hannahs via Duplicity-talk <
>> [email protected]> wrote:
>>
>>> Ede,
>>> It might be.  But it is intermittent!  The last 3 tests built without
>>> errors.
>>>
>>> What is that test that does an assertion "assert dup_time.curtimestr”
>>> It is trying to do a get of a gzipped incremental file name from the
>>> manifest, but with no volume number?
>>>
>>> Is the machine too fast during testing that it is creating files with
>>> duplicate date/time stamps?
>>>
>>> -Scott
>>>
>>>
>>> > On Sep 2, 2024, at 06:36, edgar.soldin--- via Duplicity-talk <
>>> [email protected]> wrote:
>>> >
>>> > hey Scott,
>>> >
>>> > looks like the brew build runs fine
>>> https://formulae.brew.sh/formula/duplicity . something with your
>>> specific setup?
>>> >
>>> > can you give more details os version etc. to the box you are building
>>> on? unfortunately i can't run latest macOS versions virtually so far, so
>>> it' difficult to try to reproduce.
>>> >
>>> > sunny regards ..ede
>>> >
>>> > On 02.09.2024 03:46, Scott Hannahs via Duplicity-talk wrote:
>>> >> First, I get a lot of errors trying to build 3.0.2, so I will get to
>>> them next.
>>> >>
>>> >> However I get this failure with 3.0.0.  It seems that there is an
>>> issue with the time of the files?  Are they being built too quickly and the
>>> timer does not have time to increment?  This doesn’t happen on every build.
>>> >>
>>> >>
>>> >> -Scott
>>> >>
>>> >> testing/unit/test_file_naming.py .F.......
>>>    [ 92%]
>>> >> ….
>>> >>
>>> >> =================================== FAILURES
>>> ===================================
>>> >> ____________________________ FileNaming.test_suffix
>>> ____________________________
>>> >>
>>> >> self = <testing.unit.test_file_naming.FileNaming
>>> testMethod=test_suffix>
>>> >>
>>> >>     def test_suffix(self):
>>> >>         """Test suffix (encrypt/compressed) encoding and generation"""
>>> >>         file_naming.prepare_regex(force=True)
>>> >>>       filename = file_naming.get("inc", manifest=1, gzipped=1)
>>> >>
>>> >> self       = <testing.unit.test_file_naming.FileNaming
>>> testMethod=test_suffix>
>>> >>
>>> >>
>>> /opt/sw/src/fink.build/duplicity-3.0.0-1/duplicity-rel.3.0.0/testing/unit/test_file_naming.py:102:
>>> >> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
>>> _ _ _ _ _
>>> >>
>>> >> type = 'inc', volume_number = None, manifest = 1, encrypted = False,
>>> gzipped = 1
>>> >> partial = False
>>> >>
>>> >>     def get(
>>> >>         type,
>>> >>         volume_number=None,
>>> >>         manifest=False,  # pylint: disable=redefined-builtin
>>> >>         encrypted=False,
>>> >>         gzipped=False,
>>> >>         partial=False,
>>> >>     ):
>>> >>         """
>>> >>         Return duplicity filename of specified type
>>> >>
>>> >>         type can be "full", "inc", "full-sig", "new-sig",
>>> "full-stat", "inc-stat". volume_number
>>> >>         can be given with the full and inc types.  If manifest is
>>> true the
>>> >>         filename is of a full or inc manifest file.
>>> >>         """
>>> >>>       assert dup_time.curtimestr
>>> >> E       AssertionError
>>> >>
>>> >> encrypted  = False
>>> >> gzipped    = 1
>>> >> manifest   = 1
>>> >> partial    = False
>>> >> type       = 'inc'
>>> >> volume_number = None
>>> >>
>>> >>
>>> /opt/sw/src/fink.build/duplicity-3.0.0-1/duplicity-rel.3.0.0/duplicity/file_naming.py:253:
>>> AssertionError
>>> >>
>>> >>
>>> >> _______________________________________________
>>> >> Duplicity-talk mailing list
>>> >> [email protected]
>>> >> https://lists.nongnu.org/mailman/listinfo/duplicity-talk
>>> >
>>> > _______________________________________________
>>> > Duplicity-talk mailing list
>>> > [email protected]
>>> > https://lists.nongnu.org/mailman/listinfo/duplicity-talk
>>>
>>>
>>> _______________________________________________
>>> Duplicity-talk mailing list
>>> [email protected]
>>> https://lists.nongnu.org/mailman/listinfo/duplicity-talk
>>>
>>
>>
>

_______________________________________________
Duplicity-talk mailing list
[email protected]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk