Re: Usage of GNU's tar format

Hi,

Our documentation efforts have fallen behind quite a bit.  Interesting
research on your part.  Will use it to fix the docs.

Since 3.8 (EOL) duplicity has been using the PAX format and not the GNU
format, so no GNU format tarballs have been made in a long time.

We probably should lock it to PAX format and that could be done in
dup_tarfile with a simple:
    tarfal.default_format = PAX_FORMAT
and it would stick since Python's tarfile iis imported there only.

Not sure what you intend to do, but any improvements would be appreciated.
Just remember, we run on Linux, BSD, and macOS, really any POSIX-compatible
system, so any change may need to be conditional on which system is in
play..

If you need help, let us know.

...Thanks,
...Ken




On Wed, Jun 25, 2025 at 12:07 PM John Scott via Duplicity-talk <
[email protected]> wrote:

> Hi,
> In several different places Duplicity claims to use the GNU tar format:
> "Standard file format: Athough archive data will be encrypted, inside it
> is in standard GNU-tar format archives."
> https://duplicity.us/features.html#xpointer(//*[@class=
> "feature-list"]/descendant::li[4])
>
> "The files used by duplicity to store backup data are tarfiles in GNU tar
> format."
> https://duplicity.us/devel/duplicity.1.html#operation-and-data-formats
>
> I was interested in seeing what I could do to change this for my purposes,
> but I can't tell where in the Duplicity source tree the choice of GNU tar
> is imposed and I'm beginning to doubt the accuracy. At
> https://gitlab.com/duplicity/duplicity/-/blob/dev/duplicity/dup_tarfile.py
> you can see that except for a shim for the sake of caching, Duplicity seems
> to use the standard 'tarfile' module that's part of Python. If Duplicity
> was relying on this module using GNU tar as the default format, it appears
> this changed in Python 3.8:
>
> "Changed in version 3.8: The default format for new archives was changed
> to PAX_FORMAT from GNU_FORMAT."
> https://docs.python.org/3/library/tarfile.html#tarfile.DEFAULT_FORMAT
>
> Could this mean that Duplicity is actually using the standard pax format
> now? That's what I would really like to make happen. It's standardized in
> POSIX/Single UNIX Specification and is very extensible.
>
> At
> https://duplicity.us/new_format.html#xpointer(%2f%2f%2a%5b%40name%3d%22nottar%22%5d%2ffollowing-sibling%3a%3aul%5b1%5d%20%2f%2fli%5b3%5d)
> there is  a list of deficiencies in the tar format that are potentially
> mitigated by pax. The pax format is deliberately extensible and robust for
> advanced use cases, so it supports arbitrarily-long pathnames, extended
> attributes, embedding application-specific metadata, and more. I don't know
> if Python's tarfile module supports these advanced features well or at all,
> but if it so happens that Duplicity is using pax now with newer versions of
> Python, I'd like to see that change stick.
>
> Please let me know where a place to start would be if I wanted to use
> ustar or pax formats instead of GNU tar, if that's not already happening.
>
> Thanks
>
> _______________________________________________
> Duplicity-talk mailing list
> [email protected]
> https://lists.nongnu.org/mailman/listinfo/duplicity-talk
>

_______________________________________________
Duplicity-talk mailing list
[email protected]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk