Re: Huge archive and last slice

Hi Denis and John,
I read again the docs and your blog, John.
I tried multiple times and dar always requires the last slice.
During the archive creation I specified the on-the-fly generation of the catalog; later I created an isolate catalog from the original archive

Here https://github.com/rjpbonnal/dar-tests/tree/main I tried to document everything.
If there is some mistake on my side I can not spot it.

R.




--

Raoul Bonnal
IFOM | Research Computing and Data Science Manager
Ph.: +39 02 57430 3016 | raoul.bonnal-G+fEVLdQ/[email protected]<mailto:raoul.bonnal-G+fEVLdQ/[email protected]>
www.ifom.eu<https://www.ifom.eu> | Linkedin<https://www.linkedin.com/school/ifom/> | Instagram<https://www.instagram.com/ifom_milan/?hl=it> | Facebook<https://www.facebook.com/ifom.eu> | Twitter<https://twitter.com/ifomresearch> | YouTube<https://www.youtube.com/channel/UCOVlG5DhrDWMR1N5NYBB4RQ>
IFOM ETS - The AIRC Institute of Molecular Oncology | Via Adamello 16, 20139 Milan, Italy

[cid:logoIfom_2d2161e0-3259-466c-8931-26f8c2a5979f.png]

________________________________

Confidentiality notice. This message, together with its annexes, contains information to be deemed strictly confidential and is destined only to the addressee(s) identified above who only may use, copy and, under his/their responsibility, further disseminate it. if anyone received this message by mistake or reads it without entitlement is forewarned that keeping, copying, disseminating or distributing this message to persons other than the addressee(s) is strictly forbidden and is asked to transmit it immediately to the sender and to erase the original message received.
Thank you
Save a tree - Do not print this email unless absolutely necessary

________________________________
From: John Goerzen <[email protected]>
Sent: Tuesday, December 12, 2023 8:22:17 PM
To: need help compiling or using dar ? This mailing list is for you.
Cc: Denis Corbin
Subject: Re: [Dar-support] Huge archive and last slice

On Tue, Dec 12 2023, Denis Corbin wrote:

>
>> I was wondering if it is possible to "extract" from the last slice only the
>> relevant information needed for the recovery. I can understand that is it
>> convenient to keep them in the last slice because usually it is the smallest
>> one, but in case of huge archives its size can be relevant.
>
> Yes, the is precisely the isolated catalog.

To amplify this point a bit:

Every dar archive contains a "catalog", which is added at the end of the
last slice, and is the directory information (including slice number) of
every file within the archive and other assorted metadata.

You can also have an "isolated catalog", in which only the catalog
portion is saved to a separate file.  This is exactly what you want.

There are two common ways to create this catalog:

1) At the same time you create the archive itself, with the
--on-fly-isolate option

2) From an existing archive, using the --isolate option.

One use for this is to make incremental backups without needing to have
the previous backup available.  You can simply reference its isolated
catalog, and then dar will figure out what to do.

I have some examples of this on my blog here:

https://changelog.complete.org/archives/10527-using-dar-for-data-archiving

(ignore the bits about dar_manager as they are irrelevant to what you're
asking)

- John



>
>> PS: I must thanks John Goerzen for his support in setting up dar w/threads
>> during my tests.
>> Bests,
>> R.
>>
> Cheers,
> Denis
>
> [[End of PGP Signed Part]]