Huge archive and last slice

Dear Denis,

thanks for your software.

My scenario: I must archive scientific data and eventually retrieving a small portion of the data. The data are huge, several tens of TBs so the slices are very useful because we can divide the global archive in pieces, then with the catalog we can identify the exact slice(s) in case of need. Data (archives/slices) can be deposited in a remote site, so optimizing the transfer (retrieving) of the data is a priority.

At first I thought that the catalog and the slices of interest were enough. Then I realize from the dar messages, and confirmed by a msg on the ml that also the last slice is required.

The point is: if the last slice is 500GB+ I need to keep it available or download it every time a file/directory must be recovered.


I was wondering if it is possible to "extract" from the last slice only the relevant information needed for the recovery. I can understand that is it convenient to keep them in the last slice because usually it is the smallest one, but in case of huge archives its size can be relevant.


PS: I must thanks John Goerzen for his support in setting up dar w/threads during my tests.


Bests,

R.

--

Raoul Bonnal
IFOM | Research Computing and Data Science Manager
Ph.: +39 02 57430 3016 | raoul.bonnal-G+fEVLdQ/[email protected]<mailto:raoul.bonnal-G+fEVLdQ/[email protected]>
www.ifom.eu<https://www.ifom.eu> | Linkedin<https://www.linkedin.com/school/ifom/> | Instagram<https://www.instagram.com/ifom_milan/?hl=it> | Facebook<https://www.facebook.com/ifom.eu> | Twitter<https://twitter.com/ifomresearch> | YouTube<https://www.youtube.com/channel/UCOVlG5DhrDWMR1N5NYBB4RQ>
IFOM ETS - The AIRC Institute of Molecular Oncology | Via Adamello 16, 20139 Milan, Italy

[cid:logoIfom_2d2161e0-3259-466c-8931-26f8c2a5979f.png]

________________________________

Confidentiality notice. This message, together with its annexes, contains information to be deemed strictly confidential and is destined only to the addressee(s) identified above who only may use, copy and, under his/their responsibility, further disseminate it. if anyone received this message by mistake or reads it without entitlement is forewarned that keeping, copying, disseminating or distributing this message to persons other than the addressee(s) is strictly forbidden and is asked to transmit it immediately to the sender and to erase the original message received.
Thank you
Save a tree - Do not print this email unless absolutely necessary