Re: Huge archive and last slice

Dear Patryk,




you have mentioned the tape library and cache system in relation to using 
dar. 


Would you mind to describe us more about your setup, configuration of dar , 
parameters of its command line or perhaps scripts you are using for handling
the large amount of data ?




Until recently I supposed I was the only one who tried to use dar together 
with LTO ULTRIUM drives. I have made a lot of experiements but so far I am 
not fully satisfied with the dar functionality on tapes. e.g. the sequential
mode recovery from slices, whether to use fixed slices or  dar_split, if it 
makes sense the parameters for limiting data flow etc. (dar_split -b -r) 


I am also not sure whether it is better to use the LTO HW compression or not
... . I have made a lot of acoustic tests (hearing the LTO drive in open 
computer on my desk during various operations, and it seems the answer is 
not easy (to compress or not) .... Even the large buffers done by mbuffer 
will not prevent shoe-shining of the tape .

So I am still looking for the best practice of using LTO  (drive or library)
together with dar to assure optimal (easy, fast) storage of tens of TB from 
big arrays (well based on zfs RAIDs). 


So I hope that not only myself would benefit a lot from you optimal big data
handling use case ....




Thanks a lot 





Petr Skoda

Astronomical Institute of the Czech Academy of Sciences.



















---------- Původní e-mail ----------
Od: Patryk Bełzak <[email protected]>
Komu: need help compiling or using dar ? This mailing list is for you. <dar-
[email protected]>
Datum: 13. 12. 2023 9:29:51
Předmět: Re: [Dar-support] Huge archive and last slice
"I guess it's also worth to mention `dar_manager`. 
I'm also dealing with PB of scentific data in Hierarchical Storage 
Management solution, and I'm considering scenario where I'll build a 
database on some nfs (so it could be accessed by all systems I need, and 
getting some sort of high availability). 
By doing so, searching for files and extracting them wouldn't require to 
recall large amounts of data from tape library to cache filesystem. 

"