Re: Compression method recommendations?

On 09/09/2021 19:04, J. Roeleveld via Dar-support wrote:
> On Thursday, September 9, 2021 6:07:28 PM CEST you wrote:
>> On Thu, Sep 09 2021, J. Roeleveld wrote:
>>> John,
>>>
>>> Thank you for this.
>>> So, for dar, the commandline option would be " -zzstd:8 " ?
>>>
>>> I don't see the " -T ", as that's for encryption?
>>> I am talking about "dar" here.
>>
>> Right, I should have been more clear.  The -T is a zstd option,
>> not a dar one.  Yes, -zzstd:8 would be a pretty good ratio with
>> pretty good performance.  You may experiment a bit; you could try
>> 3, 8, and 14 or so and see what you like about the performance and
>> size of each with your particular data set.  You might also try
>> piping dar output through zstd -T0 to see what it does, but
>> post-compressing a dar archive would cause you to lose many of the
>> benefits of dar.
>>
>> I don't immediately see a multithreaded compression option in the
>> dar manpage.  However, --multi-thread implies that it MAY do this,
>> but it says it's not been well tested and isn't encouraged for
>> use.
>>
>> - John
> 
> John,
> 
> Multi-threaded sounds nice, but not really necessary for my use-case.
> I actually prefer this part being limited to a single thread, it leaves the 
> rest to do the work the system is supposed to be doing.
> 
> I also read some comments that restoring the backup might need to be done 
> using a similar setting, in which case I prefer to keep it to the "default".
> 
> @Denis, what's the current status of the multi-threading? Has it seen sufficient 
> use yet to mark it as "reliable"? Or do you have specific test-scenarios in 
> mind you would like me to try on a test environment over an extended period?

Hi Joost,

multi-threading support (same underlying code for both crypto and
compression) is reliable to my point of view, according to the testing
it passed. I use it for my own backup without issue since early 2.7.0
pre-releases (I will never emphasis enough to always test your backups
once created, but that's not specific to multi-threading nor to dar:
this is just common sense).

Another point: the options you provide (like compression level,
algorithm, encryption algorithm, and many others) have not to be given
at restoration time, dar will figure out from the header/trailer
embedded in the archive what is needed to read the backup.

So feel free to tweak the compression level as John mentioned,
especially if you don't want to burn too much CPU cycles: zstd
compression levels ranges from 1 to 22 and defaults to 9 for dar, in
coherence with other compression algorithm. Though 9 may be too high
depending on the data to compress (the compression ratio is not much
better than with lower compression level while it may imply an important
amount of added CPU cycles). Well this is strongly dependent on the
nature of the data to compress, so, there is not one rule that fits all
needs...

Only encryption password/passphrase for symmetrical encryption (of
course) and crypto block (see -#, -%, '-*' options) have to be passed as
they were at backup time. But I guess nobody ever changed the later from
the default value.

> 
> --
> Joost
> 
> 
> 


Cheers,
Denis