Re: questions about --retry-on-change and --ref

On 27/04/2021 00:56, imschmeg wrote:
> On Mon, 26 Apr 2021 23:41:17 +0200
> Denis Corbin <[email protected]> wrote:
> 
>> On 26/04/2021 18:49, imschmeg wrote:
>>> On Sun, 25 Apr 2021 22:08:37 +0200
>>> Denis Corbin <[email protected]> wrote:
>>>   

[...]

>> You should not have to repair not even "verify" a backup unless dar
>> reports a error. CRC is calculated on any part of the archive, there
>> is still a theorical possibility for double corruption that precisely
>> compensate and make the CRC stay valid, but that's very improbable.
> 
> Isn't it worth running par2 repair on backups that would otherwise not
> be used at all for long periods of time (but are still on-line)?

You are right, it all depends on the context:
- quality of the media used
- trust (or lack of trust) you have in it
- risk you take manipulating your backup (physically)
- for some media the fact to read may alter time after time the stored
data...

> Wouldn't that help prevent the accumulation of errors that would
> otherwise eventually exceed par2's ability to repair?

definitively yes

> 
> Also, in my tests, par2 repair takes about the same time as dar does to
> test the backup, as both are mostly IO bound tasks with about the same
> amount to read. So it seems like a waste of time to only run par2
> repair after a dar test fails.  Unless that dar test is part of some
> other operation one is going to do anyway, and is similarly capable
> of detecting errors to par2.

Correct, storage is often the bottleneck when it comes to manipulate
large sets of data.

You can speed up things by using slices with dar, chosing their
size smaller than the amount of RAM you have. Then using the par2
integration with dar (see par2 in /etc/darrc) or your own DUC command
(see -E option), par2 verifying eventually repairing and dar testing for
the slice would require only once the amount of I/O from the storage
thanks to the system cache.

> 
>>
[...]

>> To be noted that compression as well as encryption are both very
>> sensible to corruption, most of the time long before CRC can be
>> triggered, the deciphering engine will fail and dar will propagate the
>> error, then the decompression engine, if by any chance the first
>> missed the corruption (very improbable) or if the archive was not
>> encrypted.
> 
> Oh, I didn't expect the checksum or CRC to be such that it would be
> less sensitive than the encryption or compression it is guarding.
> 
> In that case, I will create isolated catalogues, and create par2 files
> for them (I noticed that dar's par2 option does not create par2 files
> for catalogues, so I will call par2 create directly). I will run par2
> repair on a catalogue immediately before using it as a reference.  I
> have tested par2 repair on catalogues, and it is very fast because
> catalogues are so small in comparison to their associated backups.
> 
> Thanks!
> 
> 

Cheers,
Denis
>