Re: Extracting with sequential read, isolated catalogue and no tape marks

Denis Corbin <[email protected]> Fri, 20 Dec 2024 21:53:35 +0100
Newsgroups	gmane.comp.sysutils.backup.dar.support
Message-ID	<[email protected]>
On 20/12/2024 18:45, [email protected] wrote:
> Hello.

Hello,

> 
> I have a DAR archive created with no tape marks and an isolated 
> catalogue for it. I would like to extract it using sequential read, 
> which would allow for on-the-fly unpacking when downloading slices 
> sequentially — without it, I would have to download the last part first, 
> which I would like to avoid. From what I understand, lack of tape marks 
> should not impede unpacking when the isolated catalogue is provided. 

This is a wrong assumption: A backup without tape marks cannot be read 
with --sequential-read option.

The reason is due to the fact this option inserts a tape layer between 
the slice layer and the compression layer, which role is to un-escape 
read data: tape mark use a specific byte sequence and to avoid saved 
data containing this same sequence to be considered as a tape mark, they 
are 'escaped' (I mean modified) to be recognized as data and not as tape 
mark at reading time. But if the archive has been created with -at 
option (thus without tape mark), data has not been modified to avoid 
collision with tape marks as there was no tape marks used in that 
archive. This is the reason of the "corruption" reported by dar.

The restoration of such archive must not use --sequential-read, the 
origin of the catalog (internal or isolated). Thus even if the 
restoration is sequential with an external catalog, I mean it reads 
sequentially from the first slice to the last slice and in each slice 
also reads in growing byte order, to access a given data, dar/libdar 
makes a call to lseek() for the next read() operation to fetch the 
expected data, lseek() is not possible on a pipe only on plain files.

By comparison, with --sequential-read option, in addition to un-escape 
read data, dar/libdar only call the read() system call and looks in the 
byte flow for tape mark to know what is coming next 
(directory/file/data/CRC/EA/FSA and so on), this leads to the 
construction of the catalogue during the whole process of the 
restoration (tape mark lead to add an "in-lined" catalog interleaved 
with the saved data).

> Indeed, the extraction works, but it seems that the files extracted from 
> the slice number 1 are corrupted.

Can you try using external catalog without --sequential-read option? 
This should solve the reported corruption and if you provide slices as 
files (not from a pipe) in the order they are requested, which should be 
from slice 1 to the last, you should be good.

> 
> In other words: when I have all the slices present on disk already,
> `dar -x archive -A isolated --sequential-read`
> results in
> `Error while restoring {file}: Bad CRC, data corruption occurred`
> for the files contained in the first slice. After that first slice, 
> extraction continues normally without further errors.

IMHO, this is not relative to slice 1, but to the nature of the data in 
your file, probably a dar archive with tape mark and without encryption 
is present in slice 1.

> 

Regards,
Denis
OpenPGP_signature.asc (application/pgp-signature, 840 B)