Re: --sequential-read not sequential???

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 24/02/2020 10:49, Graham Cobb wrote:
>> On 20/02/2020 17:16, Graham Cobb wrote:
>>>>> the --sequential-read option lead dar to be suitable to
>>>>> read an archive from a pipe. If for some reason
>>>>> (corruption, dirty file, etc.) it need to skip back or
>>>>> forward it tries to do so. If that fails because the system
>>>>> refused, it continues as if it had not requested to
>>>>> skipped. The reason why skipping is needed should show with
>>>>> -vm option. This may lead to request an other slice.
>> 
>>> I tried a run with -vm. It showed me many messages at the 
>>> beginning, after fetching the first slice, and finishing with:
>> 
>>> All layers have been created successfully The catalogue will
>>> be filled while sequentially reading the archive, preparing the
>>> data structure...
>> 
>>> After that, it loaded the processed slices (2-9 in this case)
>>> and then loaded slices 1 and 2 again. But with no messages
>>> displayed about why it did that.
>> 
>> this is quite weird! I can't find the reason why dar would skip 
>> backward to slices 1 and 2... I could not reproduce that with a
>> simple test... it may be dependent on your data (which would
>> explain that -vm did not show anything).
> 
> Thanks for your help. I have tried two things:
> 
> 1) Using dar_xform and an actual pipe to feed "dar -t". That caused
> dar to emit a hard error because it cannot go backwards on a pipe.
> So the "dar -t" failed. It didn't emit any message explaining why
> it wanted to go backwards.

strange, which version of dar are you using by the way?

Just double check: have you used
- --sequential-read option?

I Must find a way (and time) to reproduce it...

> 
> 2) I tried using "-va -p". The "-p" didn't do anything (does it
> only apply for writing, not reading?). The -va wrote a lot of logs
> (over 600MB - about 20% of the size of the archive I was using!).
> The logs still did not show any message about why it wanted to go
> backwards after reading the last slice. Here are the messages
> around the request to load the first archive again (The
> "Fetching..." line is written by my script when it fetches a slice
> from the NAS).
> 
> . . . OK  <ROOT>/lib32/libresolv.so.2 OK  <ROOT>/lib32/librt.so.1 
> OK  <ROOT>/lib32/libthread_db.so.1 OK  <ROOT>/lib32/libutil.so.1 OK
> <ROOT>/nas OK  <ROOT>/backup3 OK  <ROOT>/core OK
> <ROOT>/vmlinuz.old OK  <ROOT>/initrd.img.old OK  <ROOT>/vmlinuz OK
> <ROOT>/initrd.img OK  <ROOT>/GRC-thermald-temp-sensor OK
> <ROOT>/var OK  <ROOT>/var/lib Fetching
> /nas/backup/DAR-archive/black/DARsystemDiff05.1.dar.gpg... OK
> <ROOT>/var/lib/btrfs  FSA(12) OK 
> <ROOT>/var/lib/btrfs/scrub.progress.d065fa09-ebaa-4c4a-b090-bf6aa07028
a3
>
> 
OK  <ROOT>/var/lib/samba  FSA(12)
> OK  <ROOT>/var/lib/samba/private OK
> <ROOT>/var/lib/samba/private/msg.sock  FSA(12) OK
> <ROOT>/var/lib/samba/private/msg.sock/3238 OK
> <ROOT>/var/lib/samba/private/msg.sock/3331 OK  <ROOT>/var/indexes 
> OK  <ROOT>/var/indexes/cobb  FSA(12) OK
> <ROOT>/var/indexes/cobb/.spam.high-confidence  FSA(12) OK
> <ROOT>/var/indexes/cobb/.spam.high-confidence/dovecot.index.log.2 
> OK  <ROOT>/usr OK  <ROOT>/usr/local  FSA(12) OK
> <ROOT>/usr/local/sbin  FSA(12) OK
> <ROOT>/usr/local/sbin/btrfs-balance-slowly OK  <ROOT>/usr/src OK
> <ROOT>/usr/src/linux-headers-5.3.9  FSA(12) OK
> <ROOT>/usr/src/linux-headers-5.3.9/arch  FSA(12) OK
> <ROOT>/usr/src/linux-headers-5.3.9/arch/x86  FSA(12) OK
> <ROOT>/usr/src/linux-headers-5.3.9/arch/x86/include . . .
> 
> The only thing I can see is that all the files mentioned after
> going backwards **might** have been updated during the backup which
> created the slices. Many of them are things like logs and mail, and
> this backup might have been taken while I was building and testing
> some kernels so maybe even the linux-headers.

when a file changes at the time it is saved, it is resaved "in place"
unless the uderlying media does not allow skipping back in which case
it is saved a second time right after (see --retry-on-change option to
tune the amount of space you are ready to waste by this mechanism,
which is
none by default).

Thus the reason why these files are looked after again is not likely the
fact it has changed at backup time.

if you still have the whole dar ouput, can you provide the output of
this command:

grep "/var/lib/samba/private" <dar output>

> 
> But even if the files were modified during the backup, going back
> to an earlier slice can't help anything because the slice can't be
> different from when it was read earlier!

it would I agree, but to my point of view as esplained above this is not
likely the reason why those files are looked after a second time

> 
> Graham
> 

Cheers,
Denis
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCAAdFiEEOzEprx3d76WjfYGPCDGwvQPYsYIFAl5UMrEACgkQCDGwvQPY
sYLcdhAAiqXOnMcoqrW9kA/hKhcEyhyyJL2PAYUxCKNJ+F0P4IfloRaEYmV035wc
I1bmBx/exprQhAFD1vSdo/8ufK/9yynpkOvINUgAnpfFNZzTbU5HByYDsw2VGbd9
wKfSzkCCkqqmQB/2J/sUlleKRVyNTKAjpTcE38xcF9zSUSTA3g1VXLvQOa8hpFVz
V8HA26OuXBSn2+LgVAQk9veVFVfng0BUEpNfb+N1ASUhWXBuitDuQ79KsHOY7nlD
vGluUaf5rssv5/lsq2h2wPr2O9xdzzYRHiXy79XOhDkHZuQj6K2t+5FWmGBk3GQF
BkmHQwevRyZVpa4XagzVpqwg8toOZkNdqP0wrEoMKLg089jJWd3zU157NF2EZzu1
kVB7CeiJFSKS2uly4NUUhPprQxkzPIF8h8fEsP+51O0MrACKSqppwkf3AdIFzHME
haqKpgGAv7cEE8VzoYTsxehZfn+G8QJ2Gq80BF8a1r6mcXjlvWBgOjg3rJEqQ+1B
wbYo2I+embAwCX/ZnKQd1445ck7a4gSdPJMgYd50U7XkrtOTQtKYMrmYMKKR8bAU
vhsfPDP9ezEK8zy4rbEhI2Egbhn1hIZqH7juoN7O82fc15aI2mbtpbYkv64V/+NX
wm2b3VKbE4f78/L6W9ZdDUwj1BKcaWmg40BHa3sHcf+mHgQU6s4=
=+VAQ
-----END PGP SIGNATURE-----