Re: --sequential-read not sequential???

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Hi,

On 19/02/2020 12:15, Graham Cobb wrote:
> I am confused and could do with some help.
> 
> I want to check (--test) my main DAR backup archives to make sure
> they have not been corrupted. Unfortunately, the NAS that stores
> the archives can't run dar, and my desktop doesn't have enough
> diskspace to store all the slices at once. So, I have created a
> simple script to use -E to copy each slice over as it is needed,
> and delete it when the next one is requested.
> 
> As the copying takes quite a long time, I decided to use 
> --sequential-read so that the slices only get used once each. So,
> my command looks like:
> 
> dar -t "/var/tmp/$archive" \ -E 'rm "%p/%b."*".%e"' \ -E "echo
> Fetching $dir/%b.%n.%e.gpg..." \ -E "gpg -d -q -o '%p/%b.%n.%e'
> --batch  '$dir/%b.%n.%e.gpg'" \ --sequential-read

Well, this is my turn to be confuse as I don't see how either 'rm'
'echo' or 'gpg' can lead the copy of each slice from the NAS... anyway
that's not the issue here...


> 
> This seems to work. I get the following output at the end, which I 
> presume means success (a simple yes/no summary of the --test would
> be nice to add, by the way :-) ):

I let you refer to the answer just provided by Andy for that point.

> 
> -------------------------------------------- 5661770 item(s)
> treated 0 item(s) with error 0 item(s) ignored (excluded by
> filters) -------------------------------------------- Total number
> of items considered: 5661770 
> --------------------------------------------
> 
> HOWEVER, looking at the output of all the "Fetching" messages, I
> notice that after reading all 117 slices in order, dar fetched
> slices 1,8,9,10,11,13 and 117 again!
> 
> Shouldn't --sequential-read prevent that? Or am I misunderstanding
> how it works?

I would add -vm option to see more about what does...

the --sequential-read option lead dar to be suitable to read an
archive from a pipe. If for some reason (corruption, dirty file, etc.)
it need to skip back or forward it tries to do so. If that fails
because the system refused, it continues as if it had not requested to
skipped. The reason why skipping is needed should show with -vm
option. This may lead
to request an other slice.

You can do exactly what you want by mean of dar_form with dar:

dar_xform -E 'rm "%p/%b."*".%e"' \
 	  -E "echo Fetching $dir/%b.%n.%e.gpg..." \
 	  -E "gpg -d -q -o '%p/%b.%n.%e' --batch  '$dir/%b.%n.%e.gpg'" \
	  /var/tmp/$archive - | dar -t - --sequential-read

dar_xform will read sequentially the archive and build a single sliced
archive from the sliced one which will be output to its stdout and
consumed immediately by dar (nothing stored on disk). Dar will have no
choice than to read sequentially, though there is a caching layer
inside dar that can allow dar to skip backward a little, but this is
transparent to you.

If your NAS support ftp or sftp access, since release 2.6.0 you can
ask dar to read an archive remotely (for testing, restoration,
listing, etc.) using the following syntax:

dar -t [s]ftp://host/path/archive

see the man page at http://dar.linux.free.fr/doc/man/dar.html
about "remote repository syntax"

> 
> Graham
> 
> 

Regards,
Denis
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCAAdFiEEOzEprx3d76WjfYGPCDGwvQPYsYIFAl5NkRMACgkQCDGwvQPY
sYJ9ZQ/+MZt2BEJ4gs0x8i1SaJ1HEPgvCUpzoqaAlWFdowZIqgcTIKGFCPSdgcGs
lDt72Ce8BlcxcZgDykPRk+o9JQpcLQpFTRqT4c730hW39n9el0CgDlD1AjLGOGcU
Ww6ZcmGWt5G310u9wYsgEEXVtJ0btoE7mFXxJQSxjSuQHAv9J+u3tShwdOWEtZKP
7CutCqscocnzBYIWG7J+lcSaeGwJu3nWz7Qjl4GTmewXj4YwvfdMFBjO77CuWWcz
YZg6atgiBE9srX3fPevWL8Hnl9sR4q5C+a21vZydeo014pZVhWKEfbENHjYuy0nt
zEuWdSU8Cip8GAaFWeAEyPPtvfoaR4zEDydygdurl9T7SJHa4XbHq519wq/sv4Xy
50ok9b7OBJpzv/QB7BEMdFgAFhlkvw9qD596eA4S3uS4zFWGJDrSy/D26in3e+MJ
lo80lTJpIPxXKt0otw53FsAmndABa3TzKeBWCde8vD6rhaKabE3aiRKAUIl8XCCS
kSzYx9x/YERlc6gvlEy70HaAE8SPSNcqPprDwZwqtQXgymhYFIRY9AHMvi/0/76l
8LYogeD4ZnzLRcZ/wlJot0iER6iKgGpf9sMv5wzJiJGfPuxQXu9CQtVlWFUeQ0M3
PbmmedQox08/ASr7FpfpE/WeKPILFNKm6qLceok4ILgU9R2ElD0=
=RdSp
-----END PGP SIGNATURE-----