Re: Couple of questions regarding dar

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 12/05/2020 21:26, Nils Privat wrote:
> Hello,

Hello Nils,

> 
> first of all i want to thank you for your time and help.

and thank you for the tests your did:
> 
> i upgraded to v.2.6.9 and run the same script and got same results,
> so the "old" dar version was not the problem.

OK, good to know

> I also tried the script with putting in "mf -f $TAPE eof" between
> the dar creations, but that doesnt help too. Instead i got an
> "empty" file between the dar full backup and diff-backup (full
> backup start at file number=0, and diff backup at file number=2
> because i manually forced an eof in between). So it seems like
> dar/dar_split/the tape itself is writing an eof after a successful
> backup....

Strange to me. But OK, I assume the operating system adds such EOF
mark when closing the filedscriptor used to write data to tape.

> 
> Second i run the all the commands you suggested (copy back the dar 
> backup from tape to file) and got no errors - listing was fine and 
> testing too (see commands and output here:
> https://pastebin.com/raw/UUNqjWtB). It even showed the "[--- 
> REMOVED ENTRY ----]" correctly.

OK that's indicates the archive is properly written do tape, good.

> 
> Yes, for all the tests i use the sametape (no 
> ejecting/reloading/swichting the tape). I just used dar_split
> because for future backups i want to use multiple tapes. I though
> it was not a problem to use dar_split even for a single tape. For
> identifying the issue i skip the use of dar_split. See
> output/testing without using dar_split here:
> https://pastebin.com/3xZ6VTxa

the error message you get from dar "FATAL error, aborting operation:
Error while reading from pipe: Cannot allocate memory" is reported due
to a failed read() system call, and the operating system justifies the
error by "Cannot allocate memory" (!)
I'm confused, the system is provided with a memory allocated place to
copy data to... so it has not to allocate any memory to provide the
requested bytes for reading. The only possible explanation I have is
that the mount of data requested at once is too large for the
underlying driver (?)



> So without using dar_split it seems everything is working
> correctly, altought directly redirect the tape to dar (dar -l -
> --sequential-read < /dev/nst0) was not working for me, but that
> could be an blocksize specific thing. But as you can see without
> dar_split i got the correct output from dar --test and also the
> restore from diff-backup deletes files (which was both not the case
> in my testing with dar_split in my previous mail). So what to do
> next?

I will see how to adapt/enhance dar_split to let you specify more
parameters there like a block size per system read/write call. The
work done by mbuffer is perturbed by dar_split except the rate (-R
option) at which data is sent.

> 
> Maybe my explanation regarding the 4) point in the previous mail
> was a bit unclear (english is not my mother language): I understand
> that dar_split is printing "No more data available from source,
> please do something!" and its waiting for manual input of the user
> on what to do next (for e.g. switching tapes).

correct

> Pressing ctrl-c does not always exists gracefully (see
> script/output in previous mail) and pressing return/enter results
> in dar_split reading further from tape (which is correct in case of
> a tape switching).

OK, you have dar and dar_split both sending characters to terminal.
Dar_split reads from tapes and stick them toghether to its stdout, it
does not interpret its content. Upon reading an End of File, it shows
this message "... please do something!", which should maybe be changed
to something clearer... English is neither my mother tong ;^)
dar_split cannot this know whether this is the last tape or not and
asks until there is no more reader on of its standard output.
In particular, in sequential-read mode dar will read up to the EOF. So
until dar_split exit and thus expects your CTRL-C to be received.

but this CTRL-C can also be sent to dar espetially if both dar and
dar_split use the same terminal. So this makes dar interrupt the
operation and sending you this message: "Received signal: Interrupt /
Archive delayed termination engaged".

To cope with that, you could do the following using two different xterm:

term1:
	mkfifo dar_spit_to_dar
	dar_split split_input $TAPE > dar_split_to_dar

term2:
        dar -l - --sequential-read < dar_split_to_dar

eventually adding mbuffer somewhere in the middle if you need.

this way, hitting CTRL-C in term1 once all tape have been read will
avoid the terminal sending the INT signal to dar as well as dar_split.
So you will see in term2 the exact dar output without mixed messaged
from dar_split

> In my specific case, without tape switching and appended other
> backups/data on the same tape, it results in dar_split reading the
> next file on the same tape and prints messages like:
> 
> # ERR <ROOT>/1.rar : Skipping backward is not possible on a pipe #
> A problem occurred while reading this archive contents: Skipping 
> backward is not possible on a pipe # Final memory cleanup... # Some
> files are corrupted in the archive and it will not be possible to 
> restore them

this message comes from dar, which expectes to read a single sliced
archive and get in fact a new archive concatenated to the archive he
was reading. This flow of structured data is not expected by dar and
leads to this error which may leads to different error (elastic buffer
relative error below is at encryption layer), and the previous takes
place at the overall archive structure.

> 
> or
> 
> # Final memory cleanup... # Error while listing archive contents:
> elastic buffer incoherent structure
> 
> 
> Thanks
> 
> 
> 

Regards,
Denis
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCAAdFiEEOzEprx3d76WjfYGPCDGwvQPYsYIFAl69MiYACgkQCDGwvQPY
sYLrRw//dchxP4WmhEqjgGj2DmLyMj2CL3nWyqIbs0lKN7i6IUS5f6DFALm8xYq2
eCIVmgRcFqemEoo4KSzm6VrCPWG1RAgQuEb6C6isOajZ0NdvK4r5p7lbyZWkrapj
jif0XIuHI0p55qnwuob9qN7ZClTL4EidimvAMFLC2SGFvjwmazxTsyk9XK6jZbjS
PgZwWtFlPrRLBTD6E84+l4laYLOUf8cAYToM5fRzZ2PwdBQWTJv3s5JAqbFFowAh
fo/pMgxRDnd7kl7b/TYyYV0FeP11lYWQ0ocLvN3qQipDkqNUPMt1xdcutrLEA8Ap
W1TNU7/OuCsGqcnA4icdGJpZ+RaXG288auxmN1I7eFUgV0i8HsXHpbFT7Xx66j3i
6W/uAbRbjs4oO6f9BnNzGXyPlycsfiP5Udz3yu8WQEJ4QJVFOA706aGmavyED4iV
qtBpxwxR/NuTQqzbMeldqwL5Tmtd8MzsX48K1Zv9Z1fZCPHzV6z1+aQ98QUfOQbK
6eI6Ce8Hdc3L3KpQe3jHAJ4RJ+HrDmTvoncQwPGN3R8HCrOImBT50N4nzBIZ+5jI
ZqBrJ1kZVj3XgckyWb3geXd0NGyM7fudkYXLFoSKGRLCbMZ0U6K4eB1AkWHsy4/q
7BSqnhLlHDc11eUYAzwCDfGPyerzdBpFtb4FPhbFAomE5QNC63U=
=YtfU
-----END PGP SIGNATURE-----