Re: Couple of questions regarding dar

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Well,

I would first upgrade to the latest release (2.6.9 today), release
2.6.2 is more than one year old! Several bugs (in particular one fixed
in 2.6.5) could be the cause of the problem.

beside that, I'm surprised that you do not need to use "mt -f
/dev/nst0 eof" right after dropping a dar backup to file... well I
assume it would not hurt doing so. If there is no EOF seen while
reading the archive, this may explain why dar_split ends reading the
tape and asks for the next one.

I would forcibly add eof make using the magnetic tape (mt) command.

If using 2.6.9 does not change the problem then:

can you copy back the dar backup from tape to file:
mt -f /dev/nst0 rewind
cat < /dev/nst0 > full.1.dar
dar -l full
dar -t full
cat < /dev/ns0 > diff.1.dar
dar -l diff
dar -t diff

if the backup is correct once copied back to disk as showed above,
can you confirm that all backups take place on the same tape? (I could
not see any message from dar_split while both backup have been created).

If so, could you just use dar directly to tape (without dar_split)
mt -f /dev/nst0 rewind
dar -c - ... > /dev/nst0
mt -f /dev/nst0 eof
mt -f /dev/nst0 rewind
dar -l - --sequential-read < /dev/nst0
mt -f /dev/nst0 rewind
dar -t - --sequential-read < /dev/nst0

Cheers,
Denis

On 10/05/2020 00:46, Nils Privat wrote:
> 
> In Addition my testing with dar --test with tape: 
> https://pastebin.com/LTBkPX6G
> 
> Am Sa., 9. Mai 2020 um 23:53 Uhr schrieb Nils Privat 
> <[email protected] <mailto:[email protected]>>:
> 
> Hello, thanks for your answer. I am using dar v2.6.2 (from debian
> repo).
> 
> Maybe it helps when i am explain my testing in more detail:
> 
> A) First i test dar without tape, just with regular archive files. 
> From my testing everything seems to work fine, see my script /
> log: https://pastebin.com/bS02nTZz As you can see i created a full
> backup, then i renamed a folder and removed a file. After that i
> created a differential backup and 'dar --list' is correctly showing
> the renamed folder as new files and the "REMOVED ENTIES". Lastly i
> restored the full backup and it correctly restored all full-backup
> files. Then, after restoring the differential backup, the file
> '1.rar' and dir 'A' are deleted and the new dir 'Z' is restored.
> Everything like expected - works!
> 
> 
> B) Now, I used the same files and mainly the same script, but the 
> target was a tape. See script with tape:
> https://pastebin.com/KiK5YnSn (Of course I have adjusted the
> commands for the use of tape accordingly).
> 
> Some of my questions regarding the script and command outputs: 1)
> The listing from catalogue is like expected (see lines 274-301). 
> Why the listing from tape is so different (see lines 315-418)? 1a)
> Why i got "Error while listing archive contents: elastic buffer 
> incoherent structure" (line 336)? (My guess is the following:
> dar_split successfully read the content from the tape and after
> reaching EOF on tape, dar_split stops and prints "no more data
> available". If i am not changing the tape and just pressing ENTER,
> dar_split tries to read further and prints the mentioned error.
> (FYI: this error message doesnt occur without using encryption
> after pressing ENTER). If my assumption is correct, that dar_split
> tries to read further, why it does not just print the content of
> the next file (in my case the differential backup, which is the
> next file after the full backup). 1b) Why i do not see the "REMOVED
> ENTIES" when i am listing from tape but from catalogue/regular
> archive files (see line 395)? 2) Why restoring the differential
> backup from tape does not deleted files (see line 457-482)? I my
> first example A), they were deleted. 3) How do i interpret the
> testing in line 484-522 ? 4) More like a general question: No
> matter if i read, test or restore a backup from tape, if dar_split
> reaches and EOF or the end of the tape, it prints "No more data
> available from source, please do something!" and the user must
> interact. If we to not want to change or reposition the tape
> because all data is read, that is the right way to exit the 
> commands? "CTRL-C" print messages like "Archive delayed
> termination engaged. Disabling signal handler, the next time this
> signal is received the program will abort immediately." which does
> not seem the right way and pressing ENTER results in other messages
> like "elastic buffer incoherent structure" because dar_split seems
> to read further.
> 
> Thank you for your help in advance. Still hoping to use dar as my
> primary tape backup tool :)
> 
> Am Di., 5. Mai 2020 um 21:49 Uhr schrieb Denis Corbin 
> <[email protected] <mailto:[email protected]>>:
> 
> On 03/05/2020 11:45, Nils Privat wrote:
>> Hello,
> 
> Hello,
> 
> 
>> much thanks for your answer.
> 
>> During testing a few new questions have arised.
> 
>> 1) To list archives i run:
> 
>> *dar_split split_input $TAPE | dar --list - --sequential-read*
> 
>> *Output:* * * *[Data ][D][ EA  ][FSA][Compr][S]| Permission |
>> User | Group | Size |          Date                 |
>> filename*
> 
> *--------------------------------+------------+-------+-------+-------
>
> 
- --+-------------------------------+------------*
> 
> 
> *[Saved][ ][Saved][-L-][ 100%][X]  -rwxrwxr--   test test1 MioThu
> Apr 27
>> 17:47:59 2017A.rar* *..* *[Saved][ ][Saved][-L-][ 100%][X] 
>> -rwxrwxr--   test test1 MioThu Apr 27 17:47:59 2017E.rar* *No
>> more data available from source, please do something!* *Press
>> return when ready to continue or hit CTRL-C to abort* *
> 
> I guess you have to change of tape, then hit return for dar_split
> read the next part of the archive
> 
>> * *No more data available from source, please do something!*
> *Press
>> return when ready to continue or hit CTRL-C to abort*
>> *^CReceived signal: Interrupt* *Archive delayed termination
>> engaged*
> *Disabling
>> signal handler, the next time this signal is received the
>> program will abort immediately* *[Saved][ ][Saved][-L-][
>> 100%][X] -rwxrwxr--   test test1 MioThu Apr 27 17:47:59
>> 2017B.rar*
> 
> 
>> As you can see listing is working but i worry about that first
>> it says "No more data available from source, please do
>> something", then after pressing CTRL-C or enter one last entry of
>> the listing is still printing "B.rar". Is that normal? Why not
>> all items are listed together?
> 
> this is the drawback of using tapes, as tape per nature provide 
> sequential read/write operations (OK you can rewind and fast 
> forward a tape, but as you have no offset information, this is not 
> efficient to randomly read or write data whenever possible). So you
> have to provide the whole archive for dar to read its whole 
> content. If the archive has been split over several tapes you will 
> have to provide all tapes in the same order (and assume the are 
> rewinded at the position where you have started writing the
> achive).
> 
> You still have the possibility to isolate a archive (on fly or 
> afterward) to ease the listing as well as differential/incremental
> backu ps
> 
> 
> 
>> 2) Question regarding differential archives on tapes:
> 
>> I am using this command to create a full backup to tape:
> 
>> *dar --create - \* *--verbose=all \* *--compression gzip:3 \* 
>> *--alter=no-case \* *-Z "*.zip" -Z "*.rar" -Z "*.gz" -Z "*.7z"
>> -Z "*.iso" \* *-X ".DS_Store" \* *--key aes256:MyKey \* 
>> *--on-fly-isolate /root/dar_catalog_full \* *--fs-root
> ${MY_PATH} |
>> mbuffer -P 80 -m 4G -R 140M | dar_split split_output ${TAPE}*
> 
>> For differential backup i use the same command but with:
> 
>> *-A /root/dar_catalog_full \* *--on-fly-isolate 
>> /root/dar_catalog_d1 \*
> 
>> Do i have to run *'mt -f $TAPE eod'* or 'eof' between each new 
>> backup? Or is dar positioning the tape after a backup already to 
>> the end and is writing an EOF mark so i just can append a new 
>> backup?
> 
> dar and dar_split does not know about magnetic tape (mt) specific
> EOF marks (this is media specific). So yes, you should add after
> one using 'mt' after the end of an archive, espetially if you plan
> to add another backup or file after this one on the same tape.
> 
> 
>> INFO: After creating the first backup i checked the position:
>> *mt -f $TAPE status* *File number=1, block number=0,
>> partition=0.* So it seems like dar has written and EOF to file 0
>> and now i am on position/file=1, but executing *'mt -f $TAPE eod'
>> *then shows: *File number=1, block number=-1, partition=0.*
> 
> first this is dar_split that writes to the tape special device.
> And double checking dar_split code, once all bytes are written,
> dar syncs (should be useless for tape but it does not hurt) then
> just closes the file it was writing to. This should not set a mt
> EOF mark AFAIK.
> 
> 
> 
>> 3) Question regarding encryption:
> 
>> I create encrypted archives with the command above, rewind the 
>> tape, and list via:
> 
>> *dar_split split_input $TAPE | dar --list - --sequential-read 
>> --key aes256:MyKey*
> 
> for reading you don't need to use --key option (but you can) if
> you prefer to have your password visibile to all on process
> command line ;)
> 
> 
>> After pressing ENTER I got: *_Error while listing archive
> contents:
>> too large elastic buffer or elastic buffer incoherent
>> structure_* (I am not getting this message when i am pressing
>> CTRL-C after listing.)
> 
> this is either a wrong password, a wrong offset of the tape, or
> some other incoherence between the creation time and reading time.
> 
> I've just tested this using a named pipe (mkfifo) to emulate a
> tape and could create and read an ecrypted archive without
> problem. This is not likely to be a bug, which version are you
> using by the way?
> 
> 
>> If I am not using encryption/compression at all and do the same 
>> thing, i dont get the Error either by pressing ENTER or CTRL-C.
>> It just exists the listing normally back up shell. Why am i
>> getting the error?
> 
>> Much thanks for your help.
> 
> 
> [...]
> 
> Cheers, Denis
> 
> 
> _______________________________________________ Dar-support mailing
> list [email protected] 
> <mailto:[email protected]> 
> https://lists.sourceforge.net/lists/listinfo/dar-support
> 
> 
> 
> _______________________________________________ Dar-support mailing
> list [email protected] 
> https://lists.sourceforge.net/lists/listinfo/dar-support
> 
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCAAdFiEEOzEprx3d76WjfYGPCDGwvQPYsYIFAl65pmAACgkQCDGwvQPY
sYKeWw//XlpCIxSwG2AZfk4JW3yS0yGEkU8PJV7dXS0++AdQan4eLPROlEEDEH79
Z1DBICQhCx7IHv05WYC4UU9i60uzYdtcbWZy47/rFrpNil1lm0KsPSYdW9nSITq8
roAFMBZX3Xg726LfAmzxhd+j/mDlV0VT/9f5bHojDNg/PZw8mXdAmm/Brol3oH5S
BFNw/3rAdbnOKH/5uM/u2M6vSXwzmPodGCb/ML4AT2tdxjC/YbUqkDUmmdID87YS
PLtUIexz2ePOGhdTi+Zi3I4md5IAkTb2H7F4aDL+bf7CdIQXWIjt4eqM78juGn8s
BnALp/03kKScoF0UEqIuM1yKepppn5c3mBBdOuHuHx3E6Fd/sj61oZRHKVHlu/xo
YprfqhD9GjSFWbo/T2kq8o9/OnsyBSn4kWyv/cDhv/Ct54CtnDYANfgNDkDl0Vkv
CIU6181g2AgaMdwo/80VXg0y/IkP7ibA7zvt2yNl4G8Y0BJXQrbT28zMu7LD6QH0
QZV2zJyw1+TPBYZM779472Oxxt+4y4VOMRuQhAk4LYP3ygiKtlS6H/i1UVtIYZgw
6zDdMfH8f0pqRkM1MQy1BRGqkyCtfDK7wd8sE+JfJPdecknf0k4oDeV9SyaB5EM0
nutkI6qZjVdGfVJxqxxm8OhfSdFCN4xgqJ8tH0KdXPw9biHLfP4=
=fGp7
-----END PGP SIGNATURE-----