Re: Couple of questions regarding dar

Hello,
thanks for your answer.
I am using dar v2.6.2 (from debian repo).

Maybe it helps when i am explain my testing in more detail:

A) First i test dar without tape, just with regular archive files.
From my testing everything seems to work fine, see my script / log:
https://pastebin.com/bS02nTZz
As you can see i created a full backup, then i renamed a folder and removed
a file.
After that i created a differential backup and 'dar --list' is correctly
showing the
renamed folder as new files and the "REMOVED ENTIES".
Lastly i restored the full backup and it correctly restored all full-backup
files.
Then, after restoring the differential backup, the file '1.rar' and dir 'A'
are deleted
and the new dir 'Z' is restored. Everything like expected - works!


B) Now, I used the same files and mainly the same script, but the target
was a tape.
See script with tape: https://pastebin.com/KiK5YnSn
(Of course I have adjusted the commands for the use of tape accordingly).

Some of my questions regarding the script and command outputs:
1) The listing from catalogue is like expected (see lines 274-301). Why the
listing from tape is so different (see lines 315-418)?
1a) Why i got "Error while listing archive contents: elastic buffer
incoherent structure" (line 336)?
(My guess is the following: dar_split successfully read the content from
the tape and after reaching EOF on tape, dar_split stops and prints "no
more data available".
If i am not changing the tape and just pressing ENTER, dar_split tries to
read further and prints the mentioned error. (FYI: this error message
doesnt occur without using encryption after pressing ENTER).
If my assumption is correct, that dar_split tries to read further, why it
does not just print the content of the next file (in my case the
differential backup, which is the next file after the full backup).
1b) Why i do not see the "REMOVED ENTIES" when i am listing from tape but
from catalogue/regular archive files (see line 395)?
2) Why restoring the differential backup from tape does not deleted files
(see line 457-482)? I my first example A), they were deleted.
3) How do i interpret the testing in line 484-522 ?
4) More like a general question: No matter if i read, test or restore a
backup from tape, if dar_split reaches and EOF or the end of the tape, it
prints "No more data available from source, please do something!" and
the user must interact. If we to not want to change or reposition the tape
because all data is read, that is the right way to exit the commands?
"CTRL-C" print messages like "Archive delayed termination engaged.
Disabling signal handler, the next time this signal is received the program
will abort immediately."
which does not seem the right way and pressing ENTER results in other
messages like "elastic buffer incoherent structure" because dar_split seems
to read further.

Thank you for your help in advance.
Still hoping to use dar as my primary tape backup tool :)

Am Di., 5. Mai 2020 um 21:49 Uhr schrieb Denis Corbin <[email protected]>:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> On 03/05/2020 11:45, Nils Privat wrote:
> > Hello,
>
> Hello,
>
> >
> > much thanks for your answer.
> >
> > During testing a few new questions have arised.
> >
> > 1) To list archives i run:
> >
> > *dar_split split_input $TAPE | dar --list - --sequential-read*
> >
> > *Output:* * * *[Data ][D][ EA  ][FSA][Compr][S]| Permission | User
> > | Group | Size |          Date                 |    filename*
> > *--------------------------------+------------+-------+-------+-------
> - --+-------------------------------+------------*
> >
> >
> *[Saved][ ][Saved][-L-][ 100%][X]  -rwxrwxr--   test test1 MioThu Apr 27
> > 17:47:59 2017A.rar* *..* *[Saved][ ][Saved][-L-][ 100%][X]
> > -rwxrwxr--   test test1 MioThu Apr 27 17:47:59 2017E.rar* *No more
> > data available from source, please do something!* *Press return
> > when ready to continue or hit CTRL-C to abort* *
>
> I guess you have to change of tape, then hit return for dar_split read
> the next part of the archive
>
> > * *No more data available from source, please do something!* *Press
> > return when ready to continue or hit CTRL-C to abort* *^CReceived
> > signal: Interrupt* *Archive delayed termination engaged* *Disabling
> > signal handler, the next time this signal is received the program
> > will abort immediately* *[Saved][ ][Saved][-L-][ 100%][X]
> > -rwxrwxr--   test test1 MioThu Apr 27 17:47:59 2017B.rar*
> >
> >
> > As you can see listing is working but i worry about that first it
> > says "No more data available from source, please do something",
> > then after pressing CTRL-C or enter one last entry of the listing
> > is still printing "B.rar". Is that normal? Why not all items are
> > listed together?
>
> this is the drawback of using tapes, as tape per nature provide
> sequential read/write operations (OK you can rewind and fast forward a
> tape, but as you have no offset information, this is not efficient to
> randomly read or write data whenever possible).
> So you have to provide the whole archive for dar to read its whole
> content. If the archive has been split over several tapes you will
> have to provide all tapes in the same order (and assume the are
> rewinded at
> the position where you have started writing the achive).
>
> You still have the possibility to isolate a archive (on fly or
> afterward) to ease the listing as well as differential/incremental backu
> ps
>
> >
> >
> > 2) Question regarding differential archives on tapes:
> >
> > I am using this command to create a full backup to tape:
> >
> > *dar --create - \* *--verbose=all \* *--compression gzip:3 \*
> > *--alter=no-case \* *-Z "*.zip" -Z "*.rar" -Z "*.gz" -Z "*.7z" -Z
> > "*.iso" \* *-X ".DS_Store" \* *--key aes256:MyKey \*
> > *--on-fly-isolate /root/dar_catalog_full \* *--fs-root ${MY_PATH} |
> > mbuffer -P 80 -m 4G -R 140M | dar_split split_output ${TAPE}*
> >
> > For differential backup i use the same command but with:
> >
> > *-A /root/dar_catalog_full \* *--on-fly-isolate
> > /root/dar_catalog_d1 \*
> >
> > Do i have to run *'mt -f $TAPE eod'* or 'eof' between each new
> > backup? Or is dar positioning the tape after a backup already to
> > the end and is writing an EOF mark so i just can append a new
> > backup?
>
> dar and dar_split does not know about magnetic tape (mt) specific EOF
> marks (this is media specific). So yes, you should add after one using
> 'mt' after the end of an archive, espetially if you plan to add
> another backup or file after this one on the same tape.
>
> >
> > INFO: After creating the first backup i checked the position: *mt
> > -f $TAPE status* *File number=1, block number=0, partition=0.* So
> > it seems like dar has written and EOF to file 0 and now i am on
> > position/file=1, but executing *'mt -f $TAPE eod' *then shows:
> > *File number=1, block number=-1, partition=0.*
>
> first this is dar_split that writes to the tape special device. And
> double checking dar_split code, once all bytes are written, dar syncs
> (should be useless for tape but it does not hurt) then just closes the
> file it was writing to. This should not set a mt EOF mark AFAIK.
>
> >
> >
> > 3) Question regarding encryption:
> >
> > I create encrypted archives with the command above, rewind the
> > tape, and list via:
> >
> > *dar_split split_input $TAPE | dar --list - --sequential-read
> > --key aes256:MyKey*
>
> for reading you don't need to use --key option (but you can) if you
> prefer to have your password visibile to all on process command line ;)
>
> >
> > After pressing ENTER I got: *_Error while listing archive contents:
> > too large elastic buffer or elastic buffer incoherent structure_*
> > (I am not getting this message when i am pressing CTRL-C after
> > listing.)
>
> this is either a wrong password, a wrong offset of the tape, or some
> other incoherence between the creation time and reading time.
>
> I've just tested this using a named pipe (mkfifo) to emulate a tape
> and could create and read an ecrypted archive without problem. This is
> not likely to be a bug, which version are you using by the way?
>
> >
> > If I am not using encryption/compression at all and do the same
> > thing, i dont get the Error either by pressing ENTER or CTRL-C. It
> > just exists the listing normally back up shell. Why am i getting
> > the error?
> >
> > Much thanks for your help.
> >
> >
> [...]
>
> Cheers,
> Denis
> -----BEGIN PGP SIGNATURE-----
>
> iQIzBAEBCAAdFiEEOzEprx3d76WjfYGPCDGwvQPYsYIFAl6xwwYACgkQCDGwvQPY
> sYLojhAAsjgWSZGK2UxcY8edk2ZkPYIMmnJ1yk+fRjip+SkwcxjF89CPTQxbg0Dw
> 4DbE8+3TM+O84zDq9gXHnJR5vfW5cs9US4KMVuXLVH2nPKhLJ5PPueaz1VftNE0r
> aEo48tXNqwRelSJuRyNvklrS7E/Jn0n7LLVkSuc6p07iYyl8hzIlkN85TN80v2V1
> 1MLBuEXPHXsg2LEY1Wyv0f9REHY66w1YTvSSsVK/e3Zjhl8iQMdPTpe42sBu+hVQ
> qdNAHKZfPxQ0JR71kOhfori+BzvvYxPUiqdONscf6N7voWa2miihyzFVvvcGn+hg
> dRqBtse8zK7gCqsl1ylnWoXNgwZX0n9jhCUzRhFVYMQz642WJ08It/bsSvVK8L7s
> 5iIYhbp6PotX5IfXXv+iw5MoIq2UnFLyCCrjhQz65tjrP7KSUJAvHsXjpWO5LQI3
> UU1fKpIe/nn7oetDMZcEoaRHJVlxMpcA0/Rt6keK+cno/JwEWoDp/uo4755lJDhd
> +HeAsN7DLy2/dFukDtPgIlEPU2xYPN5ua0OZWJ2+2bG2DAl3pP5e3R7WQbLA679/
> fcmwNfOUva417jKNoR5CPfy6aX48BjFj9wtdmLnWDwmWZFoMvcLh4nFKkssJ4JWF
> 1ivHhPKHwmsohhHnRaNafaD/CB0jMh3CessyRdoEVT/+2H37N7U=
> =+TZs
> -----END PGP SIGNATURE-----
>
>
> _______________________________________________
> Dar-support mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/dar-support
>

_______________________________________________
Dar-support mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/dar-support