Re: Backup on 2.7.1 ends prematurely with exception error + multi-threading question

On 18/08/2021 22:57, Tsukasa wrote:
> Hi everybody, Hi Denis,

Hello Takuya,

> 
> It's been a while since I wrote here, which is a good thing!
> After using version 2.5 for ages, I decided it's time to compile a shiny
> new version 2.7.1.
> I'm currently running Kubuntu 20.04. I downloaded the source packages
> for dar and libthreadar, installed most dependencies, but ended up
> disabling rsync and curl, since they prevented static linking (rsync)
> and some other error for curl.
> Compilation and installation of libthreadar and dar completed without
> errors.
> 
> Then during my first backup using this version, the backup ended
> prematurely like this:
> 
>> Finished Inspecting directory
>> /home/tx/.cache/yarn/v6/npm-prepend-http-2.0.0-e92434bfa5ea8c19f41cdfd401d741a3c819d897-integrity/node_modules
>> , saved 8 kio, compression ratio   25%
>> Finished Inspecting directory
>> /home/tx/.cache/yarn/v6/npm-prepend-http-2.0.0-e92434bfa5ea8c19f41cdfd401d741a3c819d897-integrity
>> , saved 8 kio, compression ratio   25%
>> Final memory cleanup...
>> ---- exception type = [BUG] ----------
>> [source]
>>         File escape.cpp line 858 : it seems to be a bug here
>>         stack dump : [0x515405]
>>         stack dump : [0x42ec51]
>>         stack dump : [0x5249d0]
>>         stack dump : [0x57c42f]
>>         stack dump : [0x5249d0]
>>         stack dump : [0x54f11d]
>>         stack dump : [0x5249d0]
>>         stack dump : [0x5a74c1]
>>         stack dump : [0x5b1b19]
>>         stack dump : [0x53067f]
>>         stack dump : [0x533852]
>>         stack dump : [0x534cc8]
>>         stack dump : [0x4e737c]
>>         stack dump : [0x4d3024]
>>         stack dump : [0x4d7cd8]
>>         stack dump : [0x4a8435]
>>         stack dump : [0x870a40]
>>         stack dump : [0x4aa43e]
>> [most outside call]
>> -----------------------------------
>>
>> INTERNAL ERROR, PLEASE REPORT THE PREVIOUS OUTPUT TO MAINTAINER
> 
> I tried a second time and it happened again. Then I excluded the .cache
> directory, since it was unimportant anyway, and tried again. This time
> the backup completed without showing any errors. I plan to verify the
> backup tomorrow.
> Please let me know what additional info you need about that error.

Thanks for this feedback!

the bug takes place when dar truncate an archive to re-save a file
(because this one has change or gave poor compression ratio and so on).
The escape layer (which adds tape marks and is also used for sparse file
handling) reports an incoherence in the layer just below it (its offset
is less than the offset requested to truncate at).

What would be useful to fix this bug would be for me to know the options
you have been using. In particular, have you been using slicing,
encryption (which algo)...

Ideally a reproducible scenario would let me fix the bug much quicker of
course... but I bet this will not be easy to reproduce, right?

> 
> One more question about the new multi-threading option. Since I backup
> to tape, which only supports sequential access, can multi-threading
> cause any problems? I'm used to seeing "Resaving uncompressed the inode
> data to gain space is not possible, keeping data compressed", many times
> while backing up to tape, so I'm wondering if that new block compression
> might cause problems? Is it still written sequentially?

The fact compression is done with a single or multiple thread is
invisible to the underlying layer that writes to pipe or encrypts or
that add tape marks (the escape layer that reported the bug here above)
or yet the layer that writes data to different slices...
See the global view at the end of chapter here for the big picture:
http://dar.linux.free.fr/doc/Notes.html#archive_structure

There should not be anything to worry about when using multiple threads
and writing (or reading) data to tapes/pipe or even using dar_slave.

About the message you see "Resaving uncompressed the inode data to gain
space is not possible, keeping data compressed", if this occurs too
often for small files you can increase the file size threshold below
which dar will not try to compress files (--mincompr option).

> 
> Thanks and best regards,
> Takuya
> 
> 

Cheers,
Denis