Re: problem with backup of changing file

Hi Petr,

Yes you can disable this feature with the '--retry-on-change 0' option.

What you underline about tar is that is does not take care of modified
files at the time they were backed up. Thus, you can end having
corrupted files in a backup, if for example the file has some coherent
structure at application level and this application modifies it near the
beginning and near the end in coherent manner, frequently enough for it
occurs more than once during the time required to read the whole file.

During the backup process, if you simply read sequentially the file to
backup, you will end with an incoherent file where the beginning is not
coherent with the end of file.

I would rather suggest --- if possible in regard to the service they
address in production --- to pause the docker containers during the
backup. To minimize the duration of the pause, you can rely on the
--backup-hook-include, --backup-hook-exclude and --backup-hook-execute
options of dar, that can run a script or command to pause containers
when the backup process enters the directory
/data/docker/devicemapper/devicemapper and then can call a second time
(with slightly different arguments) when the backup exits this directory
in order to unpause those containers.

Doing this way, you could keep the retry-on-change feature active and
thus be sure your "data" file is saved properly,  which means that your
containers will be able to restart from restored backup without this
risk of failure.

My 2 cents
Denis

Le 17/08/2022 à 17:45, Petr Skoda a écrit :
> Hi Denis and all Dar lovers.
> 
> I have faced a serious problem with backup of my /data directory. I am
> not able to overcome the problem.
> I am using pipe over nc  -
> dar -c - -R /data |nc lto 5000
> 
> machine lto reads
> nc -l -p 5000|mbuffer -P 80 -m 1G > betelgeuse-data.1.dar
> 
> I am using mbuffer over screen as I can always check the amount of
> transfered data.
> I do not think the porblem is here - probably the dar -c itself would
> break as well
> 
> after writting 139GB (shown by mbuffer) and looking at size of file
> betelgeuse-data.1.dar
> everything stops with message
> 
> WARNING! File modified while reading it for backup. No more retry for that
> file to not exceed the wasted byte limit. File is
> /data/docker/devicemapper/devicemapper/data
> 
> And the enter will not help to continue ...
> 
> The problem is the docker devicemapper/data as it is a kind of a block
> file specially handled by docker. But for OS it should be just another
> file (the tar goes over it smoothly).
> It is about 100GB but does not change size - only the date is updated
> after ls -la it is current.
> root@betelgeuse:devicemapper# ls -la data
> -rw------- 1 root root 107374182400 Aug 17 17:42 data
> 
> Is there some option how I can tell the dar to write what is available
> and continue ?
> 
> Now it stops and does not react .. No data are flowing through the nc .
> 
> Thanks a lot
> Petr Skoda
> 
> 
>