Re: problem with backup of changing file

---------- Původní e-mail ----------
Od: Denis Corbin <[email protected]>
Komu: [email protected]
Datum: 17. 8. 2022 18:24:51
Předmět: Re: [Dar-support] problem with backup of changing file
"Hi Petr, 

Yes you can disable this feature with the '--retry-on-change 0' option. 
"



Thank you ! - This really works 





 

"What you underline about tar is that is does not take care of modified 
files at the time they were backed up. Thus, you can end having 
corrupted files in a backup, if for example the file has some coherent 
structure at application level and this application modifies it near the 
beginning and near the end in coherent manner, frequently enough for it 
occurs more than once during the time required to read the whole file. 

During the backup process, if you simply read sequentially the file to 
backup, you will end with an incoherent file where the beginning is not 
coherent with the end of file. 

I would rather suggest --- if possible in regard to the service they 
address in production --- to pause the docker containers during the 
backup."



Yes - this is a nice idea - using the hooks seems to be a good trick.

Thanks for suggestion. However for some cases I want simple backup the 
server as it is and I do not know what specific packages are installing some
special files disk handlers etc .... 


I Could in principle ignore just the known special files but I may not know 
what all is there.










 
" To minimize the duration of the pause, you can rely on the 
--backup-hook-include, --backup-hook-exclude and --backup-hook-execute 
options of dar, that can run a script or command to pause containers 
when the backup process enters the directory 
/data/docker/devicemapper/devicemapper and then can call a second time 
(with slightly different arguments) when the backup exits this directory 
in order to unpause those containers. 

Doing this way, you could keep the retry-on-change feature active and 
thus be sure your "data" file is saved properly, which means that your 
containers will be able to restart from restored backup without this 
risk of failure. 
"



yes it would be nice to be sure that all is on tape . However IMHO you may 
never be sure that all is there.

Anyway thanks for quick reply



"
My 2 cents 
Denis 

Le 17/08/2022 à 17:45, Petr Skoda a écrit : 
> Hi Denis and all Dar lovers. 
> 
> I have faced a serious problem with backup of my /data directory. I am 
> not able to overcome the problem. 
> I am using pipe over nc  - 
> dar -c - -R /data |nc lto 5000 
> 
> machine lto reads 
> nc -l -p 5000|mbuffer -P 80 -m 1G > betelgeuse-data.1.dar 
> 
> I am using mbuffer over screen as I can always check the amount of 
> transfered data. 
> I do not think the porblem is here - probably the dar -c itself would 
> break as well 
> 
> after writting 139GB (shown by mbuffer) and looking at size of file 
> betelgeuse-data.1.dar 
> everything stops with message 
> 
> WARNING! File modified while reading it for backup. No more retry for that

> file to not exceed the wasted byte limit. File is 
> /data/docker/devicemapper/devicemapper/data 
> 
> And the enter will not help to continue ... 
> 
> The problem is the docker devicemapper/data as it is a kind of a block 
> file specially handled by docker. But for OS it should be just another 
> file (the tar goes over it smoothly). 
> It is about 100GB but does not change size - only the date is updated 
> after ls -la it is current. 
> root@betelgeuse:devicemapper# ls -la data 
> -rw------- 1 root root 107374182400 Aug 17 17:42 data 
> 
> Is there some option how I can tell the dar to write what is available 
> and continue ? 
> 
> Now it stops and does not react .. No data are flowing through the nc . 
> 
> Thanks a lot 
> Petr Skoda 
> 
> 
> 
"