Re: Truncated archive/backup reparation

--===============1934707729912558443==
Content-Type: multipart/alternative; boundary="000000000000fdf86106303967a3"

--000000000000fdf86106303967a3
Content-Type: text/plain; charset="UTF-8"

Hi Denis,

Thanks so much for all the advice and for implementing/patching so quickly.
I appreciate that it will take some time for these to be rolled out into
the stable releases.

RE: the -Q option, after seeing that -Q is not compatible with -p, I've
been playing around with the possibility of trapping the SIGTERM signal in
my the bash script I've written to run dar with a view towards
programmatically providing the sequence of ESC/ENTER commands required to
respond to the user input prompts to lead to a clean delayed termination.

In doing so I managed to trigger the following exception which the dar
command has asked me to report:

Aborting program. An error occurred while calling libdar: Cannot write down
the archive: user_interaction::inherited_pause should not throw an
exception toward libdar
###############################################
#   NOT CAUGHT EXCEPTION,                     #
#                         E X I T I N G !     #
#                                             #
###############################################
 THANKS TO REPORT THE PREVIOUS OUTPUT TO MAINTAINER
 GIVING A DESCRIPTION OF THE CIRCUMSTANCES.
 IF POSSIBLE TRY TO PRODUCE THIS ERROR, A
 SCENARIO THAT CAN REPRODUCE IT WOULD HELP MUCH
 IN SOLVING THIS PROBLEM.                THANKS

I think this uncaught exception can be reliably reproduced by calling dar
asynchronously from a bash script and then killing the script:

```
#/bin/bash

# Catch interrupt signals and cleanup
function cleanup {
   # do stuff
   exit 1
}
trap cleanup SIGHUP SIGINT SIGQUIT SIGTERM SIGUSR1

# Run dar asynchronously and wait for the response, so we can use trap
dar -c path/to/backup -s 10GB &
wait
```

Execute above script, then kill -15 the the process running the bash script
without killing the running dar process.

Hopefully this provides helpful insight into one of the possible ways this
scenario can be triggered.

Best,

Scott

On Wed, 12 Mar 2025 at 23:18, Denis Corbin <[email protected]> wrote:

> On 12/03/2025 16:32, Scott Ritchie wrote:
> > Thanks Dennis,
>
> Hi Scott
>
> As said, you have a fix in 2.7.17.RC2 for the repairing of a sliced
> backup in git with tag v2.7.17.RC2 and now in addition a package here:
>
>   https://dar.edrusb.org/dar.linux.free.fr/Interim_releases
>
> I currently run the non-regression tests on that code, which takes
> several days, so be patient before the release 2.7.17.
>
> I could add the possibility to create an isolated a catalog from a
> truncated backup, but this requires the -affs feature that is only
> available in the 2.8.0 dev branch (= git master branch), so I have added
> this new feature there it will be released with 2.8.0, in a few months.
> Also added a FAQ on the possible way to manage a truncated backup, ---
> OK this is not a Frequently Asked question as nobody ever asked on that
> feature :) ---- but that was the easiest place to add this short
> documentation.
>
> >
> > RE: archive truncation, another thing that I've come across is the
> > behaviour of dar when sent the SIGTERM (or similar) signals. Normally
> > this leads to delayed termination including appending the catalog to the
> > end of the archive; however, this is not the case if dar is sent the
> > SIGTERM signal while paused between slices dar continues to wait for
> > user input. Would it be possible to override this so that if dar is sent
> > a SIGTERM signal while waiting for user input it instead continues with
> > the intended delayed termination so that the archive doesn't end up in a
> > truncated state (e.g. if the process is killed due to non-response to
> > the SIGTERM signal)? Also worth noting that the same behaviour occurs
> > when dar is interrupted while waiting for par2 to finish running between
> > slices.
>
> AFAIR, the process is suspending on a system call when it waits for user
> to answer... To address this type of use case, where you want to
> automate a process, there is the -Q option that has been added. Maybe
> this will address you need better than having the dar questions to
> answer manually.
>
> >
> > The backup I am trying to create (100s of Tb) is unfortunately going to
> > take quite a long time to run, longer than the routinely scheduled
> > reboot periods of the cluster I am working on, so am trying to code
> > around the scenario where a reboot signal is sent while par2 is running
> > on a slice or during a period when dar2 is otherwise paused (which I
> > have been asked to schedule in every few TBs).
>
> You could also split the data to backup in several smaller backups (one
> per top directory or something in that taste that makes sense) and
> gather then into a single dar_manager database. You can then choose
> different rotation frequency and retention period between these smaller
> backups and have a single point of access through dar_manager.
>
> >
> > Best,
> >
> > Scott
>
> Cheers,
> Denis
>

--000000000000fdf86106303967a3
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi Denis,<br><br>Thanks so much for all the advice and for=
 implementing/patching so quickly. I appreciate that it will take some time=
 for these to be rolled out into the stable releases.<div><br></div><div>RE=
: the -Q option, after seeing that -Q is not compatible with -p, I&#39;ve b=
een playing around with the possibility of trapping the SIGTERM signal in m=
y the bash script I&#39;ve written to run dar with a view towards programma=
tically providing the sequence of ESC/ENTER commands required to respond to=
 the user input prompts to lead to a clean delayed termination. <br><br>In =
doing so I managed to trigger the following exception which the dar command=
 has asked me to report:<br><br><font face=3D"monospace" size=3D"1">Abortin=
g program. An error occurred while calling libdar: Cannot write down the ar=
chive: user_interaction::inherited_pause should not throw an exception towa=
rd libdar<br>###############################################<br># =C2=A0 NO=
T CAUGHT EXCEPTION, =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0=
 =C2=A0 =C2=A0 #<br># =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 E X I T I N G ! =C2=A0 =C2=A0 #<br># =C2=A0=
 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 #=
<br>###############################################<br>=C2=A0THANKS TO REPO=
RT THE PREVIOUS OUTPUT TO MAINTAINER<br>=C2=A0GIVING A DESCRIPTION OF THE C=
IRCUMSTANCES.<br>=C2=A0IF POSSIBLE TRY TO PRODUCE THIS ERROR, A<br>=C2=A0SC=
ENARIO THAT CAN REPRODUCE IT WOULD HELP MUCH<br>=C2=A0IN SOLVING THIS PROBL=
EM. =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0THANKS</font><br=
>=C2=A0<br>I think this uncaught exception can be reliably reproduced by ca=
lling dar asynchronously from a bash script and then killing the script:<br=
><br>```<br>#/bin/bash<br><br># Catch interrupt signals and cleanup<br>func=
tion cleanup {<br>=C2=A0 =C2=A0# do stuff<br>=C2=A0 =C2=A0exit 1<br>}<br>tr=
ap cleanup SIGHUP SIGINT SIGQUIT SIGTERM SIGUSR1<br><br># Run dar asynchron=
ously and wait for the response, so we can use trap=C2=A0<br>dar -c path/to=
/backup -s 10GB &amp;<br>wait<br>```<br><br>Execute above script, then kill=
 -15 the the process running the bash script without killing the running da=
r process.<br><br>Hopefully this provides helpful insight into one of the p=
ossible ways this scenario can be triggered.</div><div><br></div><div>Best,=
</div><div><br></div><div>Scott=C2=A0=C2=A0<br></div></div><br><div class=
=3D"gmail_quote gmail_quote_container"><div dir=3D"ltr" class=3D"gmail_attr=
">On Wed, 12 Mar 2025 at 23:18, Denis Corbin &lt;<a href=3D"mailto:dar.linu=
[email protected]">[email protected]</a>&gt; wrote:<br></div><blockquote class=3D"g=
mail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204=
,204,204);padding-left:1ex">On 12/03/2025 16:32, Scott Ritchie wrote:<br>
&gt; Thanks Dennis,<br>
<br>
Hi Scott<br>
<br>
As said, you have a fix in 2.7.17.RC2 for the repairing of a sliced <br>
backup in git with tag v2.7.17.RC2 and now in addition a package here:<br>
<br>
=C2=A0 <a href=3D"https://dar.edrusb.org/dar.linux.free.fr/Interim_releases=
" rel=3D"noreferrer" target=3D"_blank">https://dar.edrusb.org/dar.linux.fre=
e.fr/Interim_releases</a><br>
<br>
I currently run the non-regression tests on that code, which takes <br>
several days, so be patient before the release 2.7.17.<br>
<br>
I could add the possibility to create an isolated a catalog from a <br>
truncated backup, but this requires the -affs feature that is only <br>
available in the 2.8.0 dev branch (=3D git master branch), so I have added =
<br>
this new feature there it will be released with 2.8.0, in a few months. <br=
>
Also added a FAQ on the possible way to manage a truncated backup, --- <br>
OK this is not a Frequently Asked question as nobody ever asked on that <br=
>
feature :) ---- but that was the easiest place to add this short <br>
documentation.<br>
<br>
&gt; <br>
&gt; RE: archive truncation, another=C2=A0thing that I&#39;ve come across i=
s the <br>
&gt; behaviour of dar when sent the SIGTERM (or similar) signals. Normally =
<br>
&gt; this leads to delayed termination including appending the catalog to t=
he <br>
&gt; end of the archive; however, this is not the case if dar is sent the <=
br>
&gt; SIGTERM signal while paused between slices=C2=A0dar continues to wait =
for <br>
&gt; user input. Would it be possible to override this so that if dar is se=
nt <br>
&gt; a SIGTERM signal while waiting for user input it instead continues wit=
h <br>
&gt; the intended delayed termination so that the archive doesn&#39;t end u=
p in a <br>
&gt; truncated state (e.g. if the process is killed due to non-response to =
<br>
&gt; the SIGTERM signal)? Also worth noting that the same behaviour occurs =
<br>
&gt; when dar is interrupted while waiting for par2 to finish running betwe=
en <br>
&gt; slices.<br>
<br>
AFAIR, the process is suspending on a system call when it waits for user <b=
r>
to answer... To address this type of use case, where you want to <br>
automate a process, there is the -Q option that has been added. Maybe <br>
this will address you need better than having the dar questions to <br>
answer manually.<br>
<br>
&gt; <br>
&gt; The backup I am trying to create (100s of Tb) is unfortunately going t=
o <br>
&gt; take quite a long time to run, longer than the routinely scheduled <br=
>
&gt; reboot periods of the cluster I am working on, so am trying to code <b=
r>
&gt; around the scenario where a reboot signal is sent while par2 is runnin=
g <br>
&gt; on a slice or during a period when dar2 is otherwise paused (which I <=
br>
&gt; have been asked to schedule in every few TBs).<br>
<br>
You could also split the data to backup in several smaller backups (one <br=
>
per top directory or something in that taste that makes sense) and <br>
gather then into a single dar_manager database. You can then choose <br>
different rotation frequency and retention period between these smaller <br=
>
backups and have a single point of access through dar_manager.<br>
<br>
&gt; <br>
&gt; Best,<br>
&gt; <br>
&gt; Scott<br>
<br>
Cheers,<br>
Denis<br>
</blockquote></div>

--000000000000fdf86106303967a3--


--===============1934707729912558443==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline


--===============1934707729912558443==--