Re: Truncated archive/backup reparation

--===============0734933530584946727==
Content-Type: multipart/alternative; boundary="000000000000a0ec21063026eb33"

--000000000000a0ec21063026eb33
Content-Type: text/plain; charset="UTF-8"

Thanks Dennis,

RE: archive truncation, another thing that I've come across is the
behaviour of dar when sent the SIGTERM (or similar) signals. Normally this
leads to delayed termination including appending the catalog to the end of
the archive; however, this is not the case if dar is sent the SIGTERM
signal while paused between slices dar continues to wait for user input.
Would it be possible to override this so that if dar is sent a SIGTERM
signal while waiting for user input it instead continues with the intended
delayed termination so that the archive doesn't end up in a truncated state
(e.g. if the process is killed due to non-response to the SIGTERM signal)?
Also worth noting that the same behaviour occurs when dar is interrupted
while waiting for par2 to finish running between slices.

The backup I am trying to create (100s of Tb) is unfortunately going to
take quite a long time to run, longer than the routinely scheduled reboot
periods of the cluster I am working on, so am trying to code around the
scenario where a reboot signal is sent while par2 is running on a slice or
during a period when dar2 is otherwise paused (which I have been asked to
schedule in every few TBs).

Best,

Scott

On Wed, 12 Mar 2025 at 10:23, Denis Corbin <[email protected]> wrote:

> On 11/03/2025 22:49, Scott Ritchie wrote:
> > Hi Dennis,
>
> Hi Scott,
>
> >
> > Thanks for the quick response, and for confirming that the behaviour I
> > am seeing is actually a bug and not user error on my part!
>
> OK, I have found and fixed this issue (available on git, branch_2.7.x
> for now, will provide a package for 2.7.17.RC2 very soon).
> >
> > Regarding the new repaired backup that is created by -y, am I right in
> > understanding this would create a copy of all the slice data from the
> > truncated archive?
>
> correct
>
> > I'm in the process of attempting to do a backup of
> > several hundred TBs, so ideally would like to be able to resume backing
> > up without duplicating all the existing backup if i need to repair a
> > truncated archive during the process. Is there an alternative mechanism
> > by which the repair process can read through a truncated archive to
> > create an additional file with the missing catalogue data at the end,
> > from which a new incremental backup can be run to continue the backup
> > from where the truncated one left off?
>
> Well, *in theory*, you could try isolating the backup and use the
> isolated catalogue as backup of the one which is missing in the
> truncated backup. This way you could do whatever operation you want with
> that truncated backup without having to copy its data: from file
> restoration in direct access mode (much quicker than sequentially
> reading as tar does and which is available with dar with the
> --sequential-read option) use the isolated catalog for differential
> backup and so on.
>
> *In practice*, I tried this yesterday, and it failed due to archive
> corruption, because the isolation was not designed to be used on
> truncated backups. However looking at the code, the gap to have this
> working does not seem big, I'm currently looking into this.
>
> >
> > Best,
> >
> > Scott
> >
>
> Cheers,
> Denis
>
>

--000000000000a0ec21063026eb33
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Thanks Dennis,<div><br></div><div>RE: archive truncation, =
another=C2=A0thing that I&#39;ve come across is the behaviour of dar when s=
ent the SIGTERM (or similar) signals. Normally this leads to delayed termin=
ation including appending the catalog to the end of the archive; however, t=
his is not the case if dar is sent the SIGTERM signal while paused between =
slices=C2=A0dar continues to wait for user input. Would it be possible to o=
verride this so that if dar is sent a SIGTERM signal while waiting for user=
 input it instead continues with the intended delayed termination so that t=
he archive doesn&#39;t end up in a truncated state (e.g. if the process is =
killed due to non-response to the SIGTERM signal)? Also worth noting that t=
he same behaviour occurs when dar is interrupted while waiting for par2 to =
finish running between slices.</div><div><br></div><div>The backup I am try=
ing to create (100s of Tb) is unfortunately going to take quite a long time=
 to run, longer than the routinely scheduled reboot periods of the cluster =
I am working on, so am trying to code around the scenario where a reboot si=
gnal is sent while par2 is running on a slice or during a period when dar2 =
is otherwise paused (which I have been asked to schedule in every few TBs).=
</div><div><br></div><div>Best,</div><div><br></div><div>Scott</div></div><=
br><div class=3D"gmail_quote gmail_quote_container"><div dir=3D"ltr" class=
=3D"gmail_attr">On Wed, 12 Mar 2025 at 10:23, Denis Corbin &lt;<a href=3D"m=
ailto:[email protected]">[email protected]</a>&gt; wrote:<br></div><blockqu=
ote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px=
 solid rgb(204,204,204);padding-left:1ex">On 11/03/2025 22:49, Scott Ritchi=
e wrote:<br>
&gt; Hi Dennis,<br>
<br>
Hi Scott,<br>
<br>
&gt; <br>
&gt; Thanks for the quick response, and for confirming that the behaviour I=
 <br>
&gt; am seeing is actually a bug and not user error on my part!<br>
<br>
OK, I have found and fixed this issue (available on git, branch_2.7.x <br>
for now, will provide a package for 2.7.17.RC2 very soon).<br>
&gt; <br>
&gt; Regarding the new repaired backup that is created by -y, am I right in=
 <br>
&gt; understanding this would create a copy of all the slice data from the =
<br>
&gt; truncated archive? <br>
<br>
correct<br>
<br>
&gt; I&#39;m in the process of attempting to do a backup of <br>
&gt; several hundred TBs, so ideally would like to be able to resume backin=
g <br>
&gt; up without duplicating all the existing backup if i need to repair a <=
br>
&gt; truncated archive during the process. Is there an alternative mechanis=
m <br>
&gt; by which the repair process can read through a truncated archive to <b=
r>
&gt; create an additional file with the missing catalogue data at the end, =
<br>
&gt; from which a new incremental backup can be run to continue the backup =
<br>
&gt; from where the truncated one left off?<br>
<br>
Well, *in theory*, you could try isolating the backup and use the <br>
isolated catalogue as backup of the one which is missing in the <br>
truncated backup. This way you could do whatever operation you want with <b=
r>
that truncated backup without having to copy its data: from file <br>
restoration in direct access mode (much quicker than sequentially <br>
reading as tar does and which is available with dar with the <br>
--sequential-read option) use the isolated catalog for differential <br>
backup and so on.<br>
<br>
*In practice*, I tried this yesterday, and it failed due to archive <br>
corruption, because the isolation was not designed to be used on <br>
truncated backups. However looking at the code, the gap to have this <br>
working does not seem big, I&#39;m currently looking into this.<br>
<br>
&gt; <br>
&gt; Best,<br>
&gt; <br>
&gt; Scott<br>
&gt; <br>
<br>
Cheers,<br>
Denis<br>
<br>
</blockquote></div>

--000000000000a0ec21063026eb33--


--===============0734933530584946727==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline


--===============0734933530584946727==--