Re: Issue with Both Diskful Nodes Being Outdated in DRBD9

--000000000000a322b0062bce25a1
Content-Type: text/plain; charset="UTF-8"

Hi Rui,

This should be fixed by the following commit from Phil:
https://github.com/LINBIT/drbd/commit/d8214d47d13a4c9b5c8f8cae7989de7996983688

Please test it to ensure that it fixes your precise scenario.

Best regards,
Joel


On Tue, 7 Jan 2025 at 17:12, Joel Colledge <[email protected]> wrote:

> Hi Rui,
>
> > I installed DRBD 9.2.12 and retested, but the issue persists.
> >
> > I think the logic of this problem is quite clear. First, an Inconsistent
> replication serving as a sync target can be promoted to the primary when it
> is connected to an uptodate replication. Next, if the connection with the
> primary node is lost, the uptodate replication becomes outdated. Finally,
> after the network is restored and synchronization is completed, the sync
> target updates its metadata to match the sync source, which causes its
> state to also become outdated.
>
> I am able to reproduce the problem too now.
>
> > Would it be possible to introduce a parameter that allows users to
> prevent the promotion of Inconsistent replications? This could help avoid
> the the issue. If you have other solutions, that would be great as well, of
> course.
>
> I believe the correct fix is at step 5. of your original reproduction
> steps. Node B should know that A does not have access to any UpToDate
> data and so cannot complete any writes. So node B should not become
> Outdated.
>
> I'm not sure how the exact implementation would work. We may need to
> add more information to the two-phase commit packets to allow node B
> to reliably determine its disk state.
>
> The fix should also make sure that node A does not consider node B to
> be Outdated at this stage.
>
> > Additionally, are there other methods to remove the outdated tag from
> the metadata, aside from using the primary --force command?
>
> It should be possible with the "get-gi" and "set-gi" commands, but I
> recommend using "primary --force", because it is much simpler.
>
> Best regards,
> Joel
>

--000000000000a322b0062bce25a1
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div>Hi Rui,</div><div><br></div><div>This should be fixed=
 by the following commit from Phil:<br></div><div><a href=3D"https://github=
.com/LINBIT/drbd/commit/d8214d47d13a4c9b5c8f8cae7989de7996983688">https://g=
ithub.com/LINBIT/drbd/commit/d8214d47d13a4c9b5c8f8cae7989de7996983688</a></=
div><div><br></div><div>Please test it to ensure that it fixes your precise=
 scenario.</div><div><br></div><div>Best regards,</div><div>Joel</div><div>=
<br></div></div><br><div class=3D"gmail_quote gmail_quote_container"><div d=
ir=3D"ltr" class=3D"gmail_attr">On Tue, 7 Jan 2025 at 17:12, Joel Colledge =
&lt;<a href=3D"mailto:[email protected]">[email protected]</a=
>&gt; wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px=
 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hi =
Rui,<br>
<br>
&gt; I installed DRBD 9.2.12 and retested, but the issue persists.<br>
&gt;<br>
&gt; I think the logic of this problem is quite clear. First, an Inconsiste=
nt replication serving as a sync target can be promoted to the primary when=
 it is connected to an uptodate replication. Next, if the connection with t=
he primary node is lost, the uptodate replication becomes outdated. Finally=
, after the network is restored and synchronization is completed, the sync =
target updates its metadata to match the sync source, which causes its stat=
e to also become outdated.<br>
<br>
I am able to reproduce the problem too now.<br>
<br>
&gt; Would it be possible to introduce a parameter that allows users to pre=
vent the promotion of Inconsistent replications? This could help avoid the =
the issue. If you have other solutions, that would be great as well, of cou=
rse.<br>
<br>
I believe the correct fix is at step 5. of your original reproduction<br>
steps. Node B should know that A does not have access to any UpToDate<br>
data and so cannot complete any writes. So node B should not become<br>
Outdated.<br>
<br>
I&#39;m not sure how the exact implementation would work. We may need to<br=
>
add more information to the two-phase commit packets to allow node B<br>
to reliably determine its disk state.<br>
<br>
The fix should also make sure that node A does not consider node B to<br>
be Outdated at this stage.<br>
<br>
&gt; Additionally, are there other methods to remove the outdated tag from =
the metadata, aside from using the primary --force command?<br>
<br>
It should be possible with the &quot;get-gi&quot; and &quot;set-gi&quot; co=
mmands, but I<br>
recommend using &quot;primary --force&quot;, because it is much simpler.<br=
>
<br>
Best regards,<br>
Joel<br>
</blockquote></div>

--000000000000a322b0062bce25a1--