Re: delayed child restart with incremental back-off

--000000000000b103db05c1f967b4
Content-Type: text/plain; charset="UTF-8"

What happened at the time was that I met up with the OTP team and discussed
it, and they eventually agreed that this was a good thing. However, it
needed more work to be accepted (and I realized a couple of weaknesses in
the implementation that I needed to address), but I never found time to do
more work on it.

        /Richard


Den fre 30 apr. 2021 kl 09:28 skrev Nicolas Martyanoff <[email protected]>:

>
> Hi,
>
> Nine years ago, an interesting patch [1] was submitted by Richard Carlsson
> allowing to delay the re-creation of failed children in supervisors.
>
> After a quick discussions, the official answer was that the OTP team
> would discuss about it [2]. There is no further message on the mailing
> list.
>
> Was there an official response ?
>
> I have various supervisors whose children handle network connections.
> When something goes wrong with the connection, children die and are
> immediately restarted. Most of the times, errors are transient (remote
> server restarting, temporary network issue, etc.), but retrying without
> any delay is pretty much guaranteed to fail again. And of course after a
> few retries, the application dies which is unacceptable.
>
> This kind of behaviour is a huge problem: it fills logs with multiple
> copies of identical errors and causes a system failure.
>
> In general, if I could, I would use restart delays with exponential
> backoff everywhere because in practice, restarting immediately is almost
> never the right approach: code errors do not disappear when restarting
> so they are going to get triggered again immediately, and external errors
> are not magically fixed by retrying without any delay.
>
> Is there still interest for this patch ?
>
> [1] https://erlang.org/pipermail/erlang-patches/2012-January/002575.html
> [2] https://erlang.org/pipermail/erlang-patches/2012-January/002597.html
>
> --
> Nicolas Martyanoff
> http://snowsyn.net
> [email protected]
>

--000000000000b103db05c1f967b4
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">What happened at the time was that I met up with the OTP t=
eam and discussed it, and they eventually agreed that this was a good thing=
. However, it needed more work to be accepted (and I realized a couple of w=
eaknesses in the implementation that I needed to address), but I never foun=
d time to do more work on it.<br clear=3D"all"><div><div dir=3D"ltr" class=
=3D"gmail_signature" data-smartmail=3D"gmail_signature"><br>=C2=A0 =C2=A0 =
=C2=A0 =C2=A0 /Richard</div></div><br></div><br><div class=3D"gmail_quote">=
<div dir=3D"ltr" class=3D"gmail_attr">Den fre 30 apr. 2021 kl 09:28 skrev N=
icolas Martyanoff &lt;<a href=3D"mailto:[email protected]">[email protected]=
m</a>&gt;:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0=
px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><br>
Hi,<br>
<br>
Nine years ago, an interesting patch [1] was submitted by Richard Carlsson<=
br>
allowing to delay the re-creation of failed children in supervisors.<br>
<br>
After a quick discussions, the official answer was that the OTP team<br>
would discuss about it [2]. There is no further message on the mailing<br>
list.<br>
<br>
Was there an official response ?<br>
<br>
I have various supervisors whose children handle network connections.<br>
When something goes wrong with the connection, children die and are<br>
immediately restarted. Most of the times, errors are transient (remote<br>
server restarting, temporary network issue, etc.), but retrying without<br>
any delay is pretty much guaranteed to fail again. And of course after a<br=
>
few retries, the application dies which is unacceptable.<br>
<br>
This kind of behaviour is a huge problem: it fills logs with multiple<br>
copies of identical errors and causes a system failure.<br>
<br>
In general, if I could, I would use restart delays with exponential<br>
backoff everywhere because in practice, restarting immediately is almost<br=
>
never the right approach: code errors do not disappear when restarting<br>
so they are going to get triggered again immediately, and external errors<b=
r>
are not magically fixed by retrying without any delay.<br>
<br>
Is there still interest for this patch ?<br>
<br>
[1] <a href=3D"https://erlang.org/pipermail/erlang-patches/2012-January/002=
575.html" rel=3D"noreferrer" target=3D"_blank">https://erlang.org/pipermail=
/erlang-patches/2012-January/002575.html</a><br>
[2] <a href=3D"https://erlang.org/pipermail/erlang-patches/2012-January/002=
597.html" rel=3D"noreferrer" target=3D"_blank">https://erlang.org/pipermail=
/erlang-patches/2012-January/002597.html</a><br>
<br>
-- <br>
Nicolas Martyanoff<br>
<a href=3D"http://snowsyn.net" rel=3D"noreferrer" target=3D"_blank">http://=
snowsyn.net</a><br>
<a href=3D"mailto:[email protected]" target=3D"_blank">[email protected]</a=
><br>
</blockquote></div>

--000000000000b103db05c1f967b4--