Re: Suppress alert when restarting
M/Monit Support <[email protected]> Tue, 11 Feb 2025 22:53:22 +0100
| Newsgroups | gmane.comp.monitoring.monit.general |
|---|---|
| Message-ID | <[email protected]> |
--Apple-Mail=_4DAC21E6-2D8B-402E-9BFC-9010BC2FB1C4 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Hello Tobias, Splitting the check as you did is a correct workaround. Internals: Both status tests in the same service check context trigger = the same "status" event type. The "restart" action triggers an implicit = "alert" action (thus restart is kind of combo). The alert can be = suppressed by the event type, but in this case, you want to be alerted = if the status error persists, so the filter is not granular enough in = your use case. An alternative workaround could be to suppress the alert and call a = custom notification script instead (which may send a mail), something = like: --8<-- CHECK PROGRAM MyContainer PATH /bin/check_container my_container noalert foo@bar if status !=3D 0 for 5 cycles then restart if status !=3D 0 for 10 cycles then exec '/usr/local/bin/notify.sh = foo@bar' --8<-- Best regards, The M/Monit team > On 9. 2. 2025, at 22:11, Tobias Langner via This is the general = mailing list for monit <[email protected]> wrote: >=20 > Hey, >=20 > I use monit to monitor the health of a bunch of docker containers that = sometimes die/become unhealthy for various reasons, when they are = configured to be restarted by monit, which most of the time fixes the = issue. >=20 > I'd like to only be notified by monit when the restart does not fix = the problem and the container is still unhealthy. I thought to configure = something like >=20 > CHECK PROGRAM MyContainer PATH /bin/check_container my_container > if status !=3D 0 for 5 cycles then restart > if status !=3D 0 for 10 cycles then alert > ... >=20 > in the hope that I would only get notified when the script returns a = failure for 10 cycles. However, I also get a notification after five = cycles, usually followed by one immediately after, telling me that = everything is good again. >=20 > How can I configure monit in a way to not get notified in such a case? = One way I came up with was to create to separate rules, one to restart = only that uses noalert to entirely supress alerts, and another one to = alert. But that seems like overkill. >=20 > Thanks for your help, best, >=20 > Tobias >=20 > PS: My request is the same as in this thread = <https://lists.nongnu.org/archive/html/monit-general/2014-05/msg00016.html= > but since a) the solution there does not work for me as I generally = don't have an execution failure and b) it is more than 10 years old, I = thought to rather create a new thread. >=20 >=20 --Apple-Mail=_4DAC21E6-2D8B-402E-9BFC-9010BC2FB1C4 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii <html><head><meta http-equiv=3D"content-type" content=3D"text/html; = charset=3Dus-ascii"></head><body style=3D"overflow-wrap: break-word; = -webkit-nbsp-mode: space; line-break: after-white-space;">Hello = Tobias,<div><br></div><div>Splitting the check as you did is a correct = workaround.</div><div><br></div><div>Internals: Both status tests in the = same service check context trigger the same "status" event type. The = "restart" action triggers an implicit "alert" action (thus restart is = kind of combo). The alert can be suppressed by the event type, but in = this case, you want to be alerted if the status error persists, so the = filter is not granular enough in your use = case.</div><div><br></div><div>An alternative workaround could be to = suppress the alert and call a custom notification script instead (which = may send a mail), something like:</div><div>--8<--</div><div>CHECK = PROGRAM MyContainer PATH /bin/check_container = my_container</div><div> noalert foo@bar</div><div> if = status !=3D 0 for 5 cycles then restart</div><div> if status = !=3D 0 for 10 cycles then exec '/usr/local/bin/notify.sh = foo@bar'</div><div>--8<--</div><div><br></div><div><br = id=3D"lineBreakAtBeginningOfMessage"><div> <div style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = line-break: after-white-space;"><div style=3D"color: rgb(0, 0, 0); = font-family: Helvetica; font-style: normal; font-variant-caps: normal; = font-weight: normal; letter-spacing: normal; text-align: start; = text-indent: 0px; text-transform: none; white-space: normal; = word-spacing: 0px; -webkit-text-stroke-width: 0px;">Best regards,<br>The = M/Monit team<br></div></div> </div> <div><br><blockquote type=3D"cite"><div>On 9. 2. 2025, at 22:11, Tobias = Langner via This is the general mailing list for monit = <[email protected]> wrote:</div><br = class=3D"Apple-interchange-newline"><div><div>Hey,<br><br>I use monit to = monitor the health of a bunch of docker containers that sometimes = die/become unhealthy for various reasons, when they are configured to be = restarted by monit, which most of the time fixes the issue.<br><br>I'd = like to only be notified by monit when the restart does not fix the = problem and the container is still unhealthy. I thought to configure = something like<br><br>CHECK PROGRAM MyContainer PATH = /bin/check_container my_container<br> if status !=3D 0 for 5 = cycles then restart<br> if status !=3D 0 for 10 cycles then = alert<br> ...<br><br>in the hope that I would only get notified = when the script returns a failure for 10 cycles. However, I also get a = notification after five cycles, usually followed by one immediately = after, telling me that everything is good again.<br><br>How can I = configure monit in a way to not get notified in such a case? One way I = came up with was to create to separate rules, one to restart only that = uses noalert to entirely supress alerts, and another one to alert. But = that seems like overkill.<br><br>Thanks for your help, = best,<br><br> Tobias<br><br>PS: My request is the same as in this = thread = <https://lists.nongnu.org/archive/html/monit-general/2014-05/msg00016.h= tml> but since a) the solution there does not work for me as I = generally don't have an execution failure and b) it is more than 10 = years old, I thought to rather create a new = thread.<br><br><br></div></div></blockquote></div><br></div></body></html>= --Apple-Mail=_4DAC21E6-2D8B-402E-9BFC-9010BC2FB1C4--