Re: [PATCH 1/2] hung_task: Reset warning budget when problem gets resolved

On Tue, Aug 04, 2026 at 11:18:39AM +0800, Lance Yang wrote:
> 
> On Mon, Aug 03, 2026 at 08:15:41PM -0400, Aaron Tomlin wrote:
> >From: Petr Mladek <[email protected]>
> >
> >sysctl_hung_task_warnings counts how many hung tasks are reported.
> >The watchdog does not report anything once the limit is reached.
> >Currently, this budget is decremented permanently, meaning the kernel is
> >left blind to subsequent hung tasks even after the original issue resolves.
> >
> >Keep the global sysctl_hung_task_warnings intact, and instead decrement
> >a copy (hung_task_warnings_printed) when warnings are printed. Reset
> >the copy back to the configured sysctl_hung_task_warnings limit once the
> >problem on the system gets resolved and check_hung_uninterruptible_tasks()
> >detects no hung tasks in a check interval.
> >
> >Also keep the copy updated when the global sysctl_hung_task_warnings
> >value is updated via sysctl, and update documentation to reflect
> >the new behavior.
> >
> >Assisted-by: gemini-3.5-flash
> >Signed-off-by: Petr Mladek <[email protected]>
> >---
> > Documentation/admin-guide/sysctl/kernel.rst |  6 ++--
> > kernel/hung_task.c                          | 34 ++++++++++++++++-----
> > 2 files changed, 31 insertions(+), 9 deletions(-)
> >
> >diff --git a/Documentation/admin-guide/sysctl/kernel.rst b/Documentation/admin-guide/sysctl/kernel.rst
> >index c6994e55d141..eaa1d837c739 100644
> >--- a/Documentation/admin-guide/sysctl/kernel.rst
> >+++ b/Documentation/admin-guide/sysctl/kernel.rst
> >@@ -459,8 +459,10 @@ hung_task_warnings
> > ==================
> > 
> > The maximum number of warnings to report. During a check interval
> >-if a hung task is detected, this value is decreased by 1.
> >-When this value reaches 0, no more warnings will be reported.
> >+if a hung task is detected, the internal warning budget is decreased by 1.
> >+When this budget reaches 0, no more warnings will be reported.
> >+The warning budget is reset back to the configured limit when the problem on the
> >+system gets resolved and no hung task is found.
> 
> Maybe just go with this?
> 
> "
> The maximum number of warnings to report. During a check interval
> if a hung task is detected, the internal warning budget is decreased by 1.
> When this budget reaches 0, no more detailed warnings will be reported.
> The warning budget is reset to the configured limit when no hung task is
> found.
> "
Hi Lance,

Acknowledged.

> 
> > This file shows up if ``CONFIG_DETECT_HUNG_TASK`` is enabled.
> > 
> > -1: report an infinite number of warnings.
> >diff --git a/kernel/hung_task.c b/kernel/hung_task.c
> >index 6fcc94ce4ca9..4e1fb0db79d1 100644
> >--- a/kernel/hung_task.c
> >+++ b/kernel/hung_task.c
> >@@ -59,6 +59,8 @@ static unsigned long __read_mostly sysctl_hung_task_check_interval_secs;
> > 
> > static int __read_mostly sysctl_hung_task_warnings = 10;
> > 
> >+static int hung_task_warnings_printed = 10;
> >+
> > static int __read_mostly did_panic;
> > static bool hung_task_call_panic;
> > 
> >@@ -247,9 +249,9 @@ static void hung_task_info(struct task_struct *t, unsigned long timeout,
> > 	 * CONFIG_DEFAULT_HUNG_TASK_TIMEOUT. Therefore, complain
> > 	 * accordingly
> > 	 */
> >-	if (sysctl_hung_task_warnings || hung_task_call_panic) {
> >-		if (sysctl_hung_task_warnings > 0)
> >-			sysctl_hung_task_warnings--;
> >+	if (hung_task_warnings_printed || hung_task_call_panic) {
> >+		if (hung_task_warnings_printed > 0)
> >+			hung_task_warnings_printed--;
> > 		pr_err("INFO: task %s:%d blocked%s for more than %ld seconds.\n",
> > 		       t->comm, t->pid, t->in_iowait ? " in I/O wait" : "",
> > 		       (jiffies - t->last_switch_time) / HZ);
> >@@ -264,7 +266,7 @@ static void hung_task_info(struct task_struct *t, unsigned long timeout,
> > 		sched_show_task(t);
> > 		debug_show_blocker(t, timeout);
> > 
> >-		if (!sysctl_hung_task_warnings)
> >+		if (!hung_task_warnings_printed)
> > 			pr_info("Future hung task reports are suppressed, see sysctl kernel.hung_task_warnings\n");
> > 	}
> > 
> >@@ -304,7 +306,7 @@ static void check_hung_uninterruptible_tasks(unsigned long timeout)
> > 	unsigned long last_break = jiffies;
> > 	struct task_struct *g, *t;
> > 	unsigned long this_round_count;
> >-	int need_warning = sysctl_hung_task_warnings;
> >+	int need_warning = hung_task_warnings_printed;
> > 	unsigned long si_mask = hung_task_si_mask;
> > 
> > 	/*
> >@@ -340,8 +342,10 @@ static void check_hung_uninterruptible_tasks(unsigned long timeout)
> >  unlock:
> > 	rcu_read_unlock();
> > 
> >-	if (!this_round_count)
> >+	if (!this_round_count) {
> 
> Hmm, should max_count, or more generally scan completion, be part of this
> condition too?
> 
> Maybe not, since a limited scan is already allowed to miss tasks ...
> 
> If max_count stopped us early, we’d reset the budget even though a hung
> task may still be later in the list ...
> 
> I probably wouldn’t change it. Just wanted to throw the question out here.

Thanks for raising the question!

I agree with your intuition that no change is needed here.

So, 'sysctl_hung_task_check_count' is designed to cap CPU usage during task
iteration. If an admin caps max_count below the active thread count,
requiring a complete scan before resetting would prevent the warning budget
from ever resetting after system recovery.

Furthermore, if a hung task happens to be beyond max_count in a given pass
and gets missed, resetting the budget ensures that when khungtaskd does
eventually reach and detect that hung task, it will log full diagnostic
details rather than suppressing them.

> >+		hung_task_warnings_printed = sysctl_hung_task_warnings;
> > 		return;
> >+	}
> > 
> > 	if (need_warning || hung_task_call_panic) {
> > 		si_mask |= SYS_INFO_LOCKS;
> >@@ -425,6 +429,22 @@ static int proc_dohung_task_timeout_secs(const struct ctl_table *table, int writ
> > 	return ret;
> > }
> > 
> >+static int proc_dohung_task_warnings(const struct ctl_table *table, int write,
> >+				     void *buffer,
> >+				     size_t *lenp, loff_t *ppos)
> >+{
> >+	int ret;
> >+
> >+	ret = proc_dointvec_minmax(table, write, buffer, lenp, ppos);
> >+
> >+	if (ret || !write)
> >+		return ret;
> >+
> >+	hung_task_warnings_printed = sysctl_hung_task_warnings;
> >+
> >+	return 0;
> >+}
> >+
> > /*
> >  * This is needed for proc_doulongvec_minmax of sysctl_hung_task_timeout_secs
> >  * and hung_task_check_interval_secs
> >@@ -480,7 +500,7 @@ static const struct ctl_table hung_task_sysctls[] = {
> > 		.data		= &sysctl_hung_task_warnings,
> > 		.maxlen		= sizeof(int),
> > 		.mode		= 0644,
> >-		.proc_handler	= proc_dointvec_minmax,
> >+		.proc_handler	= proc_dohung_task_warnings,
> > 		.extra1		= SYSCTL_NEG_ONE,
> > 	},
> > 	{
> >-- 
> 
> Apart from that, LGTM.
> 
> Reviewed-by: Lance Yang <[email protected]>
> Tested-by: Lance Yang <[email protected]>
> 
> Nice and simple ;) Thanks for carrying this forward!
> 
> Cheers, Lance


Thank you!

Kind regards,
-- 
Aaron Tomlin