Re: [PATCH v2] rt-tests: hwlatdetect: Add MTBF calculation

On Tue, 6 Jan 2026, Costa Shulyupin wrote:

> Hwlatdetect reports the number of latency spikes but provides no
> information about their frequency distribution over time. This makes it
> difficult to compare results - a test with 10 spikes over 1 hour is very
> different from 10 spikes over 24 hours, but both show 'spikes = 10'.
> 
> Add Mean Time Between Failures (MTBF) calculation to quantify spike
> frequency.
> 
> By definition MTBF = total operating time / number of failures.
> 
> When the failure interval is large relative to test duration, this
> formula is biased.  For example, imagine stable periodic failures.  The
> total operating time will include time before the first failure and
> after the last failure.  These intervals are determined by when the test
> starts and stops, not by the system’s failure behavior, which adds
> measurement bias.  The resulting MTBF will vary between runs even for
> stable periodic failures.
> 
> To reduce this bias, calculate MTBF using only the time between the
> first and last failure, divided by the number of intervals (failures
> minus one):
> 
> MTBF = (timestamp of last failure - timestamp of first failure)
> / (number of failures - 1)
> 
> In hwlatdetect, the failures are called samples.
> 
> This metric enables meaningful comparison of real-time performance
> consistency across different test runs, hardware configurations, and
> kernel versions.  It can be considered a KPI for real-time stability,
> relevant for certification and SLA evaluation.
> 
> ---
> 
> Changed in v2:
> - Use another more stable calculation of MTBF
> 
> Signed-off-by: Costa Shulyupin <[email protected]>
> ---
>  src/hwlatdetect/hwlatdetect.py | 8 ++++++++
>  1 file changed, 8 insertions(+)
> 
> diff --git a/src/hwlatdetect/hwlatdetect.py b/src/hwlatdetect/hwlatdetect.py
> index 68f312db639f..6d9db9aec933 100755
> --- a/src/hwlatdetect/hwlatdetect.py
> +++ b/src/hwlatdetect/hwlatdetect.py
> @@ -19,6 +19,7 @@
>  debugging = False
>  quiet = False
>  watch = False
> +first = last = 0
>  
>  
>  def debug(dstr):
> @@ -306,6 +307,10 @@     def detect(self):
>                  pollcnt += 1
>                  val = self.get_sample()
>                  while val:
> +                    global first, last
> +                    if not first:
> +                        first = val.timestamp
> +                    last = val.timestamp
>                      self.samples.append(val)
>                      if watch:
>                          val.display()
> @@ -527,6 +532,9 @@     def cleanup(self):
>      exceeding = detect.get("count")
>      info(f"Samples exceeding threshold: {exceeding}")
>  
> +    if exceeding > 1:
> +        info(f"MTBF: {(float(last)-float(first))/ (exceeding - 1):.3f} seconds")

exceeding = detect.get("count"), but now we have a count that isn't the 
same as a sample, you might have to rework this for newer kernels.

> +
>      if detect.have_msr:
>          finishsmi = detect.getsmicounts()
>          total_smis = 0
> -- 
> 2.52.0
> 
> 
>