[RFC PATCH 0/3] virtio_net: introduce an RX queue watchdog for stuck detection

From: Longjun Tang <[email protected]>

The virtio_net RX path relies on backend interrupt notifications. 
The backend(vhost_net/vhost-user) appends buffers to the used ring
and then notifies the guest, relying on the hypervisor(KVM) to inject
the interrupt. On receiving it, virtio_net schedules NAPI, drains the
used ring and refills descriptors.

If the hypervisor fails to inject the interrupt(e.g. a transient KVM
failure), the guest never schedules NAPI, so it neither consumes buffers
nor returns descriptors. The backend keeps appending until the ring is full,
then stops and, with no free descriptors left, also stops sending
notifications. Both sides now wait for the other, and the RX queue is
permanently stuck.

The root cause is the lost interrupt in the hypervisor, and the proper
fix belongs there. Nevertheless, I believe virtio_net needs a way to 
observe and diagnose it. Today, once a queue hangs, there is no signal
to the operator that anything is wrong: the RX path has no equivalent of
the TX watchdog (ndo_tx_timeout). This series lets virtio_net detect a 
stuck RX queue. It is detection only: the driver records the event and 
logs a warning, leaving recovery to a follow-up if desired.


Implementation
--------------

Patch 1 exports virtqueue_get_last_used_idx(), a read-only accessor for
the last_used_idx. Together with the existing virtqueue_poll(), it lets
a driver ask whether the device has produced buffers that the driver has
not consumed yet ("backlog").

Patch 2 adds a per-device RX watchdog that scans all RX queues once per
second. A queue is considered stuck when, for rx_watchdog_timeo seconds
(default 5, 0 disables it), all of the following hold simultaneously:
  - the queue has a non-zero backlog (used.idx != last_used_idx);
  - the driver makes no consumption progress (last_used_idx unchanged);
  - no new interrupt arrives (rq->calls unchanged).
On detection it logs a warning, rate-limited to once per timeout.

Patch 3 adds a per-queue rx_timeouts statistic, exposed via ethtool -S,
so the number of watchdog events can be observed per queue.


RFC
---

This is sent as an RFC to gather feedback on a few open points:

  - Scope: this series only detects the stuck queue. Is it better to
    keep detection and recovery separate, or should recovery (forcing a
    NAPI poll to drain the queue) be included?

  - Default: rx_watchdog_timeo defaults to 5 seconds (enabled). Should
    the watchdog be enabled by default, or disabled unless explicitly
    requested?

Parts of this series, including portions of this cover letter, were drafted
with AI assistance. I have carefully reviewed everything; 
questions and comments are welcome. 

Thanks!
Lange

Longjun Tang (3):
  virtio: add virtqueue_get_last_used_idx() helper
  virtio_net: introduce an RX queue watchdog for stuck detection
  virtio_net: add rx_timeouts per-queue statistic

 drivers/net/virtio_net.c     | 103 +++++++++++++++++++++++++++++++++++
 drivers/virtio/virtio_ring.c |  23 +++++++-
 include/linux/virtio.h       |   2 +
 3 files changed, 126 insertions(+), 2 deletions(-)


base-commit: 2709dd5ae32f0828f386327c76bba9f39f63a1c6
-- 
2.43.0