[PATCH ath-next v3 0/9] wifi: ath11k: airtime queue limits, fairness and a driver TXQ scheduler

ath11k advertises neither NL80211_EXT_FEATURE_AQL nor
NL80211_EXT_FEATURE_AIRTIME_FAIRNESS, and every user of either inside
mac80211 is gated on the corresponding bit. Both mechanisms are inert on
this driver, and the driver keeps no queue of its own, so on a saturated
download every queued byte sits in the hardware TX rings, where there is
no AQM and no flow separation.

Measured on an IPQ8074 access point serving one HE station: saturated TCP
downstream, a 20 Hz probe from that station through the same access
category, arms interleaved, three pairs each.

  stock                             94.6 Mbit/s   155 ms
  series, default limit             94.0 Mbit/s   157 ms
  series, BE aql_txq_limit 500/1000 74.3 Mbit/s    22 ms
  the same, patch 6 reverted        33.7 Mbit/s    21 ms

On stock, mac80211 reports no pending airtime and one frame of FQ-CoDel
backlog, and aql_txq_limit has the same effect at either setting, which
is none. The series puts the queue back under mac80211's control at no
cost where the limit does not bind, and roughly seven times lower latency
where it does. Patch 6, the completion-driven scheduling round, is worth
2.2x throughput at identical latency and nothing at the default limit.

The default limit does not hold the queue back because the queue reaches
it only at peaks. Sampling one station's own aql file at 1 Hz across
three 60 s runs, pending airtime averages 4.0 to 5.5 ms against the
12000 us high limit and peaks at 12.0 to 14.0 ms. At 500/1000 it sits at
the limit continuously, which is where the latency goes.

That accounting is close to what the hardware reports. Comparing what
mac80211 charges against the radio's own measurement for the same
station, as pending airtime over outstanding MSDUs and as tx_duration
over completed MSDUs, the charge is 47.2, 50.9 and 50.4 us per MSDU
against 90.9, 92.6 and 91.4 measured, a ratio near 1.85. Part of that is
structural rather than estimator error: fes_duration_us spans the frame
exchange sequence including the responding BlockAck, while
ieee80211_rate_expected_tx_airtime() amortises the overhead over an
assumed aggregate. Patch 3 exposes the driver-side outstanding MSDU count
so the two can be compared at all.

With two stations on one radio, a 1x1 VHT client and a 2x2 HE client both
receiving at once, the slow client takes 63 to 72 per cent of the medium
on stock for 95 Mbit/s while the fast one gets the rest, and
airtime_weight is refused with -EOPNOTSUPP. With the series and the limit
binding, the split is 44.4, 43.9 and 44.0 per cent at equal weights.

No default is changed. The series makes the existing knobs reach the
hardware; where the limit sits is left to whoever configures the access
point, and the curve is a continuous trade rather than a free win: on
this hardware 900/1800 us gives 92 Mbit/s at 127 ms, 550/1100 gives
81 Mbit/s at 31-46 ms, and 500/1000 gives 72 Mbit/s at 19-21 ms.

Patch 1 is a prerequisite for patch 2 rather than an unrelated fix, and
the series should not be split on it: the tx completion path it corrects
is also the path that returns the AQL charge, so with the feature bit set
and that path still calling dev_kfree_skb_any(), pending airtime for an
access category rises without bound and no station on the radio is
scheduled in it again.

Patch 2 advertises AQL. Patch 3 reports the outstanding MSDU count.
Patch 4 registers per-station transmit airtime and advertises airtime
fairness, which cannot be set without the reporting because the deficit
has no estimator fallback. Patch 5 moves the scheduling round into the
driver so a second context can drive it, patch 6 drives it from tx
completion, and patch 8 ends it when the hardware has no room. Patch 7
registers the hardware's per-PPDU receive duration into the same deficit.
Patch 9 gives the tx completion handler a budget, which it did not have
and now needs.

Patch 9 was unmeasured in v2 and is measured here: two soaks on this
series, a 91 s TCP soak at 1033 Mbit/s aggregate over five streams,
about 86000 completions per second, and a 60 s soak of 200-byte UDP
datagrams at 60906 packets per second. Neither produced a
status_fifo-is-full warning or a TCL ring-full failure. The bound the
patch adds is on the consumer of that FIFO; a device sustaining a
materially higher completion rate than this bench can produce would be
the case that changes the answer.

Two notes on overlap and prior art.

"wifi: ath11k/ath12k: implement TX flow control" [1] also replaces
.wake_tx_queue in ath11k, to stop -ENOMEM under a full TCL ring. Its
handler serves only the TXQ it is handed and calls neither
ieee80211_txq_schedule_start() nor ieee80211_next_txq(), so it removes
the deficit round robin that ieee80211_handle_wake_tx_queue() performs
today and forecloses airtime fairness on this driver. This series keeps
that scheduling and adds the airtime accounting it needs; patch 8 handles
descriptor exhaustion by ending the round on -ENOSPC or -ENOMEM rather
than by reading the ring pointer once per frame.

An AQL enable with a custom .wake_tx_queue and a tx-completion driven
push was proposed for ath11k in 2023 [2]. The design was right. What it
was asked for and did not have is the measurement above.

A related mac80211 change is posted separately: the airtime
queue limit is the same for every station, so it binds before the deficit
does and airtime_weight has no effect on the airtime a station receives.
Scaling the per-station limit by the weight, bounded by aql_threshold,
makes a 4:1 weight produce a 3.6:1 airtime ratio on the pair above and
1:4 produce 4.2:1, where without it the same three weight settings all
read 43 to 44 per cent. It is only useful on a driver that has the
accounting this series adds.

v3:
 - cover letter: v2 claimed the AQL estimate runs an order of magnitude
   under real occupancy. That was wrong; measured against the hardware's
   own tx_duration the ratio is 1.85, reported above
 - cover letter: v2 described the mac80211 weight change as posted
   separately while it was not; it is posted alongside this version
 - patch 1: state that the AQL half of the rationale needs the feature
   bit patch 2 sets, so the stable backport is not read as describing a
   stall that cannot occur without it
 - patch 2: correct "nothing restarts it until the next frame arrives
   from the network stack"; a round also starts on a block ack session
   change, a station leaving powersave, and a stopped queue restarting
 - patch 2: drop "cannot behave worse", which the same message's note
   about a stale rate contradicts
 - patch 2: say why pending airtime stands above the limit it is tested
   against
 - patch 3: the idr holds one entry short of DP_TX_IDR_SIZE
 - patch 4: drop the assertion about firmware TID markers that no ath11k
   header defines
 - patch 5: correct the claim that only a frame from the network stack
   can start a round, and the claim that mt76 relies on bottom halves
   being disabled; mt76 runs every round from one per-device tx worker
 - patch 5: state the bound conditionally. A selection ends where the
   airtime queue limit binds and drains the station's queue where it does
   not, which is what the generic handler does today
 - patch 6: the comment no longer says nothing else starts a round
 - patch 8: -ENOMEM also covers a DMA mapping failure
 - v2: https://lore.kernel.org/all/[email protected]/
 - v1: https://lore.kernel.org/all/[email protected]/

[1] https://lore.kernel.org/all/[email protected]/
[2] https://lore.kernel.org/all/[email protected]/

Julius Bairaktaris (9):
  wifi: ath11k: free tx skbs through ieee80211_free_txskb()
  wifi: ath11k: enable airtime queue limits
  wifi: ath11k: report the pending tx MSDU count in soc_dp_stats
  wifi: ath11k: report tx airtime and enable airtime fairness
  wifi: ath11k: schedule TXQs from the driver
  wifi: ath11k: run the TXQ scheduler on tx completion
  wifi: ath11k: charge received airtime to the station deficit
  wifi: ath11k: stop a scheduling round when the hardware refuses a
    frame
  wifi: ath11k: budget the tx completion handler

 drivers/net/wireless/ath/ath11k/core.h    |  5 ++
 drivers/net/wireless/ath/ath11k/debugfs.c | 12 ++++
 drivers/net/wireless/ath/ath11k/dp.c      | 10 ++-
 drivers/net/wireless/ath/ath11k/dp_rx.c   | 40 +++++++++++
 drivers/net/wireless/ath/ath11k/dp_tx.c   | 26 +++++--
 drivers/net/wireless/ath/ath11k/dp_tx.h   |  3 +-
 drivers/net/wireless/ath/ath11k/mac.c     | 84 +++++++++++++++++++++--
 drivers/net/wireless/ath/ath11k/mac.h     |  1 +
 8 files changed, 168 insertions(+), 13 deletions(-)

-- 
2.53.0