[PATCH v2 13/13] rcutorture: Make {,s}rcu_read_delay() better handle forward-progress testing

The rcu_read_delay() and srcu_read_delay() functions are suppose to
avoid (or at least to minimize) read-side delays during call_rcu()-based
forward-progress testing.  Although rcu_read_delay() does account for
this by testing rcu_fwd_cb_nodelay, it only does so in some cases, and
srcu_read_delay() does not bother at all.  And rcutorture testing does
occasionally turn up the MIN_FWD_CBS_LAUNDERED failure when running
forward-progress tests on Tree SRCU flavors.

This commit therefore makes both rcu_read_delay() and srcu_read_delay()
take an immediate exit if rcu_fwd_cb_nodelay is set.

Signed-off-by: Paul E. McKenney <[email protected]>
---
 kernel/rcu/rcutorture.c | 16 +++++++++++-----
 1 file changed, 11 insertions(+), 5 deletions(-)

diff --git a/kernel/rcu/rcutorture.c b/kernel/rcu/rcutorture.c
index 46d97055c2fcde..9c7b293d1cafbd 100644
--- a/kernel/rcu/rcutorture.c
+++ b/kernel/rcu/rcutorture.c
@@ -469,12 +469,14 @@ rcu_read_delay(struct torture_random_state *rrsp, struct rt_read_seg *rtrsp)
 	unsigned long longdelay_ms = 300;
 	unsigned long long ts;
 
-	/* We want a short delay sometimes to make a reader delay the grace
-	 * period, and we want a long delay occasionally to trigger
-	 * force_quiescent_state. */
+	// If there is a forward-progress test in flight, don't delay.
+	if (atomic_read(&rcu_fwd_cb_nodelay))
+		return;
 
-	if (!atomic_read(&rcu_fwd_cb_nodelay) &&
-	    !(torture_random(rrsp) % (nrealreaders * 2000 * longdelay_ms))) {
+	// We want a short delay sometimes to make a reader delay the grace
+	// period, and we want a long delay occasionally to trigger
+	// force_quiescent_state.
+	if (!(torture_random(rrsp) % (nrealreaders * 2000 * longdelay_ms))) {
 		started = cur_ops->get_gp_seq();
 		ts = rcu_trace_clock_local();
 		if ((preempt_count() & HARDIRQ_MASK) || softirq_count())
@@ -766,6 +768,10 @@ srcu_read_delay(struct torture_random_state *rrsp, struct rt_read_seg *rtrsp)
 	const long uspertick = 1000000 / HZ;
 	const long longdelay = 10;
 
+	// If there is a forward-progress test in flight, don't delay.
+	if (atomic_read(&rcu_fwd_cb_nodelay))
+		return;
+
 	// We want there to be long-running readers, but not all the time.
 	// The !rcu_preempt_depth() is for RCU Tasks Trace.
 
-- 
2.40.1