Re: [TEST] CX7 timeouts on reconfig w/ page pool failure injection
Nimrod Oren <[email protected]>
| Newsgroups | org.kernel.vger.netdev |
|---|---|
| Message-ID | <[email protected]> |
On 04/08/2026 19:45, Jakub Kicinski wrote: > Hi! > > The CX7 testing in NIPA is looking pretty good, one major source of > noise, tho, is that the pp-alloc test sometimes locks rtnl for > a minute. > > Here's a failure in that test itself: > > # # Exception| subprocess.TimeoutExpired: Command '['ethtool', '-G', 'ens25f1np1', 'rx', '2048']' timed out after 20 seconds > # # Exception| > # not ok 1 pp_alloc_fail.test_pp_alloc > > https://netdev.bots.linux.dev/logview.html?f=/logs/hwksft/CX7-dbg/results/763842/test-outputs/36-pp-alloc-fail-py/stdout > > Then the next test that runs also times out during setup: > > # subprocess.TimeoutExpired: Command '['ip', '--json', '-d', 'link', 'show', 'dev', 'ens25f1np1']' timed out after 20 seconds > not ok 1 selftests: drivers/net/hw: rss_api.py # exit=1 > > https://netdev.bots.linux.dev/logs/hwksft/CX7-dbg/results/763842/test-outputs/37-rss-api-py/stdout > > pp_alloc test can fail, but spilling into the next test is a problem. > Seems like the bigger culprit is the earlier xdp test, which leaves MTU at 9000: # # Defer Exception| net.lib.py.utils.CmdExitFailure: Command failed # # Defer Exception| CMD: ip link set dev ens25f1np1 mtu 1500 xdpdrv off # # Defer Exception| EXIT: -15 # # Defer Exception| # not ok 17 xdp.test_xdp_native_adjst_head_grow_data.ipv4 # # Totals: pass:14 fail:3 xfail:0 xpass:0 skip:0 error:0 not ok 1 selftests: drivers/net: xdp.py # TIMEOUT 360 seconds https://netdev.bots.linux.dev/logview.html?f=/logs/hwksft/CX7-dbg/results/763842/test-outputs/13-xdp-py/stdout I'll try to reproduce this internally and investigate what happened. Also, I noticed that the pp-alloc test isn't properly restoring the rx ring size due to the timeout. In the subsequent retry, it's attempting to double it again, this time from 2K to 4K: # # Exception| subprocess.TimeoutExpired: Command '['ethtool', '-G', 'ens25f1np1', 'rx', '4096']' timed out after 20 seconds # # Exception| # not ok 1 pp_alloc_fail.test_pp_alloc https://netdev.bots.linux.dev/logs/hwksft/CX7-dbg/results/763842/test-outputs/36-pp-alloc-fail-py-retry/stdout The high MTU and ring size along with the heavy debug kernel likely make the pp-alloc test significantly slower. I'll also prepare a patch to fix this pp-alloc cleanup issue. Thanks, Nimrod