[Bug 280037] KTLS with Intel QAT may trigger kernel panics

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=280037

--- Comment #17 from ss3bsd <[email protected]> ---
Created attachment 256343
  --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=256343&action=edit
experimental patch that utilizes extra IRQ instances of QAT driver

I have created an experimental patch that utilizes extra IRQ instances (with a
queue for each) of QAT driver, which might improve the performance on high load
where driver stalls (EAGAIN) occur.

This patch introduces a sysctl entry to switch the behavior.

With the option enabled (dev.qat.0.use_extra_instances=1), 
 - I see fewer driver stalls in question
 - I could not actually confirm the throughput improvement because 10GbE is
saturated even with the option disabled


IRQ statistics shown in sysctl after some traffic:

<dev.qat.0.use_extra_instances: 0>

% sysctl -a | grep qat | grep irq

irq105: qat0:b0:217 @cpu0(domain0): 315445
irq106: qat0:b1:219 @cpu0(domain0): 197384
irq107: qat0:b2:221 @cpu0(domain0): 0
irq108: qat0:b3:223 @cpu0(domain0): 0
irq109: qat0:b4:225 @cpu0(domain0): 0
irq110: qat0:b5:227 @cpu0(domain0): 0
irq111: qat0:b6:229 @cpu0(domain0): 0
irq112: qat0:b7:231 @cpu0(domain0): 0
irq113: qat0:b8:233 @cpu0(domain0): 0
irq114: qat0:b9:235 @cpu0(domain0): 0
irq115: qat0:b10:237 @cpu0(domain0): 0
irq116: qat0:b11:239 @cpu0(domain0): 0
irq117: qat0:b12:241 @cpu0(domain0): 0
irq118: qat0:b13:243 @cpu0(domain0): 0
irq119: qat0:b14:245 @cpu0(domain0): 0
irq120: qat0:b15:247 @cpu0(domain0): 0
irq121: qat0:ae:249 @cpu0(domain0): 0


<dev.qat.0.use_extra_instances: 1>

% sysctl -a | grep qat | grep irq
irq105: qat0:b0:217 @cpu0(domain0): 117087
irq106: qat0:b1:219 @cpu1(domain0): 303001
irq107: qat0:b2:221 @cpu2(domain0): 631575
irq108: qat0:b3:223 @cpu3(domain0): 536653
irq109: qat0:b4:225 @cpu4(domain0): 534724
irq110: qat0:b5:227 @cpu5(domain0): 697166
irq111: qat0:b6:229 @cpu6(domain0): 708141
irq112: qat0:b7:231 @cpu7(domain0): 726079
irq113: qat0:b8:233 @cpu0(domain0): 0
irq114: qat0:b9:235 @cpu1(domain0): 0
irq115: qat0:b10:237 @cpu2(domain0): 0
irq116: qat0:b11:239 @cpu3(domain0): 0
irq117: qat0:b12:241 @cpu4(domain0): 0
irq118: qat0:b13:243 @cpu5(domain0): 0
irq119: qat0:b14:245 @cpu6(domain0): 0
irq120: qat0:b15:247 @cpu7(domain0): 0
irq121: qat0:ae:249 @cpu0(domain0): 0


Related topic found:
https://forums.freebsd.org/threads/qat-engines-and-cryptodev-usage-underutilised-card.95415/

Note that how many engines used (discussed at the topic) is perhaps irrelevant
to how many IRQs used.

-- 
You are receiving this mail because:
You are the assignee for the bug.