Re: Intermittent failure of routing/gateway with ix(4) (x86_64)

On Sat, Aug 23, 2025 at 9:16 PM R Tyler Croy <[email protected]> wrote:
>
> Howdy! I am hoping y'all could help me add more debugging to a FreeBSD 14.x based router/gateway. The device has two ix(4) NICs for the WAN/LAN. Periodically the device stops routing traffic entirely, but the machine is still online and interactive.
>
> What is particularly interesting is when I console into the machine I can still send traffic out from the device over the WAN interface, but anything that relies on the gateway seems to fail. I.e. routing to any host on the LAN interface (ix1) no longer works.
>
> `service routing restart` does nothing, `service netif restart` also does nothing. Downing the ix1 interface and bringing it back up does nothing.
>
> A reboot however does puts it back into working order.
>
>
> On one instance the console had a lot of arpresolve "cannot allocate llinfo" errors, chasing that rabbit led nowhere.
>
> Because this freezing behavior is intermittent but _does_ seem to correlate with higher network utilization, I'm wondering if there are debugging logs I can enable, or a way to inspect driver specific counters that could be overflowing, etc?
>
> I'm open to any recommendations that prevent me from setting up a cron every minute to validate connectivity or reboot the box! :)
>
>
> Cheers

I'd be interested in seeing your tcpdump and netstat -nr captures when
the problem is happening.
Might not turn up anything but _any_ information as scarce as it may
be indeed may be helpful.

I've got a couple of ixl devices that I beat the piss out of and
haven't had any issues:

ixl0: <Intel(R) Ethernet Controller X710 for 10GbE SFP+ - 2.3.3-k> mem
0xf3800000-0xf3ffffff,0xf4808000-0xf480ffff irq 24 at device 0.0 on
pci1
ixl0: fw 5.0.40043 api 1.5 nvm 5.05 etid 80002927 oem 1.261.0
ixl0: PF-ID[0]: VFs 64, MSI-X 129, VF MSI-X 5, QPs 768, I2C
ixl0: Using 1024 TX descriptors and 1024 RX descriptors
ixl0: Using 8 RX queues 8 TX queues
ixl0: Using MSI-X interrupts with 9 vectors
ixl0: Ethernet address: 3c:fd:fe:aa:df:a8
ixl0: Allocating 8 queues for PF LAN VSI; 8 queues active
ixl0: PCI Express Bus: Speed 8.0GT/s Width x4
ixl0: ixl_set_link: Error getting phy capabilities -7, aq error: 5
ixl0: SR-IOV ready
ixl0: netmap queues/slots: TX 8/1024, RX 8/1024
ixl1: <Intel(R) Ethernet Controller X710 for 10GbE SFP+ - 2.3.3-k> mem
0xf3000000-0xf37fffff,0xf4800000-0xf4807fff irq 24 at device 0.1 on
pci1
ixl1: fw 5.0.40043 api 1.5 nvm 5.05 etid 80002927 oem 1.261.0
ixl1: PF-ID[1]: VFs 64, MSI-X 129, VF MSI-X 5, QPs 768, I2C
ixl1: Using 1024 TX descriptors and 1024 RX descriptors
ixl1: Using 8 RX queues 8 TX queues
ixl1: Using MSI-X interrupts with 9 vectors
ixl1: Ethernet address: 3c:fd:fe:aa:df:a9
ixl1: Allocating 8 queues for PF LAN VSI; 8 queues active
ixl1: PCI Express Bus: Speed 8.0GT/s Width x4
ixl1: SR-IOV ready

# uname -a
FreeBSD nas.myhome 14.3-RELEASE FreeBSD 14.3-RELEASE
releng/14.3-n271432-8c9ce319fef7 GENERIC amd64

~Paul

-- 
__________________

:(){ :|:& };: