Re: Performance issues with vnet jails + epair + bridge

On 2024-09-16 07:32, Miroslav Lachman wrote:
> On 15/09/2024 19:56, Sad Clouds wrote:
>> On Sun, 15 Sep 2024 18:01:07 +0100
>> Doug Rabson <[email protected]> wrote:
>> 
>>> I just did a throughput test with iperf3 client on a FreeBSD 14.1 host 
>>> with
>>> an intel 10GB nic connecting to an iperf3 server running in a vnet jail on
>>> a truenas host (13.something) also with an intel 10GB nic and I get full
>>> 10GB throughput in this setup. In the past, I had to disable LRO on the
>>> truenas host for this to work properly.
>>> 
>>> Doug.
>> 
>> Hello Doug, can you please confirm that you are NOT using if_epair(4)? I
>> imagine you dedicate one of the Intel 10Gb ports to a jail. This is not
>> an option for some of us, so a virtual NIC of some sort is the only
>> option with vnet jails. Other people also mentioned that vnet by itself
>> is not an issue and your test confirms this, however I'm observing poor
>> scalability specifically with the epair virtual NIC.
>> 
>> I will be trying netgraph when I have some more time. If there are
>> other alternatives to if_epair then I would be interested to learn
>> about them.
> 
> Try ngbuddy, it will help you with configuring netgraph.
> 
> Or you can create second loopback interface, for example lo1 and put your 
> jails on
> it. I don't know what your scenario is, but I used lo1 in the past with 
> private
> IPs (10.x.x.x or 172.16.x.x) for jails not facing the outside world, just
> communicating with the host)
> 
> You can put something similar to rc.conf:
> 
> cloned_interfaces="lo1"
> # interface where all jails will bind
> ifconfig_lo1="inet 172.16.55.22 netmask 255.255.255.0"

I was going to suggest something like this as well. I often employ this with
jails. You can also facilitate exchanging traffic with the outside via pf(4)

(host) rc.conf(5):
cloned_interfaces="lo1"
pf_enable="YES"
pflog_enable="YES"

(host) pf.conf:
EXT_ADDR="192.168.1.2"
set skip on { lo0, lo1 }
nat pass on wlan0 from { lo1 } to any -> $EXT_ADDR
rdr pass on wlan0 proto tcp from any to { lo1 } -> $EXT_ADDR

Exchanging the EXT_ADDR value with your hosts NIC address. I use
the loopback interface as 120.0.0.1-N as needed.

--Chris

> 
> Then run: service netif cloneup
> 
> Kind regards
> Miroslav Lachman

-- 
sent from a device written from and running on FreeBSD