[Bug 286179] Passing a vlan interface to a vnet jail result in a stuck dying state

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=286179

--- Comment #27 from Zhenlei Huang <[email protected]> ---
(In reply to Peter Ganzhorn from comment #12)
> I think I am also affected by this bug and would like to share my observations and thoughts.
> My jail configuration file is very similar to Florian Millet's and I observe my
> vnet-enabled jails root path (zfs dataset) won't unmount after stopping the jail as
> well as the vnet network interface not being returned to the host.

> In my case I am not using a vlan interface, but a sr-iov subdevice of an Intel 
> XXV710-DA2 (iavf).

> However if I use a 'real' network interface (I have a spare Intel i210 which I used
> for testing) and just use the igb instead of the iavf interface, the jail stops as
>  expected, i.e. the root path can be unmounted and the igb interface is returned to
> the host.

> If it matters, I am running a AMD Ryzen 5700x on an AsRock X570 plus a Intel X710-

> The iavf interfaces fail to be returned to the host after stopping the jail, the igb
> interface is returned as expected.
The in-tree iavf(4) and igb(4) drivers are all iflib(4) based. There's also an
igb driver in ports. Can you please confirm which igb(4) driver are you using ?

> Since the iavf interfaces are virtual interfaces just like the vlan interfaces
> Florian Millet uses, my best uneducated guess is that virtual network interfaces
> behave differently than real interfaces when vnet-enabled jails are stopped.
No. iavf(4) is driver for (SR-IOV) Virtual Functions. The name of Virtual
Function is a bit misleading, but to the OS they are still treated as if they
are real hardwares.
You can consider vlan(4) interface as a pseudo device. It functions along with
other ethernet drivers, such as epair(4). (SR-IOV) Virtual Functions and
vlan(4) almost share nothing in the concept.

> Since I have absolutely no knowledge about how vnet jails are implemented nor where 
> to look for that in the sources, I would like to as the following questions as food 
> for further thoughts on this.

> Does the process of terminating a vnet-enabled jail involve resetting the network
> interface before returning it to the host?
A simplified procedure to remove a vnet-enabled jail ( let's ignore syscalls
and focus on the kernel part ),
 1. Kill all processes in the jail, and notify other components the jail is
dying
 2. Check if the ref count reach zero.
 3. If yes, then continue, do teardown process of the vnet.
 4. The vnet runs all VNET SYSUNINIT. The verify first one is return the loaned
interfaces to their home vnet. ( loaned via `ifconfig em0 vnet foo` or `jail
vnet.interface=em0,em1` )
 5. The return procedure involves detach ( from the dying vnet ) and attach (
to its home vnet ) operations and eventually you can retrieve them via
`ifconfig em0`.

> If that is true, is it possible that this process fails, hangs or returns 
> differently for virtual interfaces (like vlan or sr-iov interfaces) and real 
> interfaces?
A common case is the 3rd step, the ref count of jail does not reach zero, then
the teardown process of the vnet has no chances to run, hence the loaned
interfaces stick to the dying jail/vnet. The symptoms is `jls -d` shows the
removed jail keep in dying state for a long long time, and does not proceed.

Hope the above explain helps.

-- 
You are receiving this mail because:
You are on the CC list for the bug.