Re: virtio_net failover and initramfs

On Tue, Aug 21, 2018 at 6:44 AM, Harald Hoyer <[email protected]> wrote:
> On 17.08.2018 21:09, Samudrala, Sridhar wrote:
>> On 8/17/2018 2:56 AM, Harald Hoyer wrote:
>>> On 17.08.2018 11:51, Harald Hoyer wrote:
>>>> On 16.08.2018 00:17, Siwei Liu wrote:
>>>>> On Wed, Aug 15, 2018 at 12:05 PM, Samudrala, Sridhar
>>>>> <[email protected]> wrote:
>>>>>> On 8/14/2018 5:03 PM, Siwei Liu wrote:
>>>>>>> Are we sure all userspace apps skip and ignore slave interfaces by
>>>>>>> just looking at "IFLA_MASTER" attribute?
>>>>>>>
>>>>>>> When STANDBY is enabled on virtio-net, a failover master interface
>>>>>>> will appear, which automatically enslaves the virtio device. But it is
>>>>>>> found out that iSCSI (or any network boot) cannot boot strap over the
>>>>>>> new failover interface together with a standby virtio (without any VF
>>>>>>> or PT device in place).
>>>>>>>
>>>>>>> Dracut (initramfs) ends up with timeout and dropping into emergency shell:
>>>>>>>
>>>>>>> [  228.170425] dracut-initqueue[377]: Warning: dracut-initqueue
>>>>>>> timeout - starting timeout scripts
>>>>>>> [  228.171788] dracut-initqueue[377]: Warning: Could not boot.
>>>>>>>            Starting Dracut Emergency Shell...
>>>>>>> Generating "/run/initramfs/rdsosreport.txt"
>>>>>>> Entering emergency mode. Exit the shell to continue.
>>>>>>> Type "journalctl" to view system logs.
>>>>>>> You might want to save "/run/initramfs/rdsosreport.txt" to a USB stick or
>>>>>>> /boot
>>>>>>> after mounting them and attach it to a bug report.
>>>>>>> dracut:/# ip l sh
>>>>>>> 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
>>>>>>> mode DEFAULT group default qlen 1000
>>>>>>>       link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
>>>>>>> 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue
>>>>>>> state UP mode DEFAULT group default qlen 1000
>>>>>>>       link/ether 9a:46:22:ae:33:54 brd ff:ff:ff:ff:ff:ff\
>>>>>>> 3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast
>>>>>>> master eth0 state UP mode DEFAULT group default qlen 1000
>>>>>>>       link/ether 9a:46:22:ae:33:54 brd ff:ff:ff:ff:ff:ff
>>>>>>> dracut:/#
>>>>>>>
>>>>>>> If changing dracut code to ignore eth1 (with IFLA_MASTER attr),
>>>>>>> network boot starts to work.
>>>>>>
>>>>>> Does dracut by default tries to use all the interfaces that are UP?
>>>>>>
>>>>> Yes. The specific dracut cmdline of our case is "ip=dhcp
>>>>> netroot=iscsi:... ", but it's not specific to iscsi boot. And because
>>>>> of same MAC address for failover and standby, while dracut tries to
>>>>> run DHCP on all interfaces that are up it eventually gets same route
>>>>> for each interface. Those conflict route entries kill off the network
>>>>> connection.
>>>>>
>>>>>>> The reason is that dracut has its own means to differentiate virtual
>>>>>>> interfaces for network boot: it does not look at IFLA_MASTER and
>>>>>>> ignores slave interfaces. Instead, users have to provide explicit
>>>>>>> option e.g. bond=eth0,eth1 in the boot line, then dracut would know
>>>>>>> the config and ignore the slave interfaces.
>>>>>>
>>>>>> Isn't it possible to specify the interface that should be used for network
>>>>>> boot?
>>>>> As I understand it, one can only specify interface name for running
>>>>> DHCP but not select interface for network boot.  We want DHCP to run
>>>>> on every NIC that is up (excluding the enslaved interfaces), and only
>>>>> one of them can get a route entry to the network boot server (ie.g.
>>>>> iSCSI target).
>>>>>
>>>>>>
>>>>>>> However, with automatic creation of failover interface that assumption
>>>>>>> is no longer true. Can we change dracut to ignore all slave interface
>>>>>>> by checking  IFLA_MASTER? I don't think so. It has a large impact to
>>>>>>> existing configs.
>>>>>>
>>>>>> What is the issue with checking for IFLA_MASTER? I guess this is used with
>>>>>> team/bonding setups.
>>>>> That should be discussed within and determined by the dracut
>>>>> community. But the current dracut code doesn't check IFLA_MASTER for
>>>>> team or bonding specifically. I guess this change might have broader
>>>>> impact to existing userspace that might be already relying on the
>>>>> current behaviour.
>>>>>
>>>>> Thanks,
>>>>> -Siwei
>>>> Is there a sysfs flag for IFF_SLAVE? Or any "ip" output I can use to detect, that it is a IFF_SLAVE?
>>>>
>>> Oh, it's the other way around.. dracut should ignore "master" (eth1).
>> In the above example eth0 is the net_failover device and eth1 is the lower virtio_net device.
>> "ip" output of eth1 shows "master eth0". It indicates that eth0 is its upper/master device.
>> This information can also be obtained via sysfs too. /sys/class/net/eth1/upper_eth0
>>>
>>> Can the master enslave the "eth0", if it is already "UP" and busy later on?
>> eth0 is the master/failover device and eth1 gets registered as its slave via NETDEV_REGISTER event.
>> dracut should ignore eth1 in this setup.
>
>
> Care to test, if that fixes your case?
> https://github.com/dracutdevs/dracut/pull/450/files

Sorry, I forgot to mention that some of our setups do not have
'ip=dhcp' explicitly specified, but are still able to boot from iSCSI
target using "netroot=iscsi:...". Move the iface_is_enslaved check out
from the parsing of ip= lines?

-Siwei