multiple UNDI NICs in VMWare ESX 3.5 cause gPXE DHCP to fail on 1st NIC

"Bill Lortz" <[email protected]>
Newsgroups gmane.network.etherboot.user
Message-ID <[email protected]> 
I am using ESX 3.5 and have successfully used gPXE to iSCSI boot Linux
and Windows Guests using E1000 and PCNET.   But, with VMWare moving
towards the enhanced VMNET, I'd like to be able to configure guests with
the enhanced VMXNet NICs.    I'm trying to get gPXE to work with the
UNDI drivers.

 

If I configure the guest with only 1 NIC, gPXE seems to work just fine.
It is able to use DHCP to acquire an address and then load an http
script file that was specified in the dhcp options.   But, if I add a
2nd NIC to the guest, the gPXE is unable to successfully use the
original NIC although it seems to be able to use the 2nd NIC and acquire
an address.

 

With the two NICS, gPXE does recognize there are two NICS and names them
net0 and net1.   But, even though gPXE thinks it is sending DHCP
requests out on net0, a WireShark shows no traffic is actually leaving
the NIC's MAC Address.   Eventually it times out and tries a DHCP
session on the 2nd NIC and is successful with the 2nd NIC.

 

While I could use a workaround and switch my iSCSI to the 2nd NIC, I am
uncomfortable that it won't break in the future.    Also, I don't know
what will happen if I need to add a 3rd NIC later.   Windows is very
touchy in the iSCSI boot process and doesn't want the iSCSI nic to move
to another PCI slot.  Moving it requires re-attaching the iSCSI volume
locally and making configuration changes in Windows and moving the local
volume back to iSCSI and rebooting.   This is a very time-consuming
process.

 

So, I would prefer to help debug what is going on in gPXE.

 

I'd like to know if this is a known restriction and not a bug.   If it
is a bug, I'd like to help try to fix it.   If it is a known
restriction, I'd like advice on the best way of approaching this.

 

I'm using the top of the tree build with source that I copied today.
I've turned on various debug commands and am attaching the console
output under both scenarios (1 NIC and then 2 NICs).

 

In the 2 NIC output, you'll notice that gPXE sees both NICs, and even
thinks it is sending data out of the 1st NIC.   But, wireshark shows no
traffic is leaving that NIC's MAC address, and the console output sees
no traffic coming in.   Eventually, it gives up and switches to the
other NIC which works fine (but isn't setup in DHCP yet with gPXE
scripts and iSCSI parameters).

 

Thanks in advance for any help...

 

Bill

 

-----------------------------------------------------------

 

The console output for a 1 NIC boot sequence looks like (I've truncated
it once the IP Address is acquired and it starts receiving traffic):

 

canning for PXE expansion ROMs

[31mUNDIROM 0x18814 trying expansion ROM at c000:0000 (32kB)

[0m[31mUNDIROM 0x18814 has no PXE ROM ID

[0m[31mUNDIROM 0x18814 trying expansion ROM at c800:0000 (4kB)

[0m[31mUNDIROM 0x18814 has PXE ROM ID at c800:0020

[0m[31mUNDIROM 0x18814 has UNDI loader at c800:0ba5 (code 0a9f data
2600)

[0m[31mUNDIROM 0x18814 has PCI expansion header at c800:0040

[0m[31mUNDIROM 0x18814 is for PCI devices 15ad:0720

[0m[31mUNDIROM 0x18814 registered

[0m[32mUNDIROM 0x18844 trying expansion ROM at dc00:0000 (16kB)

[0m[32mUNDIROM 0x18844 has no PXE ROM ID

[0m[32mUNDIROM 0x18844 trying expansion ROM at e000:0000 (16kB)

[0m[32mUNDIROM 0x18844 PXE ROM ID outside ROM

[0m[31mUNDIROM 0x18814 matched PCI 15ad:0720 (00000000)

[0m[33mUNDI 0x187a4 loading UNDI ROM 0x18814 to CS 9b56 DS 98f6 for
[0m[33mPCI 00:10.0

[0m[33mUNDI 0x187a4 loaded PXENV+ 9b56:00d0 !PXE 9b56:0070 entry
9b56:0106

[0m[33mUNDI 0x187a4 using [611,624) kB of base memory

[0m[34mUNDINIC 0x18924 using UNDI 0x187a4

[0m[34mUNDINIC 0x18924 is 00:50:56:bd:27:65 on IRQ 9

[0m[34mUNDINIC 0x18924 has type DIX+802.3, speed 10000000, flags
0000cc1b

[0m[32mNETDEV 0x18844 registered as net0 (phys UNDI-PCI00:10.0 hwaddr
00:50:56:bd:27:65)

[0m[34mUNDINIC 0x18924 added

[0m[0m

 

 

[1mgPXE 0.9.7+[0m -- Open Source Boot Firmware --
[36mhttp://etherboot.org[0m

Features: HTTP DNS TFTP AoE iSCSI bzImage COMBOOT ELF Multiboot PXE
PXEXT

 

Press Ctrl-B for the gPXE command line...

                                         

[32mNETDEV 0x18844 opening

[0m[34mUNDINIC 0x18924 opened

[0mnet0: 00:50:56:bd:27:65 on UNDI-PCI00:10.0 (open)

  [Link:up, TX:0 TXE:0 RX:0 RXE:0]

DHCP (net0 00:50:56:bd:27:65)[35mUDP 0x189d4 allocated

[0m[35mUDP 0x189d4 bound to port 68

[0m[36mDHCP 0x18934 entering discovery state

[0m.[36mDHCP 0x18934 DHCPDISCOVER

[0m[35mUDP 0x189d4 TX 68->67 len 395

[0m[32mNETDEV 0x18844 received 0x199e8 (0x19980+66)

[0m[32mNETDEV 0x18844 received 0x1a1e8 (0x1a180+66)

[0m[32mNETDEV 0x18844 received 0x1a9c0 (0x1a980+3c)

[0m[32mNETDEV 0x18844 received 0x1b1c0 (0x1b180+3c)

[0m[32mNETDEV 0x18844 transmitting 0x193f0 (0x1919e+1ad)

[0m[32mNETDEV 0x18844 transmission 0x193f0 complete

[0m[32mNETDEV 0x18844 received 0x19334 (0x19180+1b3)

[0m[32mNETDEV 0x18844 processing 0x199e8 (0x19980+66)

[0m[32mNETDEV 0x18844 processing 0x1a1e8 (0x1a180+66)

[0m[32mNETDEV 0x18844 processing 0x1a9c0 (0x1a980+3c)

[0m[32mNETDEV 0x18844 processing 0x1b1c0 (0x1b180+3c)

[0m[32mNETDEV 0x18844 processing 0x19334 (0x19180+1b3)

[0m[35mUDP 0x189d4 RX 68<-67 len 401

[0m[36mDHCP 0x18934 DHCPOFFER from 172.29.33.1:67[0m[36m for
172.29.33.250[0m[36m

[0m[32mNETDEV 0x18844 received 0x191c0 (0x19180+3c)

[0m[32mNETDEV 0x18844 processing 0x191c0 (0x19180+3c)

[0m[32mNETDEV 0x18844 received 0x191c0 (0x19180+3c)

[0m[32mNETDEV 0x18844 received 0x199c0 (0x19980+3c)

[0m[32mNETDEV 0x18844 received 0x1a1d8 (0x1a180+55)

[0m[32mNETDEV 0x18844 received 0x1a9d8 (0x1a980+55)

[0m[32mNETDEV 0x18844 processing 0x191c0 (0x19180+3c)

[0m[32mNETDEV 0x18844 processing 0x199c0 (0x19980+3c)

[0m[32mNETDEV 0x18844 processing 0x1a1d8 (0x1a180+55)

[0m[32mNETDEV 0x18844 unknown network protocol 0055

[0m[32mNETDEV 0x18844 processing 0x1a9d8 (0x1a980+55)

[0m[32mNETDEV 0x18844 unknown network protocol 0055

[0m[32mNETDEV 0x18844 received 0x191e8 (0x19180+66)

[0m[32mNETDEV 0x18844 received 0x199e8 (0x19980+66)

[0m[32mNETDEV 0x18844 received 0x1a5b8 (0x1a180+436)

[0m[32mNETDEV 0x18844 received 0x1a9e8 (0x1a980+66)

[0m[32mNETDEV 0x18844 processing 0x191e8 (0x19180+66)

[0m[32mNETDEV 0x18844 processing 0x199e8 (0x19980+66)

[0m[32mNETDEV 0x18844 processing 0x1a5b8 (0x1a180+436)

[0m[32mNETDEV 0x18844 processing 0x1a9e8 (0x1a980+66)

[0m[32mNETDEV 0x18844 received 0x19214 (0x19180+92)

[0m[32mNETDEV 0x18844 processing 0x19214 (0x19180+92)

[0m[32mNETDEV 0x18844 unknown network protocol 0084

[0m[32mNETDEV 0x18844 received 0x191e8 (0x19180+66)

[0m[32mNETDEV 0x18844 received 0x199e8 (0x19980+66)

[0m[32mNETDEV 0x18844 received 0x1a73c (0x1a180+5ba)

 

 

 

The console output for a 2 NIC boot looks like:

 

Scanning for PXE expansion ROMs

[31mUNDIROM 0x18814 trying expansion ROM at c000:0000 (32kB)

[0m[31mUNDIROM 0x18814 has no PXE ROM ID

[0m[31mUNDIROM 0x18814 trying expansion ROM at c800:0000 (4kB)

[0m[31mUNDIROM 0x18814 has PXE ROM ID at c800:0020

[0m[31mUNDIROM 0x18814 has UNDI loader at c800:0ba5 (code 0a9f data
2600)

[0m[31mUNDIROM 0x18814 has PCI expansion header at c800:0040

[0m[31mUNDIROM 0x18814 is for PCI devices 15ad:0720

[0m[31mUNDIROM 0x18814 registered

[0m[32mUNDIROM 0x18844 trying expansion ROM at c900:0000 (4kB)

[0m[32mUNDIROM 0x18844 has PXE ROM ID at c900:0020

[0m[32mUNDIROM 0x18844 has UNDI loader at c900:0ba5 (code 0a9f data
2600)

[0m[32mUNDIROM 0x18844 has PCI expansion header at c900:0040

[0m[32mUNDIROM 0x18844 is for PCI devices 15ad:0720

[0m[32mUNDIROM 0x18844 registered

[0m[33mUNDIROM 0x18874 trying expansion ROM at dc00:0000 (16kB)

[0m[33mUNDIROM 0x18874 has no PXE ROM ID

[0m[33mUNDIROM 0x18874 trying expansion ROM at e000:0000 (16kB)

[0m[33mUNDIROM 0x18874 PXE ROM ID outside ROM

[0m[32mUNDIROM 0x18844 matched PCI 15ad:0720 (00000000)

[0m[34mUNDI 0x187a4 loading UNDI ROM 0x18844 to CS 9b56 DS 98f6 for
[0m[34mPCI 00:10.0

[0m[34mUNDI 0x187a4 loaded PXENV+ 9b56:00d0 !PXE 9b56:0070 entry
9b56:0106

[0m[34mUNDI 0x187a4 using [611,624) kB of base memory

[0m[35mUNDINIC 0x18954 using UNDI 0x187a4

[0m[35mUNDINIC 0x18954 is 00:50:56:bd:27:65 on IRQ 9

[0m[35mUNDINIC 0x18954 has type DIX+802.3, speed 10000000, flags
0000cc1b

[0m[33mNETDEV 0x18874 registered as net0 (phys UNDI-PCI00:10.0 hwaddr
00:50:56:bd:27:65)

[0m[35mUNDINIC 0x18954 added

[0m[32mUNDIROM 0x18844 matched PCI 15ad:0720 (00000000)

[0m[36mUNDI 0x189d4 loading UNDI ROM 0x18844 to CS 9816 DS 95b6 for
[0m[36mPCI 00:11.0

[0m[36mUNDI 0x189d4 loaded PXENV+ 9816:00d0 !PXE 9816:0070 entry
9816:0106

[0m[36mUNDI 0x189d4 using [598,611) kB of base memory

[0m[31mUNDINIC 0x18b24 using UNDI 0x189d4

[0m[31mUNDINIC 0x18b24 is 00:50:56:bd:57:32 on IRQ 11

[0m[31mUNDINIC 0x18b24 has type DIX+802.3, speed 10000000, flags
0000cc1b

[0m[34mNETDEV 0x18a44 registered as net1 (phys UNDI-PCI00:11.0 hwaddr
00:50:56:bd:57:32)

[0m[31mUNDINIC 0x18b24 added

[0m[0m

 

 

[1mgPXE 0.9.7+[0m -- Open Source Boot Firmware --
[36mhttp://etherboot.org[0m

Features: HTTP DNS TFTP AoE iSCSI bzImage COMBOOT ELF Multiboot PXE
PXEXT

 

Press Ctrl-B for the gPXE command line...

                                         

[33mNETDEV 0x18874 opening

[0m[35mUNDINIC 0x18954 opened

[0mnet0: 00:50:56:bd:27:65 on UNDI-PCI00:10.0 (open)

  [Link:up, TX:0 TXE:0 RX:0 RXE:0]

DHCP (net0 00:50:56:bd:27:65)[32mUDP 0x18bd4 allocated

[0m[32mUDP 0x18bd4 bound to port 68

[0m[36mDHCP 0x18b34 entering discovery state

[0m.[36mDHCP 0x18b34 DHCPDISCOVER

[0m[32mUDP 0x18bd4 TX 68->67 len 395

[0m[33mNETDEV 0x18874 transmitting 0x193f0 (0x1919e+1ad)

[0m[33mNETDEV 0x18874 transmission 0x193f0 complete

[0m.[36mDHCP 0x18b34 DHCPDISCOVER

[0m[32mUDP 0x18bd4 TX 68->67 len 395

[0m[33mNETDEV 0x18874 transmitting 0x193f0 (0x1919e+1ad)

[0m[33mNETDEV 0x18874 transmission 0x193f0 complete

[0m.[36mDHCP 0x18b34 DHCPDISCOVER

[0m[32mUDP 0x18bd4 TX 68->67 len 395

[0m[33mNETDEV 0x18874 transmitting 0x193f0 (0x1919e+1ad)

[0m[33mNETDEV 0x18874 transmission 0x193f0 complete

[0m....[36mDHCP 0x18b34 DHCPDISCOVER

[0m[32mUDP 0x18bd4 TX 68->67 len 395

[0m[33mNETDEV 0x18874 transmitting 0x193f0 (0x1919e+1ad)

[0m[33mNETDEV 0x18874 transmission 0x193f0 complete

[0m........[32mUDP 0x18bd4 closed

[0m Connection timed out (0x4c106035)

[33mNETDEV 0x18874 closing

[0m[35mUNDINIC 0x18954 closed

[0m[34mNETDEV 0x18a44 opening

[0m[31mUNDINIC 0x18b24 opened

[0mnet1: 00:50:56:bd:57:32 on UNDI-PCI00:11.0 (open)

  [Link:up, TX:0 TXE:0 RX:0 RXE:0]

DHCP (net1 00:50:56:bd:57:32)[32mUDP 0x18bd4 allocated

[0m[32mUDP 0x18bd4 bound to port 68

[0m[36mDHCP 0x18b34 entering discovery state

[0m.[36mDHCP 0x18b34 DHCPDISCOVER

[0m[32mUDP 0x18bd4 TX 68->67 len 395

[0m[34mNETDEV 0x18a44 transmitting 0x193f0 (0x1919e+1ad)

[0m[34mNETDEV 0x18a44 transmission 0x193f0 complete

[0m[34mNETDEV 0x18a44 received 0x192e4 (0x19180+162)

[0m[34mNETDEV 0x18a44 processing 0x192e4 (0x19180+162)

[0m[32mUDP 0x18bd4 RX 68<-67 len 320

[0m[36mDHCP 0x18b34 DHCPOFFER from 172.24.100.23:67[0m[36m for
172.24.105.95[0m[36m

[0m[34mNETDEV 0x18a44 received 0x191d8 (0x19180+55)

[0m[34mNETDEV 0x18a44 processing 0x191d8 (0x19180+55)

[0m[34mNETDEV 0x18a44 unknown network protocol 0055

[0m[34mNETDEV 0x18a44 received 0x191c0 (0x19180+3c)

[0m[34mNETDEV 0x18a44 received 0x199c0 (0x19980+3c)

[0m[34mNETDEV 0x18a44 processing 0x191c0 (0x19180+3c)

[0m.[36mDHCP 0x18b34 DHCPDISCOVER

[0m[32mUDP 0x18bd4 TX 68->67 len 395

[0m[34mNETDEV 0x18a44 transmitting 0x193f0 (0x1919e+1ad)

[0m[34mNETDEV 0x18a44 transmission 0x193f0 complete

[0m[34mNETDEV 0x18a44 received 0x192e4 (0x19180+162)

[0m[34mNETDEV 0x18a44 processing 0x199c0 (0x19980+3c)

[0m[34mNETDEV 0x18a44 processing 0x192e4 (0x19180+162)

[0m[32mUDP 0x18bd4 RX 68<-67 len 320

[0m[36mDHCP 0x18b34 DHCPOFFER from 172.24.100.23:67[0m[36m for
172.24.105.95[0m[36m

[0m[34mNETDEV 0x18a44 received 0x191c0 (0x19180+3c)

[0m[34mNETDEV 0x18a44 received 0x199c0 (0x19980+3c)

[0m[34mNETDEV 0x18a44 processing 0x191c0 (0x19180+3c)

[0m[34mNETDEV 0x18a44 processing 0x199c0 (0x19980+3c)

[0m.[34mNETDEV 0x18a44 received 0x191d8 (0x19180+55)

[0m[34mNETDEV 0x18a44 received 0x199d8 (0x19980+55)

[0m[34mNETDEV 0x18a44 processing 0x191d8 (0x19180+55)

[0m[34mNETDEV 0x18a44 unknown network protocol 0055

[0m.[36mDHCP 0x18b34 entering request state

[0m[34mNETDEV 0x18a44 processing 0x199d8 (0x19980+55)

[0m[34mNETDEV 0x18a44 unknown network protocol 0055

[0m[36mDHCP 0x18b34 DHCPREQUEST to 172.24.100.23:67[0m[36m for
172.24.105.95

[0m[32mUDP 0x18bd4 TX 68->67 len 407

[0m[34mNETDEV 0x18a44 transmitting 0x193f0 (0x1919e+1b9)

[0m[34mNETDEV 0x18a44 transmission 0x193f0 complete

[0m[34mNETDEV 0x18a44 received 0x192e4 (0x19180+162)

[0m[34mNETDEV 0x18a44 processing 0x192e4 (0x19180+162)

[0m[32mUDP 0x18bd4 RX 68<-67 len 320

[0m[36mDHCP 0x18b34 DHCPACK from 172.24.100.23:67[0m[36m for
172.24.105.95[0m[36m

[0m[32mUDP 0x18bd4 closed

[0m ok

net1: 172.24.105.95/255.255.0.0 gw 172.24.101.1

No filename or root path specified

No more network devices

 

Press Ctrl-B for the gPXE command line...

                                         

[34mNETDEV 0x18a44 closing

[0m[31mUNDINIC 0x18b24 closed

[0m[34mNETDEV 0x18a44 unregistered

[0m[31mUNDINIC 0x18b24 removed

[0m[33mUNDI 0x189d4 unloading

[0m[33mUNDI 0x189d4 freeing [598,611) kB of base memory

[0m[35mNETDEV 0x18874 unregistered

[0m

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day 
trial. Simplify your report design, integration and deployment - and focus on 
what you do best, core application coding. Discover what's new with 
Crystal Reports now.  http://p.sf.net/sfu/bobj-july

_______________________________________________
Etherboot-discuss mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/etherboot-discuss
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.