Re: issue while bridging VF on mellanox

Hi Benoit, one more thing, Im not sure whats the current support for v6, 
specially for ND.

Best regards.

Santi


On 3/17/25 16:48, Santiago Martinez wrote:
>
> Hi Benoit,
>
> I have a similar setup to yours (not using MLX4), and it works.
>
> I have two VMs acting as routers (R-A and R-B), each with two interfaces.
>
> On Router-A:
>
>   * VTNET1 → P2P to Router-B.
>   * VTNET0 → Sub-interface (VLAN 333) - Client-A.
>   * VXLAN0 → Towards Router-B.
>   * BR-TEST → VXLAN0 and VTNET0.333 as members.
>
> On Router-B:
>
>   * VTNET1 → P2P to Router-A.
>   * VTNET0 → Untagged - Client-B.
>   * VXLAN0 → Towards Router-A.
>   * BR-TEST → VXLAN0 and VTNET0 as members.
>
> Traffic passes.
>
> One thing to note—VXLAN0 will appear as UP/Active even if it’s not 
> reaching the other side.
>
> Best regards,
> Santi
>
>
> On 3/17/25 15:32, Benoit Chesneau wrote:
>> Hi all,
>>
>> I'm trying to make some kind of cross connect between a vlan and a vxlan, but as far as I  understand the mac address are not learnt from the vlan When I listen using tcpdump on the vlan i can see the arp request but not in the in the vxlan. ANy idea how should I fix this? Is there any other way to do a cross conect or shoud I go back to not use sriov when i need to build bridges?
>>
>>
>> Any idea is welcome.
>>
>> Benoit
>>
>> More details follow
>>
>> The vlan is created over a VF device in  a bhyve vm:
>>
>> vlan601: flags=1008943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1356
>>          options=18080000<LINKSTATE,TXTLS4,TXTLS6>
>>          ether 4a:75:f1:f5:a5:0a
>>          inet6 fe80::4875:f1ff:fef5:a50a%vlan601 prefixlen 64 scopeid 0x5
>>          groups: vlan
>>          vlan: 601 vlanproto: 802.1q vlanpcp: 0 parent interface: mce0
>>          media: Ethernet 25GBase-SR <full-duplex,rxpause,txpause>
>>          status: active
>>          nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
>>
>> The vxlan is
>>
>> vxlan0: flags=1008903<UP,BROADCAST,PROMISC,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1356
>>          options=80020<JUMBO_MTU,LINKSTATE>
>>          ether 58:9c:fc:10:12:01
>>          inet6 fe80::5a9c:fcff:fe10:1201%vxlan0 prefixlen 64 tentative scopeid 0x7
>>          groups: vxlan
>>          vxlan vni 400 local [2001:db8:1000::1]:4789 remote [2001:db8:1000::2]:4789
>>          media: Ethernet autoselect (autoselect <full-duplex>)
>>          status: active
>>          nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
>>
>>
>> And the bridge:
>>
>> # ifconfig bridge0
>> bridge0: flags=1028943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,PPROMISC,LOWER_UP> metric 0 mtu 1356
>>          options=0
>>          ether 58:9c:fc:10:ff:d3
>>          id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
>>          maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
>>          root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
>>          member: vlan601 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
>>                  ifmaxaddr 0 port 5 priority 128 path cost 800
>>          member: vxlan0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
>>                  ifmaxaddr 0 port 7 priority 128 path cost 55
>>          groups: bridge
>>          nd6 options=3<PERFORMNUD,ACCEPT_RTADV>
>>
>> The PF on the machine is a Mellanox [ConnectX-4 Lx
>>
>> #  pciconf -lvbce mlx5_core0
>> mlx5_core0@pci0:17:0:0:	class=0x020000 rev=0x00 hdr=0x00 vendor=0x15b3 device=0x1015 subvendor=0x1590 subdevice=0x00d4
>>      vendor     = 'Mellanox Technologies'
>>      device     = 'MT27710 Family [ConnectX-4 Lx]'
>>      class      = network
>>      subclass   = ethernet
>>      bar   [10] = type Prefetchable Memory, range 64, base 0xe4000000, size 33554432, enabled
>>      cap 10[60] = PCI-Express 2 endpoint max data 256(512) FLR RO NS
>>                   max read 4096
>>                   link x8(x8) speed 8.0(8.0) ASPM L1(L1)
>>      cap 03[48] = VPD
>>      cap 11[9c] = MSI-X supports 64 messages, enabled
>>                   Table in map 0x10[0x2000], PBA in map 0x10[0x3000]
>>      cap 09[c0] = vendor (length 24)
>>      cap 01[40] = powerspec 3  supports D0 D3  current D0
>>      ecap 0001[100] = AER 1 0 fatal 0 non-fatal 1 corrected
>>      ecap 000e[150] = ARI 1
>>      ecap 0010[180] = SR-IOV 1 IOV enabled, Memory Space enabled, ARI enabled
>>                       8 VFs configured out of 16 supported
>>                       First VF RID Offset 0x0002, VF RID Stride 0x0001
>>                       VF Device ID 0x1016
>>                       Page Sizes: 4096 (enabled), 8192, 16384, 32768, 65536, 131072, 262144, 524288, 1048576, 2097152, 4194304
>>      iov bar  [1a4] = type Prefetchable Memory, range 64, base 0xc7ffe000000, size 1048576, enabled
>>      ecap 0019[1c0] = PCIe Sec 1 lane errors 0
>>      ecap 000d[230] = ACS 1 Source Validation unavailable, Translation Blocking unavailable
>>                       P2P Req Redirect unavailable, P2P Cmpl Redirect unavailable
>>                       P2P Upstream Forwarding unavailable, P2P Egress Control unavailable
>>                       P2P Direct Translated unavailable, Enhanced Capability unavailable
>>    PCI-e errors = Correctable Error Detected
>>                   Unsupported Request Detected
>>       Corrected = Advisory Non-Fatal Error
>>
>> Other side is the same except that the card is an intel one:
>>
>> #  pciconf -lvbce ixl0
>> ixl0@pci0:23:0:0:	class=0x020000 rev=0x02 hdr=0x00 vendor=0x8086 device=0x1572 subvendor=0x15d9 subdevice=0x093b
>>      vendor     = 'Intel Corporation'
>>      device     = 'Ethernet Controller X710 for 10GbE SFP+'
>>      class      = network
>>      subclass   = ethernet
>>      bar   [10] = type Prefetchable Memory, range 64, base 0xc4800000, size 8388608, enabled
>>      bar   [1c] = type Prefetchable Memory, range 64, base 0xc5808000, size 32768, enabled
>>      cap 01[40] = powerspec 3  supports D0 D3  current D0
>>      cap 05[50] = MSI supports 1 message, 64 bit, vector masks
>>      cap 11[70] = MSI-X supports 129 messages, enabled
>>                   Table in map 0x1c[0x0], PBA in map 0x1c[0x1000]
>>      cap 10[a0] = PCI-Express 2 endpoint max data 256(2048) FLR RO
>>                   max read 512
>>                   link x8(x8) speed 8.0(8.0)
>>      cap 03[e0] = VPD
>>      ecap 0001[100] = AER 2 0 fatal 0 non-fatal 1 corrected
>>      ecap 0003[140] = Serial 1 b8f678ffff55c27c
>>      ecap 000e[150] = ARI 1
>>      ecap 0010[160] = SR-IOV 1 IOV enabled, Memory Space enabled, ARI enabled
>>                       16 VFs configured out of 64 supported
>>                       First VF RID Offset 0x0010, VF RID Stride 0x0001
>>                       VF Device ID 0x154c
>>                       Page Sizes: 4096 (enabled), 8192, 65536, 262144, 1048576, 4194304
>>      iov bar  [184] = type Prefetchable Memory, range 64, base 0xc5000000, size 65536, enabled
>>      iov bar  [190] = type Prefetchable Memory, range 64, base 0xc5100000, size 16384, enabled
>>      ecap 0017[1a0] = TPH Requester 1
>>      ecap 000d[1b0] = ACS 1 Source Validation unavailable, Translation Blocking unavailable
>>                       P2P Req Redirect unavailable, P2P Cmpl Redirect unavailable
>>                       P2P Upstream Forwarding unavailable, P2P Egress Control unavailable
>>                       P2P Direct Translated unavailable, Enhanced Capability unavailable
>>      ecap 0019[1d0] = PCIe Sec 1 lane errors 0
>>    PCI-e errors = Correctable Error Detected
>>                   Unsupported Request Detected
>>       Corrected = Advisory Non-Fatal Error