Re: 13.1R problems on Pi3

On Mon, Jul 04, 2022 at 12:17:15PM -0400, Karl Denninger wrote:
> 
> On 7/4/2022 11:28, bob prohaska wrote:
> > On Sun, Jul 03, 2022 at 10:36:35PM -0400, Karl Denninger wrote:
> > 
> > Can any sense be made of the few ping responses obtained when ntp
> > is coming up? It's looks as if something happens after ntp runs
> > that blocks subsequent network traffic, but why starting an outbound
> > ping should partly unblock things is obscure to me.
> 
> Yes.?? The odds are reasonably high that there is confusion as to which MAC
> address maps to which device.?? This implies there's a loop between the two
> switches (e.g. there is more than one way for packets to get into and out of
> each said switch to the other) or the two devices are claiming the same MAC
> address and thus when each "speaks" and performs ARP it "grabs" the map
> which works until the next one pipes up and it grabs it.
>
 
Looks like that's the problem. There's only one cable between switches, but
here's what I get from ifconfig on each host:

On the machine running 13.1-R attached to switch 2:
bob@www:~ % ifconfig
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
	options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
	inet6 ::1 prefixlen 128
	inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
	inet 127.0.0.1 netmask 0xff000000
	groups: lo
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
ue0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=80009<RXCSUM,VLAN_MTU,LINKSTATE>
>>>>>>>	ether b8:27:eb:71:46:4e
	inet 50.1.20.28 netmask 0xffffff00 broadcast 50.1.20.255
	media: Ethernet autoselect (100baseTX <full-duplex>)
	status: active
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
bob@www:~ % hostname
www.zefox.org
bob@www:~ % 
bob@www:~ % uname -a
FreeBSD www.zefox.org 13.1-RELEASE FreeBSD 13.1-RELEASE releng/13.1-n250148-fc952ac2212 GENERIC arm64
bob@www:~ %

On the machine running an updated stable/13 system attached to switch 1: 
bob@pelorus:~ % ifconfig
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
	options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
	inet6 ::1 prefixlen 128
	inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
	inet 127.0.0.1 netmask 0xff000000
	groups: lo
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
ue0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=80009<RXCSUM,VLAN_MTU,LINKSTATE>
>>>>>>	ether b8:27:eb:71:46:4e
	inet 50.1.20.24 netmask 0xffffff00 broadcast 50.1.20.255
	media: Ethernet autoselect (100baseTX <full-duplex>)
	status: active
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
bob@pelorus:~ % hostname
pelorus.zefox.org
bob@pelorus:~ % 
bob@pelorus:~ % uname -a
FreeBSD pelorus.zefox.org 13.1-STABLE FreeBSD 13.1-STABLE #6 stable/13-n251601-2353343b324: Sun Jul  3 21:43:04 PDT 2022     [email protected]:/usr/obj/usr/src/arm64.aarch64/sys/GENERIC arm64


Thinking it over, I added the extra switch some time ago and didn't 
immediately notice any problems. Both Pi3s started out on the first
switch (NetGear), with no obvdious problems. Later I probably moved 
one Pi3 to the second switch (D-Link) and started to notice troubles. 
Does this story make sense? 
 
> Each interface device from the factory is supposed to have a unique MAC
> address.?? This can, for most interfaces, be overridden (modern Android
> phones "randomize" it if told to as a "security" measure) but for obvious
> reasons doing that can lead to problems. Collisions where multiple devices
> are using the same MAC will lead to exactly the sort of thing you're seeing
> because the switch is sending the packets to the wrong place.
> 
> I've got a decent number of Pis of everything back to the "2" here and most
> of the time several of them are on my network at once.?? I've not seen this
> problem but I wouldn't exclude that both are claiming the same MAC and, if
> so, that's what's causing the problem.
> 
[example ifconfig output snipped]
> 
> That MUST be unique on your LAN; the prefix (first three octets) is a vendor
> code /*and the last three should never be duplicated by a vendor. */If you
> are not setting it in /etc/rc.conf or elsewhere and there /are /duplicates
> then a very bad thing happened when those units were manufactured -- set one
> of them to something else.
>

Any pointers to MAC-setting methods appreciated.....

Thanks very much for writing!

bob prohaska