Re: An interesting intermittent problem

Hello all,

    Well, this has just not been my week for using eepro100's... I spent 
several days earlier this week tracking down an intermittent problem with 
one of my Penguin Computing servers which uses an Intel eepro100... I 
believe it's problem turned out to be sleep mode on the ethernet chipset... 
Turned it off, and it seems to be ok now...

    However, I'm trying to put one of my VALinux machines (with an intel 
eepro100) back into service and am running into an intermittent problem... 
It may or may not be related to the NIC, but from what I can gather after 
several days of chasing it down, it seems to be the NIC...

    Basically, this server works fine... For a while. Then all of the 
sudden it will cease talking to anything else EXCEPT computers on it's 
local subnet/segment. If I logon to a box connected to the same switch as 
the server in question and ping it with an ICMP packet, all of the sudden 
everything is fine and dandy again... For a while.

    It seems to occur when the box is sitting idle -- the longer it is not 
receiving traffic, the more of a chance that it will cease to respond to 
network traffic outside of the local subnet...

    My first thought was that there may be something conflicting with it's 
IP -- I've seen that cause all sorts of goofiness. I did an arp scan of my 
network and the only machine that responded to the arp request for that IP 
was the server that I expected to respond. Only one answer... So, it doesnt 
seem to be an IP conflict problem.

    My next guess was a routing issue, so I started doing traceroutes and 
such... All the traceroutes showed what I already knew -- the server works 
fine, until it doesnt work fine anymore, at which time it quits answering 
requests from outside the local network. Nothing surprising there...

    I turned on packet debugging on my Cisco router and packets ARE being 
routed through the router to the local subnet when the machine is not 
answering...

    I verified this by doing packet dumps on the local LAN and grabbing the 
traffic destined to or from the machine in question. I can indeed see it 
reaching the switch on the local side of the router (and the monitor port 
on the switch which is where I was watching from)...

    Packet dumps show, for instance, the ping requests coming from the 
originating box, going through the router, and coming onto the local 
subnet, yet the machine in question never replies (unless of course it is 
in its working stage)...

    I used eepro100-diag to verify that sleep mode was not enabled on this 
server -- it was not. It also reports all the checksums as being correct... 
Here's the output:

eepro100-diag.c:v2.12 4/15/2003 Donald Becker ([email protected])
  http://www.scyld.com/diag/index.html
Index #1: Found a Intel i82557/8/9 EtherExpressPro100 adapter at 0x2800.
i82557 chip registers at 0x2800:
   00000050 379b6578 00000000 00080002 183f0000 000005f0
   No interrupt sources are pending.
    The transmit unit state is 'Suspended'.
    The receive unit state is 'Ready'.
   This status is normal for an activated but idle interface.
EEPROM contents, size 64x16:
     00: d000 89b7 bf46 0c03 0003 0201 4701 0000  ____F________G__
   0x08: 0000 0000 40a0 3000 8086 0000 0000 0000  _____@_0________
       ...
   0x38: 0000 0000 0000 0000 0000 0000 0000 5b8f  _______________[
  The EEPROM checksum is correct.
Intel EtherExpress Pro 10/100 EEPROM contents:
   Station address 00:D0:B7:89:46:BF.
   Board assembly 000000-000, Physical connectors present: RJ45
   Primary interface chip i82555 PHY #1.
Primary transceiver is MII PHY #1. MII PHY #1 transceiver registers:
    3000 782d 02a8 0154 05e1 0021 0000 0000
    0000 0000 0000 0000 0000 0000 0000 0000
    0400 0000 0001 0000 0000 0000 0000 0000
    0000 0000 0000 0000 0000 0000 0000 0000.

    I have tried both the Becker driver, and the Intel driver with the same 
result. Everything works fine for a random amount of time, then it ceases 
to respond and the only medicine that cures it is a ping or other traffic 
from the local subnet...

    I cleared the ARP cache on the switch, and all our routers yesterday, 
and that didnt help anything. I also checked the arp tables on all the 
routers and servers -- they show the correct entries.

    The real kicker is that none of the syslogs or dmesg on the machine in 
question show ANYTHING unusual at all... In fact, no messages about the 
ethernet interface at all after the initial detection at boot and such.

    I'm at a loss.... I could use any advice that anyone has.

Thanks in advance,
Brad