Re: ARP interfering with Etherboot TFTP?

On Mar 7, 2006, at 8:55 AM, John Connett/ENG/UK/Plasmon wrote:
> Same hardware setup using the Broadcom NIC.  Linux server running
> atftpd (0.7-7).  Client running /usr/bin/tftp on FreeBSD.  Transferred
> a large file in binary mode using 'get'.  Trace output below.

Good debugging strategy.

> Timings suggest that the Linux ARP cache may be being flushed about
> every 30 seconds.

OK.

> Both ARP request/response pairs cleanly handled between TFTP Data
> Packet and Acknowledgement, with no retries.
> Looks like the hardware does the right thing when driven by suitable
> software.

It certainly sounds like a bug in Etherboot.  Have you tried the  
pulling down the version in CVS?
If it's a problem, I can make a tarball and put it somewhere for you.

Another thing you can do is to test with an old 10BASE-T hub to see  
if there's a speed issue.  I suspect not, but it's something I often  
do when exercising drivers.

It very well may be a problem with the driver for the card, since it  
appears other cards do not exhibit this problem on the same network  
using the same software and hardware infrastructure.

Would you log into SourceForge and add a bug to the tracker:

     https://sourceforge.net/tracker/?func=add&group_id=4233&atid=104233

That way people looking for something to debug can find it more  
easily, and it won't be forgotten in ML (mailing list) traffic.

The Broadcom driver is probably the problem, but this should be  
debuggable, since it's during protocol, and the rest of Etherboot  
seems intact.  The Gigabit drivers vary in complexity from easy to  
insane.  It's really astonishing how some companies make driver  
writing easier, and others make it harder.  I believe the Broadcom is  
one of the longer, more complicated ones.

I just checked the gigabit nics:

    $ wc -l e1000* forcedeth* r8169* tg3*

    3744 e1000.c
    2058 e1000_hw.h

    1436 forcedeth.c

    1205 r8169.c

    3398 tg3.c
    2211 tg3.h

As you can see, they vary from 1200 to 5500 lines.  That's pretty  
amazing, considering that some common 10/100 Nics are:

    $ wc -l natsemi.c 3c90x.c sis900.c eepro100.c
     773 natsemi.c
    1017 3c90x.c
    1271 sis900.c
     834 eepro100.c

The size of drivers (and buffers) for Gigabit NICs, and the desire to  
do more was what led to adding relocation to Etherboot, allowing us  
to run in high memory.

Now, there are a few strategies generally used to do Etherboot  
drivers.  One is to keep the Etherboot driver as close to the Linux  
Driver as possible.  This has the advantage of making the driver  
easier to maintain, in some ways.  You can source-compare with the  
Linux driver and see if you're close to what they are doing.

If you go this way, you can often get away with not worrying about a  
lot of details, because if it worked for Linux, it will probably work  
with Etherboot, modulo interrupt code, buffer sizes, and  
synchronization code (those pesky "sleep for a second to let the card  
reset" kinds of things).

Another way to go is take the Linux driver, and strip it down to bare  
essentials, creating an Etherboot driver that has only what is needed  
to do what Etherboot needs done.  I happen to prefer this approach,  
and have found that by knowing what version you used of the Linux  
driver, you can always source-compare with the current Linux version,  
and see what has changed.

Now, if you rewrite a Linux driver, you really have to understand  
what every line does.  You generally have to get the datasheet for  
the NIC controller chip to verify each and every bit pattern.  You  
have to understand exactly what is being done.  The downside is that  
the next person has to be able to debug the same way.  You can't  
easily just source-compare the driver with Linux and see some  
transposed characters or a new variant. You can source-compare Linux  
versions, though.

I have generally found, however, that both approaches work, to  
varying degrees.

Stripping Linux drivers down makes them easier to understand, and  
enforces the discipline of the driver writer that he/she must know  
what everything does.  It also means that the driver is shorter, and  
easier to read because there isn't all this interrupt code and kernel  
interface code that we don't use in Etherboot.

Keeping it close to the kernel means you only have to maintain one  
version (usually), and theoretically, fixing bugs in one version  
fixes them in the other.  (although the closer you get to the  
machine, the less I think this applies).

Since it's more likely to find people who can diff than people who  
can low-level debug, some would argue for staying close the Linux  
source.  On the other hand, some would argue that at this level, you  
really better know what all those port writes and bit-shifts are doing.

Truth (and what is appropriate), as always, wanders back and forth in  
between the extremes.

Anyway, I just felt like saying that.  It doesn't really fix your  
problem, but maybe it will get somebody curious enough to take a look  
at the Broadcom (tg3) driver.

I'm really sorry we don't have more active/available low-level  
hackers right now.  We'll have to work harder getting people  
interested in debugging and developing.  If you (or anyone else)  
would like to take a shot at debugging this, we'll help as we can.   
It's not as hard as it looks if you know C, and are willing to pore  
over Linux driver source code a bit.

Marty




-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642