Packet loss and data corruption

Hi
I have a curious problem with a firewall using 3c509 adapter. I tried
looking on the net for similar problem, but could not find any
information.

I have a firewall (P133 class machine) which originally had an old
10mbit ISA 3com adapter and a 3c905 Boomerang 100baseTx which worked
perfectly (sustained packet rate and stuff).

Then I replaced the ISA card with a new 3c905C Tornado card to get a
full 100mbit throughput. And strange things started to happen.


If I netcat some binary data across the firewall to the hosts lying on
both sides of the firewall I get strange hick-ups in the transfer and
the transfer slows down to a complete hang. It is like TCP/IP is trying
to compensate for 'nonexistent' packet loss by throttling down the
connection.

If I netcat data from firewall host and to the respective hosts by using
each of the network cards, then I get a nice sustained rate of which
should be available on a 100mbit network. That shows that the cards are
working properly by themselves and only fail while they are both active
by copying data across the firewall. They are not sharing an IRQ, so
that should not be a problem.

If I copy the data across the firewall using SSH client, then I get a
bit better performance then with netcat (probably because of the was SSH
controls TCP stream), but still hick-ups and inevitable halt after a
couple of megabytes. Sometimes the SSH client quits with an 'Invalid MAC
address' error.


Here comes the curious part which I discovered by accident. If I
uuencode the binary data first, before piping it through the firewall,
or only send pure text data across the firewall, then I get a sustained
transfer rate which should be on a 100mbit network. So it is like the
firewall is choking on a 7-bit unclean data?

There are no 'hairy' content filtering on the firewall - everything is
kept at the transport layer and the firewall should be indifferent to
the packet payload. I even tried clearing of all iptable rules and
running with pure forwarding - same problem.

The strange thing is that I later discovered that some parts of the data
which I transfered at full speed using uuencode were corrupted (verified
by md5sum). The corruption somehow was undetected by TCP/IP's
checksuming? To verify my assumption I tried transferring gzip'ed data
across the firewall and surely it introduced CRC errors in the gzip
file.

The above errors only occur when transferring large (over 50mb) amounts
of data across. Smaller files and protocols like SSH are working fine
probably because the packet loss is unnoticable.


After noticing that mii-diag.c program was reporting jabber on a 3c905
card when it was negotiated into 100baseTx mode I replaced it with a Tulip
chip based adapter and thought that the problem would disappear because
of a faulty network card, but it did not.

I tried a lot of different kernels - from the latest in 2.2.x series to
a couple kernels spread in 2.4 series and even the newest 2.6 kernel.
That probably rules out some subsystem of the Linux kernel being faulty.

Can it be the PCI-bus or some other hardware fault?
I can provide more information and diagnostics runs if you request it...

Here is a diagram of my setup.

  /-----> [mini 5 port 100mbit switch] <-------> [3c509C] workstation
  |
  |
  [3c509C]
  firewall with a problem
  [Tulip]
  |
  |
  \---> [24 port 100mbit switch] <------> [eepro100] server


Thanks in advance
Valery

ps: please CC to me because I am not on the list.
_______________________________________________
vortex mailing list
[email protected]
http://www.scyld.com/mailman/listinfo/vortex