Bad network/file access performance

Hi,

I've been searching for the probably cause for a while now, and I
now give up.  This is my setup:

SERVER
	Intel P3 500MHz
	224MB RAM
	4GB IDE disk for / and swap
	3Com 3c905B-TX 10/100 Ethernet (rev. 0x30)
	Promise Ultra100TX2/ATA Bus Master IDE Accelerator (rev. 0x02)
		wd1 at pciide1 channel 0 drive 0: <IC35L080AVVA07-0>
		wd1(pciide1:0:0): using PIO mode 4, Ultra-DMA mode 5
		(Ultra/100) (using DMA data transfers)

		wd2 at pciide1 channel 0 drive 1: <IC35L100AVVA07-0>
		wd2(pciide1:0:1): using PIO mode 4, Ultra-DMA mode 5
		(Ultra/100) (using DMA data transfers)

		wd3 at pciide1 channel 1 drive 0: <Maxtor 6Y200P0>
		wd3(pciide1:1:0): using PIO mode 4, Ultra-DMA mode 5
		(Ultra/100) (using DMA data transfers)
	OS: NetBSD 1.6.1

CLIENT
	Intel P4 2600MHz
	1024MB RAM
	40GB IDE disk
	3Com 3c905B-TX 10/100, latest driver from 3Com
	OS: Windows XP PRO

Now here goes... as you've read in the Subject, I get really bad
performance out of this, check this out:


Send statistics (CLIENT -> SERVER)
   Send time :            00:00:34.582
   Bytes sent :           102400000  (97.66 Mbyte)
   Bits/second sent :     23688624  (23.69 Mbit/s)

^-- A *lot* of stalls and no stable transfer rate.


Receive statistics (SERVER -> CLIENT)
   Receive time :         00:00:11.368
   Bytes received :       102400000  (97.66 Mbyte)
   Bits/second received : 72061928  (72.06 Mbit/s)

^-- Stalls now and then, going from ~20Mbit/s to ~85Mbit/s


This is overall transfer rate, same with samba, ftp, etc.
This tests were made with "tptest", http://tptest.sourceforge.net/

IMHO, I really should get better performance here, especially
CLIENT -> SERVER.  I've tried these to solve it, nothing works:

	Completely switch NICs around (All 3c905 though)

	Changing cabling (including bypassing the local switch)

	BIOS upgrades on both client and server (you never know)

	Profiled 1.6.1 kernel

	GENERIC 1.6U kernel

Another test
This time using FTP to localhost

	From wd1a to wd0a: (Out of the Promise card)
	739975168 bytes received in 01:46 (6.59 MB/s)
	A lot of stalls and bad transfer rate.

	From wd1a to wd3a: (Inside the Promise card)
	739975168 bytes received in 00:59 (11.92 MB/s)
	No stalls, stable transfer rate.

Recieving on CLIENT from wd0 (not on Promise card) has
the same effects as the other disks which are on the Promise card.

Another test
Using time cp with a file size of 706 MB

	From wd1a to wd0a
	0.049u 34.779s 1:47.70 32.3%    0+0k 0+55io 4pf+0w

	From wd1a to wd3a
	0.070u 35.657s 0:44.63 80.0%    0+0k 41+78io 3pf+0w

Is it the Promise card failing? Any help would be great, pointers
to benchmarks etc, that might help me debug.  I've tried modifying
some kernel settings using sysctl, but nothing of that helped.


*Note*, I'm not subscribed to tech-perform@, please keep me in Cc:


Best regards,

-- 
Mattias Karlsson
[email protected]
SysAdm - http://www.sergei.cc/



DMESG:

NetBSD 1.6.1 (GENERIC) #0: Tue Apr  8 12:05:52 UTC 2003
 
[email protected]:/autobuild/netbsd-1-6/i386/OBJ/autobuild/netbsd-1-6/src/sys/arch/i386/compile/GENERIC
cpu0: Intel Pentium III (Katmai) (686-class), 501.16 MHz
cpu0: I-cache 16 KB 32b/line 4-way, D-cache 16 KB 32b/line 2-way
cpu0: L2 cache 512 KB 32b/line 4-way
cpu0: features 387f9ff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,SEP,MTRR>
cpu0: features 387f9ff<PGE,MCA,CMOV,FGPAT,PSE36,PN,MMX>
cpu0: features 387f9ff<FXSR,SSE>
cpu0: serial number 0000-0673-0001-8406-D609-4436
total memory = 223 MB
avail memory = 200 MB
using 2887 buffers containing 11548 KB of memory
BIOS32 rev. 0 found at 0xfb0a0
mainbus0 (root)
pci0 at mainbus0 bus 0: configuration mode 1
pci0: i/o space, memory space enabled, rd/line, rd/mult, wr/inv ok
pchb0 at pci0 dev 0 function 0
pchb0: VIA Technologies VT82C691 (Apollo Pro) Host-PCI (rev. 0x44)
agp0 at pchb0: aperture at 0xd0000000, size 0x10000000
ppb0 at pci0 dev 1 function 0: VIA Technologies VT82C598 (Apollo MVP3) 
CPU-AGP Bridge (rev. 0x00)
pci1 at ppb0 bus 1
pci1: i/o space, memory space enabled
vga1 at pci1 dev 0 function 0: Nvidia Corporation RIVA TNT2 Ultra (rev. 
0x11)
wsdisplay0 at vga1 kbdmux 1: console (80x25, vt100 emulation)
wsmux1: connecting to wsdisplay0
pcib0 at pci0 dev 7 function 0
pcib0: VIA Technologies VT82C596A (Apollo Pro) PCI-ISA Bridge (rev. 0x12)
pciide0 at pci0 dev 7 function 1: VIA Technologies VT82C596A (Apollo 
Pro) ATA66 controller
pciide0: bus-master DMA support present
pciide0: primary channel configured to compatibility mode
wd0 at pciide0 channel 0 drive 0: <ST34321A>
wd0: drive supports 32-sector PIO transfers, LBA addressing
wd0: 4103 MB, 8894 cyl, 15 head, 63 sec, 512 bytes/sect x 8404830 sectors
wd0: 32-bit data port
wd0: drive supports PIO mode 4, DMA mode 2, Ultra-DMA mode 2 (Ultra/33)
pciide0: primary channel interrupting at irq 14
wd0(pciide0:0:0): using PIO mode 4, Ultra-DMA mode 2 (Ultra/33) (using 
DMA data transfers)
pciide0: secondary channel configured to compatibility mode
atapibus0 at pciide0 channel 1: 2 targets
cd0 at atapibus0 drive 1: <NEC CD-ROM DRIVE:282, , 4.A2> type 5 cdrom 
removable
cd0: 32-bit data port
cd0: drive supports PIO mode 4, DMA mode 2, Ultra-DMA mode 2 (Ultra/33)
pciide0: secondary channel interrupting at irq 15
cd0(pciide0:1:1): using PIO mode 4, Ultra-DMA mode 2 (Ultra/33) (using 
DMA data transfers)
uhci0 at pci0 dev 7 function 2: VIA Technologies VT83C572 USB Controller 
(rev. 0x08)
uhci0: interrupting at irq 10
usb0 at uhci0: USB revision 1.0
uhub0 at usb0
uhub0: VIA Technologie UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub0: 2 ports with 2 removable, self powered
pchb1 at pci0 dev 7 function 3
pchb1: VIA Technologies product 0x3051 (rev. 0x20)
pciide1 at pci0 dev 9 function 0: Promise Ultra100TX2/ATA Bus Master IDE 
Accelerator (rev. 0x02)
pciide1: bus-master DMA support present
pciide1: primary channel configured to native-PCI mode
pciide1: using irq 11 for native-PCI interrupt
wd1 at pciide1 channel 0 drive 0: <IC35L080AVVA07-0>
wd1: drive supports 16-sector PIO transfers, LBA addressing
wd1: 78533 MB, 16383 cyl, 16 head, 63 sec, 512 bytes/sect x 160836480 
sectors
wd1: 32-bit data port
wd1: drive supports PIO mode 4, DMA mode 2, Ultra-DMA mode 5 (Ultra/100)
wd2 at pciide1 channel 0 drive 1: <IC35L100AVVA07-0>
wd2: drive supports 16-sector PIO transfers, LBA addressing
wd2: 98166 MB, 16383 cyl, 16 head, 63 sec, 512 bytes/sect x 201045600 
sectors
wd2: 32-bit data port
wd2: drive supports PIO mode 4, DMA mode 2, Ultra-DMA mode 5 (Ultra/100)
wd1(pciide1:0:0): using PIO mode 4, Ultra-DMA mode 5 (Ultra/100) (using 
DMA data transfers)
wd2(pciide1:0:1): using PIO mode 4, Ultra-DMA mode 5 (Ultra/100) (using 
DMA data transfers)
pciide1: secondary channel configured to native-PCI mode
wd3 at pciide1 channel 1 drive 0: <Maxtor 6Y200P0>
wd3: drive supports 16-sector PIO transfers, LBA48 addressing
wd3: 189 GB, 16383 cyl, 16 head, 63 sec, 512 bytes/sect x 398297088 sectors
wd3: 32-bit data port
wd3: drive supports PIO mode 4, DMA mode 2, Ultra-DMA mode 6 (Ultra/133)
wd3(pciide1:1:0): using PIO mode 4, Ultra-DMA mode 5 (Ultra/100) (using 
DMA data transfers)
ex0 at pci0 dev 15 function 0: 3Com 3c905B-TX 10/100 Ethernet (rev. 0x30)
ex0: interrupting at irq 12
ex0: MAC address 00:01:02:ab:3f:87
exphy0 at ex0 phy 24: 3Com internal media interface
exphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
isa0 at pcib0
3COM 3C509B Ethernet card in PnP mode
com0 at isa0 port 0x3f8-0x3ff irq 4: ns16550a, working fifo
com1 at isa0 port 0x2f8-0x2ff irq 3: ns16550a, working fifo
pckbc0 at isa0 port 0x60-0x64
pckbd0 at pckbc0 (kbd slot)
pckbc0: using irq 1 for kbd slot
wskbd0 at pckbd0: console keyboard, using wsdisplay0
lpt0 at isa0 port 0x378-0x37b irq 7
pcppi0 at isa0 port 0x61
midi0 at pcppi0: PC speaker
sysbeep0 at pcppi0
isapnp0 at isa0 port 0x279: ISA Plug 'n Play device support
npx0 at isa0 port 0xf0-0xff: using exception 16
fdc0 at isa0 port 0x3f0-0x3f7 irq 6 drq 2
isapnp0: read port 0x203
ep0 at isapnp0 port 0x210/16 irq 5
ep0: 3Com 3C509B EtherLink III
ep0: address 00:20:af:c9:ad:28, 8KB byte-wide FIFO, 1:1 Rx:Tx split
ep0: 10baseT, 10base5, 10base2 (default 10baseT)
biomask ef45 netmask ff65 ttymask ffe7
Kernelized RAIDframe activated
boot device: wd0
root on wd0a dumps on wd0b
root file system type: ffs
IP Filter: v3.4.29 initialized.  Default = pass all, Logging = enabled
wsdisplay0: screen 1 added (80x25, vt100 emulation)
wsdisplay0: screen 2 added (80x25, vt100 emulation)
wsdisplay0: screen 3 added (80x25, vt100 emulation)
wsdisplay0: screen 4 added (80x25, vt100 emulation)