Bandwidth problem

Hi Everybody:

I have recently bought 2 Myrinet card (Lanai 9 PCI64B) and a cable to form a
point-to-point connection between 2 computers. One of the computer is a Dell
Precision 530, Pentium4 Xeon 2.8 GHz dual, and the other is an AMD Athlon
MP.

Running gm_debug -L on each computer produces the following results:

On Dell Precision 530:

    Opening board 0, port 0

    ## This revised gm_debug output eliminates numerical-rounding errors
    ## produced by the pre-GM-1.5.1 (pre 1/2002) gm_debug utility.

    DMA rate for 4096 Byte pages (64bit / 66MHz bus)
    Timing 32 pages.
            bus_read  (send) = 225 MBytes/s
            bus_write (recv) = 301 MBytes/s

On AMD Athlon MP:

    Opening board 0, port 0

    ## This revised gm_debug output eliminates numerical-rounding errors
    ## produced by the pre-GM-1.5.1 (pre 1/2002) gm_debug utility.

    DMA rate for 4096 Byte pages (64bit / 33MHz bus)
    Timing 32 pages.
            bus_read  (send) = 148 MBytes/s
            bus_write (recv) = 222 MBytes/s

However, when I run the gm_allsize to test for bandwidth, I got the
following:

# Opening board 0  port 2. Sending to port 2.
# Not verifying contents of messages.
Remote GM node id PUPIL-HpCW (2).
# Reliable Point-to-Point Ping-Pong Bandwidth (MByte/s)
# size 20
#
gm_allsize --remote-host=PUPIL-HpCW --size=20 --geometric --bidirectional --
bandwidth
1 0.04
2 0.08
3 0.13
4 0.17
5 0.20
6 0.24
7 0.28
8 0.32
9 0.36
10 0.40
11 0.45
12 0.48
13 0.52
14 0.55
15 0.59
16 0.64
18 0.67
20 0.77
22 0.86
24 0.96
27 1.08
30 1.18
33 1.25
37 1.37
41 1.50
46 1.71
51 1.85
57 2.04
64 2.38
72 2.67
81 3.01
91 3.38
102 3.64
114 3.98
128 4.04
144 4.39
162 5.20
182 5.49
204 5.88
229 6.62
257 6.24
289 6.85
325 7.37
365 8.44
410 9.18
461 9.98
518 10.87
582 11.47
654 12.89
735 14.13
826 14.53
929 15.80
1045 16.63
1175 18.67
1321 18.91
1486 20.38
1671 21.14
1879 22.32
2113 23.68
2377 23.95
2674 25.64
3008 27.01
3384 27.20
3807 27.85
4282 34.31
4817 34.67
5419 36.69
6096 37.83
6858 39.84
7715 40.88
8679 42.78
9763 43.81
10983 46.77
12355 46.32
13899 49.35
15636 51.40
17590 52.57
19788 55.32
22261 55.62
25043 56.34
28173 59.76
31694 59.88
35655 62.66
40111 63.28
45124 62.40
50764 64.37
57109 64.52
64247 68.03
72277 68.12
81311 69.43
91474 68.77
102908 67.98
115771 70.57
130242 71.61
146522 72.16
164837 71.29
185441 71.69
208621 72.52
234698 73.27
264035 72.85
297039 73.62
334168 73.79
375939 73.93
422931 72.51
475797 72.38
535271 74.49
602179 71.44
677451 74.70
762132 71.75
857398 75.54
964572 74.47

Total queued   = 24300.
Total sent     = 24300.
Total received = 24300.


It occurs to me that the full bandwidth is not ultilized at all. But why?
I have read from FAQ that i860 has a bug that slows down the PCI bus and a
recompilation is needed to improve the bandwidth. Is this the case here?

Thank you.

Wilson Fung