Re: [myrinet] tcp/ip in linux 2.4 kernel

Hi Patrick,

Here are the CPU usage for messages of size 512 with sending and
receiver buffers of 32767 by way of the output of top,

I included the CPU monitor stuff as well,

Actually on this test we hit throughput of 1045 Gb/s

Here is the output of top

netperf side:
CPU0 states:  3.2% user, 59.0% system,  0.0% nice, 37.1% idle
CPU1 states:  1.4% user, 38.4% system,  0.0% nice, 59.0% idle

  PID USER     PRI  NI  SIZE  RSS SHARE STAT %CPU %MEM   TIME COMMAND
13871 cam       15   0   720  720   548 R    99.9  0.1   8:11 netperf


netserver side:
CPU0 states:  0.0% user, 26.3% system,  0.0% nice, 73.1% idle
CPU1 states:  0.3% user, 67.2% system,  0.0% nice, 31.4% idle

  PID USER     PRI  NI  SIZE  RSS SHARE STAT %CPU %MEM   TIME COMMAND
14321 cam       15   0   868  868   436 R    71.1  0.1   0:54 netserver

On Thu, 28 Jun 2001, Patrick Geoffray wrote:

> Andrew CJ Macdonell wrote:
> 
> > test our setup and can't seem to get performance above 1 Gb/s, we top out
> > at about 960 Gb/s, does any one know of other results on the linux 2.4
> > kernel?  I have seen the results from Trapeze and realize they use
> > FreeBSD and don't use GM, but still I imagine the protocol performance
> > difference should not be that great, our performance with UDP is quite
> > good, up to 1800 Gb/s.
> 
> 960 Gb/s and 1800 Gb/s on a 2 Gb/s link ? :-)
> 
> I haven't seen better than 1 Gb/s with TCP on PIII machines. The memory 
> copies in the TCP/IP stack are the bottleneck. I have seen 1,7 Gb/s 
> with UDP, because the kernel overhead is much smaller. 
> 
> The kernel 2.4 is much better than 2.2 (no more than 1 Gb/s UDP with
> 2.2)
> 
> By curiosity, what is the sender and receiver CPU load when you top out 
> at 960 Mb/s with TCP ?
> 
> Patrick
> 
> -----------------------------------------------------------
> |   Patrick Geoffray, Ph.D.      [email protected]         |
> |   Myricom, Inc.                http://www.myri.com      |
> |   Cell: 865-389-8852           325 N. Santa Anita Ave.  |
> |   Fax:  865-974-1950           Arcadia, CA 91006        |
> -----------------------------------------------------------
>