Re: Dual ported "E card" performance
Markus Fischer <[email protected]> Sun, 25 Jan 2004 16:53:19 -0800
| Newsgroups | gmane.network.myrinet.general |
|---|---|
| Message-ID | <[email protected]> |
Bill, last year at SC2003 in Phoenix we demonstrated Sockets-GM performance using 2 E - Cards, on dual Xeon nodes. The combined sustained bandwith using two clients was 7.1 GBit/s. Depending on the message size you can switch to a zero copy protocol. The reported CPU usage is 5-7 per cent for large messages. I will copy paste the netperf output which is using one E - Card: [root@asa3 netperf]# ./netperf -l 5 -H 192.168.2.201 -- -m 4000 -M 4000 TCP STREAM TEST to 192.168.2.201 Recv Send Send Socket Socket Message Elapsed Size Size Size Time Throughput bytes bytes bytes secs. 10^6bits/sec 87380 16384 4000 5.00 3397.72 [root@asa3 netperf]# ./netperf -l 5 -H 192.168.2.201 -- -m 8000 -M 8000 TCP STREAM TEST to 192.168.2.201 Recv Send Send Socket Socket Message Elapsed Size Size Size Time Throughput bytes bytes bytes secs. 10^6bits/sec 87380 16384 8000 5.00 3944.04 With CPU Usage: [root@asa3 netperf]# ./netperf -c -C -l 5 -H 192.168.2.201 -- -m 125000 -M 125000 TCP STREAM TEST to 192.168.2.201 Recv Send Send Utilization Service Demand Socket Socket Message Elapsed Send Recv Send Recv Size Size Size Time Throughput local remote local remote bytes bytes bytes secs. 10^6bits/s % T % T us/KB us/KB 87380 16384 125000 5.00 3115.48 12.00 13.00 0.631 0.684 [root@asa3 netperf]# ./netperf -c -C -l 5 -H 192.168.2.201 -- -m 150000 -M 150000 TCP STREAM TEST to 192.168.2.201 Recv Send Send Utilization Service Demand Socket Socket Message Elapsed Send Recv Send Recv Size Size Size Time Throughput local remote local remote bytes bytes bytes secs. 10^6bits/s % T % T us/KB us/KB 87380 16384 150000 5.00 3226.62 11.40 10.20 0.579 0.518 [root@asa3 netperf]# ./netperf -c -C -l 5 -H 192.168.2.201 -- -m 250000 -M 250000 TCP STREAM TEST to 192.168.2.201 Recv Send Send Utilization Service Demand Socket Socket Message Elapsed Send Recv Send Recv Size Size Size Time Throughput local remote local remote bytes bytes bytes secs. 10^6bits/s % T % T us/KB us/KB 87380 16384 250000 5.00 3436.45 8.40 7.80 0.400 0.372 [root@asa3 netperf]# ./netperf -c -C -l 5 -H 192.168.2.201 -- -m 500000 -M 500000 TCP STREAM TEST to 192.168.2.201 Recv Send Send Utilization Service Demand Socket Socket Message Elapsed Send Recv Send Recv Size Size Size Time Throughput local remote local remote bytes bytes bytes secs. 10^6bits/s % T % T us/KB us/KB 87380 16384 500000 5.00 3592.06 5.80 7.80 0.265 0.356 Bill Hartner wrote: >On a dual Xeon, the user sustained uni/bi-directional data rate for the >dual ported "E card" is quoted as 495/950 MB/s. > >http://www.myri.com/news/03a14/ > >This rate was for a GM 2.1 microbenchmark. Are there perf results for a >GM sockets application or netperf unmodified on the "E card" - including >throughput, cpu utilization, and latency ? > >I am assuming you can assign one IP address to the dual ports. Will >the packets be distributed across both ports if TCP streaming to/from >another dual card with a single ip address or TCP streaming to/from two >other clients each with a single port "D" card using Sockets GM or just >tcp/ip ? > >-Bill >_______________________________________________ >Myrinet mailing list >[email protected] >http://email.osc.edu/mailman/listinfo/myrinet > > >