Re: [myrinet] Re: Low bandwidth

On Tue, Jul 31, 2001 at 09:31:10AM -0400, Pete Wyckoff wrote:
> [email protected] said:
> > I just built up my first beowulf cluster with the latest PCI64B-2
> > cards (PCI64C was available just a month later our order :-(). 
> > Hosts are all DEC Alphastation 500/500 (alcor arch). OS is Linux
> > with kernel 2.2.19. Buses are 32bits @ 33Mhz so my guess is at least
> > a 100-130MBs of throughput. In fact, reality is well different...
> > gm_allsize gives me a maximum bandwith of 56MBs. 
> 
> What does gm_debug say for a DMA rate?


That's interesting:

Opening board 0, port 0

DMA rate for 8192 Byte transfers (32bit / 33MHz bus)
1st: 8 pages from bogus sdma pg, 8 to bogus rdma
        bus_read  (send) = 126 MBytes/s
        bus_write (recv) = 60 MBytes/s

DMA rate for 8192 Byte transfers (32bit / 33MHz bus)
2nd: sdma a page, rdma a page 8 times
        bus_read  (send) = 126 MBytes/s
        bus_write (recv) = 60 MBytes/s

DMA rate for 8192 Byte transfers (32bit / 33MHz bus)
3rd: sdma and rdma to/from alternating pages (coarse grain)
        bus_read  (send) = 126 MBytes/s
        bus_write (recv) = 60 MBytes/s

DMA rate for 8192 Byte transfers (32bit / 33MHz bus)
4th: sdma and rdma to/from alternating pages (fine grain)
        bus_read  (send) = 126 MBytes/s
        bus_write (recv) = 60 MBytes/s

First two words of LANai globals: 0x10000000, 0xcafebabe
Some counters from LANai

netsend_cnt      1810209
netrecv_cnt      1439851
error counters
  bad_header_cnt                                      0
  bad_length_cnt                                      0
  bad_type_cnt                                        0
  badcrc_cnt                                          0
  badroute_cnt                                        0
  bogus_header_cnt                                    0
  drop_cnt                                            5
  handle_connection_reset_request_cnt                 2
  misrouted_cnt                                       0
  nack_cnt                                            0
  nack_down_cnt                                       0
  nack_ignore_close_connection_cnt                    0
  nack_ignore_open_connection_cnt                     1
  nack_ignored_cnt                                   17
  nack_normal_cnt                                    17
  nack_receive_close_connection_cnt                   0
  nack_receive_open_connection_cnt                    3
  nack_received_cnt                                  41
  nack_reject_cnt                                     0
  nack_send_close_connection_cnt                      0
  nack_send_nothing1_cnt                              0
  nack_send_nothing2_cnt                              0
  nack_send_open_connection_cnt                       2
  no_match_for_datagram_recv_cnt                      0
  no_match_for_ether_recv_cnt                         0
  no_match_for_reliable_recv_cnt                      0
  no_match_for_raw_recv_cnt                           0
  out_of_sequence_cnt                                 2
  resend_cnt                                         19
  short_mapper_config_packet_cnt                      0
  short_mapper_packet_cnt                             0
  short_mapper_scout_packet_cnt                       0
  short_packet_cnt                                    0
  used_bogus_send_cnt                                 0
  used_bogus_recv_cnt                                 0
  zero_len_cnt                                        2
 
       packet_sexno 0000 0001
       ack_sexno 03e9 03e8
 
 
port counters
  port[2].active_subport_cnt                          0


So there is a difference in rcv/snd throughput !?

-- 
Francesco P. Lovergine