Re: [myrinet] limited ports in GM

Hi -

Wenchao asked:

 > My research group are working on some project involving
 > myrinet.  We had no prior experience about myrinet.
 >
 > We have set up the hardware, which is lanai4 PCI cards.  We
 > also set up the latest myricom message passing software, the
 > GM.1.2.3.  The first problem we have noticed is that GM only
 > allows 8 ports (well actuall only 6 usable ports)...

If you can tell us why you need more ports, we _might_ be able to
help you to achieve the same objective with the default 8 ports.


 > ... So we tried to recompile the GM with GM_NUM_PORTS set to
 > 16, ...

16 ports (numbers 0 through 15) is the _absolute maximum_ we
would feel comfortable with.  If you can get along with fewer
than 16, use the smaller number.

This is _especially_ true when using LANai4 cards.  Memory on
the LANai4 card is not as ample as on our later cards.


 > ... but we are stuck with the problems that keep coming up:
 >
 > 1. After compiling the GM without the --enable-mcp-build
 > option, the GM_INSTALL always failed with "failure to attatch
 > driver"
 >
 > 2. If I compile the GM with --enable-mcp-build option, even
 > though GM_INSTALL went through, my application would have
 > memory problem "No mapping for user address ....", and
 > corrupted data were transmitted across the net.

It's not clear from this whether you have installed the LANai
tools (see http://www.myri.com/scs/L3/index.html).  In order
to change the number of ports, it is necessary to rebuild the
MCP as well as GM.

Assuming that you have installed the LANai tools, the following
sequence should get you on the air with the changed number of
ports:

   1.  Change GM_NUM_PORTS in gm_types.h from 8 to the number
       you wish.
   2.  configure --enable-mcp-build
   3.  [g]make clean
   4.  [g]make
   5.  GM_INSTALL


 > So I am wondering maybe I need to reload the lanai processor
 > with the recompiled mcp program explicitly.  But I am not
 > sure what is the proper procedure to do this.

No, you don't need to take any special steps.  When you build
GM, the MCP is embedded into the GM executable, and at driver
initialization time, GM loads the MCP into the LANai.


Hope this helps.


- Max Stern
  Myricom, Inc.