Re: [myrinet] Re: limited ports in GM]

Hi. After reading these mails related to maximum number of GM ports
I am just wondering about the usage of GM ports. To be frank with you,
I've never used GM software. Instead we are developing VIA on myrinet(Lanai7.x) 
and We'd like to support at least 512 connections per a node. With luck,
We'll see it in late December. 
Anyway, how does GM communicate with other nodes in large-scaled cluster 
computer which has hundreds of nodes interconnected? Is it possible for GM
with 16 ports available? (hm.. GM port might not be a dedicated endpoint.
It might multiplex packets for an upper sub-layer endpoints.)
I'll appreciate for your comments. Thank you.

p.s. well, I'm not sure that sending this message to this mailing list.
     However, I heard that Max Stern forwarded the above message to
	 myrinet help desk and have not received any response. This is the
	 only way that I can do. :)

On Thu, Oct 05, 2000 at 01:43:59PM -0700, Max Stern wrote:
> Hi -
> 
> Wenchao asked:
> 
>  > My research group are working on some project involving
>  > myrinet.  We had no prior experience about myrinet.
>  >
>  > We have set up the hardware, which is lanai4 PCI cards.  We
>  > also set up the latest myricom message passing software, the
>  > GM.1.2.3.  The first problem we have noticed is that GM only
>  > allows 8 ports (well actuall only 6 usable ports)...
> 
> If you can tell us why you need more ports, we _might_ be able to
> help you to achieve the same objective with the default 8 ports.
> 
> 
>  > ... So we tried to recompile the GM with GM_NUM_PORTS set to
>  > 16, ...
> 
> 16 ports (numbers 0 through 15) is the _absolute maximum_ we
> would feel comfortable with.  If you can get along with fewer
> than 16, use the smaller number.
> 
> This is _especially_ true when using LANai4 cards.  Memory on
> the LANai4 card is not as ample as on our later cards.
> 
> 
>  > ... but we are stuck with the problems that keep coming up:
>  >
>  > 1. After compiling the GM without the --enable-mcp-build
>  > option, the GM_INSTALL always failed with "failure to attatch
>  > driver"
>  >
>  > 2. If I compile the GM with --enable-mcp-build option, even
>  > though GM_INSTALL went through, my application would have
>  > memory problem "No mapping for user address ....", and
>  > corrupted data were transmitted across the net.
> 
> It's not clear from this whether you have installed the LANai
> tools (see http://www.myri.com/scs/L3/index.html).  In order
> to change the number of ports, it is necessary to rebuild the
> MCP as well as GM.
> 
> Assuming that you have installed the LANai tools, the following
> sequence should get you on the air with the changed number of
> ports:
> 
>    1.  Change GM_NUM_PORTS in gm_types.h from 8 to the number
>        you wish.
>    2.  configure --enable-mcp-build
>    3.  [g]make clean
>    4.  [g]make
>    5.  GM_INSTALL
> 
> 
>  > So I am wondering maybe I need to reload the lanai processor
>  > with the recompiled mcp program explicitly.  But I am not
>  > sure what is the proper procedure to do this.
> 
> No, you don't need to take any special steps.  When you build
> GM, the MCP is embedded into the GM executable, and at driver
> initialization time, GM loads the MCP into the LANai.
> 
> 
> Hope this helps.
> 
> 
> - Max Stern
>   Myricom, Inc.

-- 
-------
Moonsang Lee	    [email protected]
CS dept. KAIST      +82-42-869-5552@calab  
Eastern Dorm. 6303  +82-42-879-2663@dormitory
Cellular            +82-11-9653-4647
homepage URL        http://camars.kaist.ac.kr/~mslee

----- End forwarded message -----

-- 
-------
Moonsang Lee	    [email protected]
CS dept. KAIST      +82-42-869-5552@calab  
Eastern Dorm. 6303  +82-42-879-2663@dormitory
Cellular            +82-11-9653-4647
homepage URL        http://camars.kaist.ac.kr/~mslee