RE: [myrinet] Re: GM-1.5, Linux-2.4.14, gm_board_info hangs

Responses to specific questions below:

> -----Original Message-----
> From: Susan Blackford [mailto:[email protected]]
> Sent: Thursday, November 15, 2001 4:27 PM
> To: Ritch, David
> Cc: [email protected]; [email protected]
> Subject: [myrinet] Re: GM-1.5, Linux-2.4.14, gm_board_info hangs
> 
> 
> Hi,
> 
> This is a kernel "oops"/seg fault.
> 
> Did you see any strange messages in the kernel log (dmesg) when you
> GM_INSTALL?

Sometimes.  I have been through several iterations of build and install.
When I don't carefully build GM using "kgcc -I/usr/src/linux/include", I see
messages like the following:

Nov 15 21:07:33 t0009 kernel: GM: NOTICE: 
Nov 15 21:07:33 t0009 kernel: GM: difference between running kernel and
compilation headers
Nov 15 21:07:33 t0009 kernel: GM: This build is unsupported.
Nov 15 21:07:33 t0009 kernel: GM: ^Irunning kernel:#10 SMP Thu Nov 15
20:30:20 GMT 2001
Nov 15 21:07:33 t0009 kernel: GM: ^Isource tree at driver compilation: #8
SMP Tue Nov 13 21:00:04 GMT 2001

However, in the most recent case, I saw nothing like this, and I was able to
run the systest from the mpich-1.2.1..7 examples without a problem.  Here is
the info from syslog:

Nov 13 23:11:12 t0009 kernel: GM: Version 1.5_Linux build 1.5_Linux
[email protected] Tue Nov 13 21:38:38 GMT 2001
Nov 13 23:11:12 t0009 kernel: GM: Unusual memory configuration:
Nov 13 23:11:12 t0009 kernel: GM: PAGE_ZERO=0x0, HIGH_MEM=0x40000000,
KERNEL_HIGH_MEM=0x38000000
Nov 13 23:11:12 t0009 kernel: GM: Memory available for registration: 229376
pages (896 MBytes)
Nov 13 23:11:15 t0009 kernel: GM: MCP for unit 0: L9 4K (new features)
Nov 13 23:11:15 t0009 kernel: GM: LANai rate set to 132 MHz (max = 134 MHz)
Nov 13 23:11:15 t0009 kernel: GM: Board 0 page hash cache has 32768 bins.
Nov 13 23:11:15 t0009 kernel: GM: Allocated IRQ11
Nov 13 23:11:15 t0009 kernel: GM: 1 Myrinet board(s) found and initialized

Then I ran a user code, and got the result I mentioned.

> What about when you ran the mapper, did you see any strange messages?

I'm not sure what constitutes a strange message for the mapper.  I didn't
see anything that was obviously out of the ordinary.

> Did the gm_board_info hang happen on all of the nodes in your cluster,
> or only a specific node?

On the node that was submitting the job.  This node is also the mapper node.
It may have hung on the other nodes running the particular job, but only on
those.

> Would it be possible for us to have remote access to this machine and
> be able to load/unload the GM module (root) and experiment?

I'm not sure - it is at a client site, and may be subject to their policies.

> This is very strange. 

Good!  I didn't think that this was the normal behavior of myrinet.  ;-)

> Susan

David