Re: [myrinet] static mapping?

As an addendum to my last email, unloading the module with processes that
have open ports will not be possible. You'll get 'device busy' errors. You
can write a small program that retrieves the PID's of the processes that
have the ports open. You cold take the program below and modify it to send
KILL signals to the offending processes.


/* Uses undocumented gm calls, need the gm source to compile.
   
   Example makefile

CC = kgcc
GM_SRC = /usr/local/src/gm-cvs
LIBDIRS = -L$(GM_SRC)/binary/lib
INCDIRS = -I$(GM_SRC)/include -I$(GM_SRC)/drivers/linux/gm
LIBS = -lgm


gm_port_info: port-info.c

        $(CC) $< -o $@ $(INCDIRS) $(LIBDIRS) $(LIBS)


   Doug Johnson, [email protected]

*/
   


#include <stdio.h>
#include <string.h>
#include <ctype.h> 
#include <unistd.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>  

#include "gm.h"
#include "gm_internal.h"

void port_use(){
  
  
  gm_myrinet_eeprom_t eeprom;
  int my_board_num=-1, board_num, port_num=0,i,j,mypid,size_pid;
  FILE *fname;
  gm_port_t *port;
  gm_status_t status;
  gm_u32_t opener_pids[GM_NUM_PORTS]; 
  char filename[20]="/proc/",char_pid[5],command[256],c;

  mypid = getpid();
  for (board_num = 0; board_num<4; board_num++){
    if ((_gm_open_common (&port, board_num, port_num, "gm_board_info",
		     GM_API_VERSION_1_0)) != GM_SUCCESS)
      continue;
    
    status = _gm_get_eeprom (port, &eeprom);
        
    for (i=0; i<GM_NUM_PORTS; i++)
      {
	opener_pids[i] = 0;
      }
    if (_gm_get_opener_pids(port,opener_pids) == GM_SUCCESS)
      {
	for (i=0; i<GM_NUM_PORTS; i++)
	  {
	    if (opener_pids[i])
	      {
		sprintf(char_pid,"%d",opener_pids[i]);
		memcpy(filename+6,char_pid,sizeof(char_pid));
		j = 0;
		while (char_pid[j++]!='\0')
		  {
		    size_pid = j;
		  }
		memcpy(filename+6+size_pid,"/cmdline\0",9);
		if ((fname = fopen(filename,"r")) == NULL){
		  printf("Could not open file: %s\n",filename);
		  exit(-1);
		}
		j = 0;
		while((c = getc(fname)) != EOF){
		  command[j++] = c;
		}
		command[j] = '\0';
		printf("%1d: %4d: %6s %s  ",board_num,i,"BUSY",command);
		if (opener_pids[i] == 1)
		  printf("%s","(ip-over-GM)");
		printf("\n");
	      }
	    else
	      printf("%1d: %4d: %6s\n",board_num,i,"free");
	  }
      }
  }
}

int main(int argc, char *argv[]){

  port_use();
  return(0);
}

Doug

On Wed, 18 Apr 2001, Oleg I. Vdovikin wrote:

> Hello,
> 
>     we're using alpha-linux-myrinet cluster which is now consists of 64
> nodes. The main problem is the gm stability - sometimes not all open ports
> closed after the process finished. So we decide to reload gm linux module
> between apps starts (i.e. something like "rmmod gm; insmod gm" performed
> between tasks). But the main problem here is the need to remap the network
> after such manipulation - so now we're performing map-once process on fixed
> node to remap the network. But this aproach is not good enough. Unfortunatly
> the node which is used for mapping is also used as computational node, and
> sometimes it crashes: sometimes we got "spinlock stuck" messages forever,
> sometimes any gm related software (such as mapper & gm_board_info & any
> other user tasks) hangs... So now we're looking for solution with no single
> point of failure for mapping. As seems the most logical solution is the
> static mapping - i.e. we need to load individual map/routing tables into
> each "rebooted" lanai chip. And we did not need to load/update this maps on
> other nodes. Is this possible?
> 
>     As seems gm comes with file_mapper but I'm not sure is this compiled to
> "sbin/mapper" in the resulting binary tree. And as seems this mapper will
> update routing maps on the entire network, not just the current node...
> 
>     Does anyone use this? Any ideas?
> 
> 
> Best regards,
>     Oleg.
> 
> 

-- 
Doug Johnson
Systems Developer/Engineer
The Ohio Supercomputer Center
[email protected]