Re: Error: Need to obtain conf in GMPI_OPTS, and GMPI_CONF variables

bobb <[email protected]> Fri, 21 Mar 2003 18:14:54 +0000
Newsgroups gmane.network.myrinet.general
Organization bobb Industries
Message-ID <[email protected]>
Pete Wyckoff hath declared on Friday the 21 day of March 2003  :-:

> > After upgrading our cluster we are getting the above error
> > on _some_ of our nodes, even though they are virtualy identical.

> > Innitally we didn't think it was a problem as our MPI jobs were running fine,
> > however one or two users now think it may be causing issues.
 
> Probably you see that message on only some of your user codes, actually.
> The message comes from libmpich.a.  This is what happens when you try to
> run a code compiled against an old mpich library using a new mpiexec.
> "Old mpich" means older than 1.2.4..8 which was released in Aug 2002 to
> be precise.

Sorry, I should have provided more details, my brain isn't awake
today.   We are not using mpiexec, just mpirun.  I have recompiled the
code with the new versions of gm and mpich, we also instructed the users
to recompile.

The odd thing is that this error only occures when running through pbs
I can't reproduct it if I ssh/rsh to the nodes and run manually.

A simple example with cpi.

qsub script:
-------------8<------------------8<---------------
#!/bin/sh

#PBS -l nodes=1:ppn=2:skuld.iitac.maths.tcd.ie
#PBS -q parallel
#PBS -l walltime=00:01:00
#PBS -N cpi_test

CPI_D=${HOME}/cpi_test

NCPUS=`wc -l <$PBS_NODEFILE`

cd $CPI_D
/usr/support/mpich/bin/mpirun -machinefile $PBS_NODEFILE -np $NCPUS ./cpi
-------------8<------------------8<---------------


The output file shows the correct answer:
-------------8<------------------8<---------------
pi is approximately 3.1416009869231241, Error is 0.0000083333333309
wall clock time = 0.000607
PBS job has successfully completed
-------------8<------------------8<---------------

However the folowing is in the error file:
-------------8<------------------8<---------------
Process 0 on heimdall.iitac.maths.tcd.ie
Process 1 on heimdall.iitac.maths.tcd.ie
Error: Need to obtain conf in GMPI_OPTS, and GMPI_CONF variables
Error: Need to obtain conf in GMPI_OPTS, and GMPI_CONF variables
-------------8<------------------8<---------------

Would this "Error" be causing any problems ?

Thanks,

- bobb

-- 
Robert "bobb" Crosbie.
Trinity Centre for High Performance Computing,
O'Reilly Institute,Trinity College Dublin.