directed send problems

Hi,

I have been banging my head against this problem for 2 weeks now and am
uncertain whether or not I am thoruoghly abusing the directed send call.

The scenario:

I am using gm to perform messaging for parallel processing. I have a
master process that sets up its port and address space "before" forking
and sends this information to remote processes when it spawns them (rsh).

The slave processes report back to the master with their ports and base
addresses in memory for directed sends.

When the master then broadcasts (i.e. sends a message to each
process) with the table of ports and addresses, the children get
garbage. I have verified that there is something getting written to the
slaves dma'able memory, it's just not what I sent.

The same transmit message function is used when the slaves report to the
master and vica-versa. Walking thorugh with a debugger verifies that the
message in the send buffer that is sent to the children is indeed what I
want to be sent and is being sent to where I want.

I am using directed sends, so I am not using any explicit tokens.

Each process has its own port and receive buffer.

The only port that is set up before forking is the master processes port,
it then forks as many processes as there are cpu's on its node. The other
nodes don't set up ports until after forking.

The "garbage" that is received is 0's EXCEPT one field has what could be
an address in the heap (or could just be nonsense). This address does not
correspond to the address of the send buffer or of any of the buffers or
classes that hold the message during packing.

Any ideas?

Thanks!
Keith

P.S. This is for running an experimental parallel processing API on
myrinet. Using some other API for parallel computing across myrinet is not
an option.

Keith Parkins                   U of R Computer Science
[email protected]       Computer Science Building, Room 606
(716)275-1118