Re: race condition?

On Wed, May 17, 2006 at 02:36:35PM -0600, Matt Dew wrote:
> Tab,
>   Expanding on your comments, here's the worst case conditions we need
> to account for (let me know if I've forgotten any.) Again, I'm not much
> of a software guy so let me know if I'm heading in the wrong direction.
> 
> 1)  remote node sends a syscall home for processing at the same time the
> home node sends a migration home command (or signal) to the remote node.
>  The network packets pass each other in flight.
> 
> 2)  The remote node sends a syscall home and the home node sends the
> response but the remote node errors out and sends home an error
> condition.  
> 
> (longer term)
> 3)  We have an SMP machine with a process that forked after it migrated,
> so both processes share the same file descriptor.   Both processes send
> syscalls home at the same time (one write and one read for example.).  
> Home node sends a migration command at the same time.
> 
> 
> A)  The home node has a queue for things it needs to send the remote
> node.
> B)  The remote node is the boss and if both nodes want to do something,
> remote always wins.
> C)  Acks are always send before anything, so both know who is talking
> and who is listening."
> D)  There's a response/ack for each communication.
> E)  The structs have two more fields,  originating node
> ('home','remote'), and msg_num.
> 
> 
> "I'm sending syscall X to you, and give me back the result",
> "sending page X, then page Y, ..." ... 
> 
> becomes
> remote: "Ok, I'm not currently processing a signal or command from the
> home node. I'm gonna set a flag saying I'm doing the talking."
> remote:  "Hey, home node,  I'm gonna send you a syscall."
> home: "OK, go ahead, I have a signal in my queue I need to send you, but
> you have priority, I'll try again later."
> remote: "I got your request but ignored it since my flag is set. Here
> comes my syscall. Let me know if you got it."
> home: "Ok, I got it. Response will be next"
> remote: "Awaiting response"
> home: "Here's the response, page X, let me know if you got it ok."
> remote:  "I have an error."
> home:  "Ok, I'll resend."
> remote: "I still have the error."
> home:  "Ok. Nothing I can do to help you."
> remote:  "I'm done with you, I'm returning an error code to the calling
> function. and I'm clearing my flag."
> ...
> home: "Hey remote, I want to send you a signal."
> remote: "OK, I'll set a flag saying you're doing the talking."
> home: "here's the signal <msg_num=12>."
> remote: "Got it <and the msg_num is 12>. Any more?"
> home: "Nope. I'm done."
> remote: "OK, you're done, I'm clearing my flag."
> 
> Thoughts?
> 
> It's really verbose, but doing everything over one port is harder than
> having two (or more) ports.
> 
> How do you put a process to sleep for event number X when the events are
> all just tcp packets?

That's seems overly complex to put that in the process kernel part. Plus
the logic have to be duplicate per process, instead of having simple
daemon that handle almost everything and hide everything to the process
kernel part.

actually having 1 port (or 2) open per node2node communication no matter
what are the number of process from this node seems even saner.

( - 1port: 1 port for "node A <-> node B"
  - 2ports: 1 port for "node A -> node B" and 1 port for "node B -> node A" ?)

Some kernel thread (kcomd) is in charge to do the multiplexing/demux.

so you have something like

   kmigd  ----------\                                   /-> kmigd
   deputy process1 ---> kcomd(1) ---network--- kcomd(2) --> process1
   deputy process2 --/                                  \-> process2
   deputy process3 -/                                    \> process3

kcomd can do all the hard work, with a packet queue (even like an iovec).
kcomd take the pid of the process to do the multiplex and don't stop
until a packet is fully received

all you have to do is put a request into the queue, with different
semantic. like:

   process(x) say:
   send(stuff, node Y) -> kcomd(1) read the request, and ack the request
                          to this process(x) (or not)

   send_with_ack(stuff, node Y) ->
	kcomd(1) read the request, and put the process to sleep. when
	it's time to send the packet kcom received from the process(x),
	send it over the network to kcomd(2), and kcomd(2) send a ack if
	received. wake up process(x) with the ack response from kcomd(2)
	(ERR or SUCCESS).

   send_with_reply(stuff, &stuff2, node Y) ->
   	kcomd(1) read the request, put the process to sleep. send the
	packet to kcomd(2). when kcomd(2) is going to reply what
	process(x) need, wake up process(x) and continue.

Now .. there's a great potential for deadlock there, but that should
be alright once we have the semantics right.
   
What do people think ?
-- 
Vincent Hanquez


-------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642