Re: race condition?

On Fri, May 19, 2006 at 05:08:31PM -0600, Matt Dew wrote:
> Tab, et al,
>   The sync stuff is definitely much too complex, not too mention the
> performance would be terrible. I was trying to see how to do it with
> just one port. If we used two per process that whole issue goes away. 
> Does kcomd buy anything other than fewer ports that can't be done with a
> restructuring of the code?  It's another process/daemon running and if
> it dies what happens to the migrated processes.   Granted, any kernel
> daemon that dies can screw things.   Scalability?

why would a kernel daemon dies ?

For the scability issue, that could be an issue to open only 2 ports for
all processes.

not sure though, since you lose lots of time, in the many port case,
with lots of TCP handshake. And i would value stability and correctness
over performance.

> Have two ports:  1 port for "node A -> node B" and 1 port for "node B ->
> node A". Replies and acks are sent back over same port.
> 
> A om_comm_send_with_ack(stuff) and a om_comm_send_with_reply(stuff)
> which would talk to the other node directly. no kcomd, but the process
> would still sleep while it waits.

yeah but it would be waiting for *data* .. instead of *command*.
data that could not be valid, or partial, ...

>  Each process having 2 ports open per process isn't all that many ports
> is it?  Even if there are 200 processes, that's only 400 ports. 
> iptables handles WAY more than that with no problems.

even if that's unlikely that 200 processes from other nodes appear on a
single node, having 400 ports open for just housekeeping and stuff, is
just gross. The kernel can certainly handle the case, that's not the
question.

Having each process to handle all what could happen over a socket is not
very good. Instead you could have this kcomd that handles everything:
communication dying, ports closed need to reopen, partial packets
sent/received, queue of packet. and then hands a valid packet to the
process.

(Now I'm not talking about duplicate code, since that's could all be
factorize in functions to handle that, but each process doing the same
logic.)

Plus having application to handle the network, mean you cannot do
everything. like sending data through network in atomic code
(recent bug: copy_from_user in atomic).
now they just go to sleep, which give all sort of other problems ;)

> If all the calling functions and receive functions were merged into one
> each, the process would still be put to sleep while it waits.

it's *already* put to sleep by separate sending and receving function.
now the thing is, it's waiting for data, not a command.
if the remote send one byte, and then what's left to send, the deputy
will read one byte, and die because.

What i'm proposing is having a separate daemon that wait for data, and
construct all packets, and then wake up process when there's command to
do. The process would not have to mess with any file descriptor,
network, ...

Plus it would be lots easier to do migration and fork/clone since you
don't need to open ports at all during this phase (port is already open
by kcomd, and you can expect kcomd to open port if needed)

> How does 2.4 do things?

2.4 does bad things. there's LOTS of assumption over the code, and any
failure destroy processes in the best case, or just crash the node in the
worst case.
And let's not forget that 2.6 is way different than 2.4, consider
PREEMPT, softirq, ... so looking what 2.4 does is not a good idea.

2.4 is skanky, and bad code. better not look what it's doing.

> This might be a silly question but is it possible to have a daemon
> 'kmigd' that takes care of all the housekeeping.  No om_pre_usermode
> stuff at all.  The daemon handles signals and sets process flags
> accordingly, rather than the process kernel stub doing it. Talks with
> other nodes and responds accordingly. checks the om_struct and puts
> process to sleep and migrates it then wakes it up on remote node.  That
> way it really is just a small kernel module and patches are unobtrusive?
>  The syscall trap in entry.S would have to stay as I think the syscall
> table isn't trappable like it was in 2.4.  Is that correct?

The process deputy stub cannot go away. It keeps lots of needed stuff
like file descriptors opened, signals mask, ...

Lots of code in the kernel expect that "current" task is doing stuff.
so you really need to be in this process context to execute some
function (and signal subsystem is one of them here), that why there's a
process stub staying on the deputy.

-- 
Vincent Hanquez


-------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642