Re: restructuring was: race condition?

Tab, et al, 
     let me know if I'm heading in the wrong (or right) direction or if
I'm just completely off my rocker. :)

more detailed info on kcomd and send_with_* functions:


For communication between kcomd daemons use a struct kinda like this:
struct omg_msg {
    char status;  // bit 0;  0=home; 1=remote
                         // bit 1; 0=new msg; 1=ack/reply
    sockaddr addr;   // ip of dest.
    unsigned int pid;
    unsigned int msg_id;   // msg counter
    unsigned int comm_size;  // size of this entire comm -- used to
verify comm received in full
    char type;  // what is this comm?
                      // 0=signal, 1=syscall_req, 2=syscall_ret,
3=fork_req, 4=fork_ret, 5=page_req
                      // 6=mmap_req, 7=execve_req, ...
    char *data;  // actual data described by type.  (send in separate
pkt(s)?)  


.......

For sending: [ send_with_ack()/reply()  ]

down(&addingtolist_mutex);  // preempty safety
list_add_tail(&stuff, &send_list_head);  // queue up our comm.
init_mutex_locked(&this_stuff_has_been_ackedreplied);  // kcomd up's
this mutex when it has received ack/reply, so process can sleep until
data is ready
up (&addtolist_mutex);  // we're good.

down(&this_stuff_has_been_ackedreplied); // sleep until kcomd has
(n)ack/reply

return(&stuff->reply)

---


kcomd():

  daemonize('kcomd');
  //  init stuff here.

// open socket

for (;;) {

  while (new_comms(send_list)) {
     if !(in_current_node_ip_list(socklist, stuff->addr) {
        newsock=open_new_connection(stuff->addr)
        addtosocklist(socklist,newsock,&stuff);
     }
     comm_send(&send_stuff, send_stuff->addr)
     list_add_tail(&send_stuff, &waiting_for_response_head);
  }

   recv=comm_recv(&recv_stuff);
   set_current_state(TASK_INTERRUPTIBLE);
   if (recv) {
   // loop through and find corresponding process
   if (recv_stuff->status && REPLY)
     list_for_each(p, &waiting for response_head) {  
        my=list_entry(p, struct omstruct, id)
        if (id ==  recv_stuff->msg_id) {
           myprocitem=get_process_pointer(recv_stuff->pid)   
           if (expected(myprocitem, stuff) { // ??
              up(myprocitem->this_stuff_has_been_ackedreplied) //
release mutex and wake up process
              list_del(myprocitem, &waiting_for_response_head);
           }
         }
    } else {  // new msg.
       list_for_each(p, &waiting for response_head)
          if (id ==  recv_stuff->msg_id) {
              myprocitem=get_process_pointer(recv_stuff->pid)
              add_to_list(myprocitem->work, &recv_stuff); // handle by
om_pre_usermode?
              down(&addingtolist_mutex);  // preempty safety
              list_add_tail(&stuff, &send_list_head);  // queue up our
response.
              up(myprocitem->has_comm);  // wake up process
              up(&addingtolist_mutex);  // preempty safety
          }
    }
    schedule();
}

kcomd is woken up by adding to send list or packets received and then
goes back to sleep when done.

1)  Do we want two kcomd daemons, one for each port? or have one daemon
doing everything?  I think one could probably handle it with no
problem...   The above would be duplicated for two ports.
2)  Can we use something similar to omg_msg above? 
2a)  Is it flexible enough that MAASK can port the mig_shm stuff without
this being an issue?
3)  What about allowing the userspace daemon to inform us of a new node
so we can establish a connection before we need it?    (another d-bus
use?)
4)  Merge all the remote_do* functions into one function?
5)  given pid can we retreive the proc pointer?
6)  What architectural changes are needed to satisfy smp and preemption?


Thoughts?
Matt




-------------------------------------------------------
All the advantages of Linux Managed Hosting--Without the Cost and Risk!
Fully trained technicians. The highest number of Red Hat certifications in
the hosting industry. Fanatical Support. Click to learn more
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=107521&bid=248729&dat=121642