Re: understanding 9grid and clustering?

On Dec 7, 2004, at 6:51 PM, andrey mirtchovski wrote:

>> I wonder if it would be useful to spin off threads and have them
>> executing or self-migrating to another node somewhere else on a 
>> cluster
>> or grid where there are more cycles available.
>
> it would be very useful, yes.  difficult to implement too.  you won't
> find much in linux to borrow from either.
>
> i don't think anyone is working at adding support along those lines.

Thank you for entertaining me on this.

I have been imagining a distributed threading library, perhaps called 
dthreads.

(I started wondering about total amount of threads that could be active 
on a CPU server, the term "max thread room" came to mind, and then 
started wondering what sort of runtime library the 1980's pseudo 
animated TV character Max Headroom would have had if he were real.)

Most of the following musing could be prefaced with "perhaps" "maybe" 
and "what if."

The idea would be a superset of the pthreads library, but everything 
would be a distributed thread even if it was local.  Variables and open 
file resources could continue to be shared across node boundaries 
through transparent proxying, but this may not be too bad if using a 
remote file server. Memory protection schemes and debugging traps could 
be used for synchronization

Enhancements to the standard facility would be threads that could:
- be started up on a different nodes
- migrate to other nodes after being preemptively interrupted.
- de/serialize variable and task states, basically swap out, copy to 
another node, swap in again.
- resources can be found and negotiated between threads on other nodes 
that share individual and grouped: CPU cycles, memory, network 
bandwidth, minimal latency, etc.
- heterogeneous threading through use of "fat" binaries as used on NeXT 
and as described at 
http://www.lfbs.rwth-aachen.de/~martin/MetaMPICH/metaframe.html
- allow object and process state introspection, perhaps through 
debugging type facility
- allow automatic thread redundancy.

This would allow sharing of grid resources at a new level.  One could 
time share machines between friends here in Ottawa and back home in 
Melbourne Australia.  Snapshots of processes could be made and 
duplicated or moved from home to iPod to work, all without needing a 
system like VMWare or others like it.  Applications could backtrack on 
a problem solving expedition by doing a thread based equivalent to 
"longjmp()"

I think much of this sort of thing is done already on VMS, but am 
wondering if it has been looked at on Plan 9 or other OSes.

Thanks,

-rh