Re: Stubs and wrappers and possibilities, oh my!

On Wed, Jul 25, 2007 at 01:54:39PM -0700, Jonathan Day wrote:
> One of the things examined was the use of a stub that
> was not migratable between nodes. I was asked to write
> a short piece describing a proposed modification to
> this, so here it is. Note that suggestions, comments
> and clue-stick beatings are welcome, but if you could
> pad the clue-stick a little, I'd be grateful.
> 
> Brief Synopsis: Linux offers a bazillion features,
> network engineers have written a bazillion others. The
> I/O-side stub, as it is, works great but there may be
> ways to make it more powerful and/or faster. We should
> consider if this would be a useful/good thing to do,
> and if so what our priorities should be.
> 
> 
> My suggestion is to first extend the stub. Linux has a
> whole host of new features which should expand the
> usefulness of the stub, and it would be good to
> discuss what we'd want the stub to actually do.
> 
> The second step I proposed was to allow the stub to
> migrate. If the stub migrates, then all user-side I/O
> would migrate with it to the new node, which must
> (therefore) support the same I/O features as are being
> used. So long as the information is available, I don't
> see that being a problem.

Are you understanding why the stubs are not migratable ?

> (By migrating the stub rather than just the data, you
> carry all of the process state, security/permission
> labels, etc, for free and you can reuse existing
> code.)
>
> The problem lies in how to migrate the stubs safely
> and maintain contact with the controlled process,
> without adding any security complications and with
> taking into account any data sent by the process
> during the migration.
> 
> Linux - after applying a couple of patches - provides
> methods for supporting the failover of network
> connections, transient addressing, network mobility,
> kernel-level switching for layers 2, 3, 4 and 7, and
> virtual network devices. In short, we have so many
> ways of getting a connection to switch that it's more
> a case of what method we should use rather than
> whether we can do it.
> 
> A worthy place to look for inspiration is the research
> that has gone into IPv6 mobility. The current version
> uses a transient home base that all connections link
> to. The home base then talks to the user node. When
> the user node moves, a redirect is sent through the
> network. Any connection not yet redirected bounces off
> the home base to wherever the user is. Redirected
> connections go directly to the user. Depending on the
> version of the mobility draft, once all connections
> are redirected, the user becomes the new home base and
> the old home base is closed down.

Now this is really getting into a joke right ?
IPv6 mobility is not a solution to any problem.
an IP/IPv6 is an OS ressource, that can be multiplex (ports)
across all the processes over the OS.

> Alternatively, why keep track at all? Have a
> process-side stub that listens for anycasts. When the
> user-side stub migrates, it sends out an anycast
> request for the UPID it was connected to. The process'
> stub will be the only one to respond, so the
> connection can then be restored.

right so, you are putting a "process side stub" that listen for
anycast to replace the stub, so that the stub can migrate..
look like to me the stubs is actually STAYING to do that, you're just
replacing one stub by another one that does the same thing.

> What about kernel threads? Again, same thing applies.
> We're not short of methods. Everything from HA
> solutions to suspend-to-file would allow you to
> transfer kernel-level operations from one machine to
> another without problem.

if you're actually talking about freezing kernel thread, it's not
realistic. actually linux kernel developers are removing the cooperation
with the suspend freezer right now (2.6.23) which going to be trickier.
Plus a kernel thread has load of private stuff that CANNOT be migrated
and tight to the OS.

> Is there anything significant that's needed that's
> new? Well, the stub should export to userspace any
> information needed to perform migration and perhaps
> resource allocation. At the meeting, we thought about
> various strategies for this. /proc and /sys were
> suggested, as was using a page of shared memory. I'd
> love to hear other people's thoughts on this.

migration passing by userspace process is possible for sending data, but
that's opening a whole can of worms for other communication (syscalls
for example).

> I also suggested that since Linux now supports
> real-time and that time-slicing is much more
> predictable under new scheduler, we might want to
> consider all processes under the control of OpenMosix
> as real-time. Each process then has a fixed-sized
> bucket of time, and the scheduler can use a
> conventional packing herustic to pack those buckets
> into the cluster.

hmm ? the processes are correctly tight to the linux scheduler, thanks
you very much. I don't know why are you talking about real-time, since it's
obviously a per process "state". What are you winning by giving
real-time to all tasks and opening a really bad remote Denial-of-Service ?

> How is this better than using, say, the load average?
> The load average doesn't tell you what's taking the
> time or why, or whether the remaining time is usable.
> (I'm not certain how much you can fragment the block
> of time that the new scheduler uses to timeslice, but
> I can be pretty certain that any desktop system is
> going to have enough events to fragment the time
> block, leaving unusable chunks.)
> 
> Ok, so we want to export the timeslice requirements.
> What else? Any universal ID would be good to export,
> along with what machine the process is running on. We
> also need to export memory tags, as RDMA should not be
> done in the kernel.
> 
> Why not export everything? Because 99% of what is on
> the stub is strictly for I/O to other kernel-based
> components. Very little is needed or wanted by
> userspace, and the less you have to copy, track or
> update, the less latency you'll suffer and the lower
> your requirements. You also get better security.
> 
> This ends the reading of the stub.

Except reading random crazy ideas, I don't see the shadow of a solid
proposal on what to do. This proposal is far too generic (and crazy) and
doesn't propose any solutions..

-- 
Vincent Hanquez

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/