Re: End-of-line or just carriage-return
Matt Rechenburg <[email protected]> Thu, 19 Jul 2007 11:07:20 +0200
| Newsgroups | gmane.linux.cluster.openmosix.devel |
|---|---|
| Message-ID | <[email protected]> |
Hi Wes, Wes Wagner wrote: > I live in Portland, Oregon and if you are willing to discuss a > continuation of the open mosix project in another form I would be very > happy to meet with you and anyone else here who is interested. :) good, i am interested. many thanks + all the best, Matt > > -Wes Wagner > > > On 7/18/07, *Jonathan Day* <imipak-/[email protected] > <mailto:imipak-/[email protected]>> wrote: > > First off, I'd like to thank Moshe Bar and all the > other programmers involved in openMosix - I firmly > believe that it is the best of the SSI solutions out > there at present. > > I also believe that SSI is going to be more important, > not less, on the next generation of machines. The next > version of PCI Express supports multiple masters on a > single network of PCI switches. However, QoS seems to > be limited to bandwidth reservation for a master to be > established at setup. There is no way of dynamically > configuring the next generation PCI-e switches > according to varying demands. > > How does this impact SSI? Well, if you can't move the > allocation, move the process. If you can migrate > between masters, you can load-balance a PCI-e system > in a way that the hardware does not support (by > design). > > There is also the case that programs are usually > smaller than the data sets being manipulated. In a > general-purpose cluster, it is sometimes the case that > shifting the program image is vastly less expensive > than shifting the data. SIMD would seem to be the > perfect market for this, because you're guaranteed to > be running the same code on all nodes, but each node > will have its own data - which it would therefore need > to collect, rather than having a central collection > point. > > DSM is a problem - and a problem I have worked hard on > for the last few years. Part of the problem is that > when you copy to N nodes using a unicast delivery > system, you must transmit the data N times. That is > slow. NACK-Oriented Reliable Multicast would reduce > the copies needed, cutting bandwidth requirements and > the total elapsed time between start of first packet > to end of last packet. > > RDMA (Remote Direct Memory Access) is also an > interesting technology, allowing one node to copy from > local memory to remote memory, or remote memory to > local memory, in a single operation without > intervention by the kernel on either side. However, > most machines are not provided with RNICs or > Infiniband cards. Pity. > > A project of mine has been to extend RDMA to support > multicasting - something the specs don't currently > allow for - so that you can transfer to multiple > memories in one go. Devising a protocol that the RDMA > groups can sign on to is proving difficult. There are > a large number of corner cases and the more I fix, the > more I find. If RDMA proves impossible, then any > zero-copy kernel-bypass protocol would work in > principle. This would avoid having specialized > hardware but you'd need an emulation layer to provide > compatibility to the bulk of interesting software. > > Multicast with zero-copy does not totally solve the > problem of DSM, but it reduces some of the issues. You > shed a whole bunch of context switches, stack > latencies, buffer copies, backoffs caused by switch > flooding, etc. It's hard to quantify, but my > calculations repeatedly show that the latency is > roughly 5.6 microseconds plus the per hop latency on > the longest path for every 4K, regardless of the > number of nodes actually delivered to. > > Another networking technology I have been taking a > close look at is Active Messaging. There's a project - > GAMMA - that implements AM on Linux. Again, the idea > is to reduce the wasted cycles involved. AM looks very > interesting, but the implementation seems to be very > card-specific and doesn't play nice with other > protocols. To use AM in a general-purpose cluster > would need a fair amount of work put into it. > > TIPC is also worthy of examination, as anything that > reduces what a clustering solution needs to do to > cluster transparently is a Good Thing. > > Ok, I've now completely flogged, beaten and trampled > this poor dead horse into the ground, I'll mutilate a > few other things, the first of which is the SSI > concept. > > Strictly speaking, openMosix does not need to be SSI. > Indeed, it would make some sense to use the IBCS > patches to load and start applications for a range of > Intel-based *nixes and then migrate the process space > to an instance of the actual OS it is intended for. We > now have a multi system image clustering technology. > > The second issue is with real-time support. Real-time, > in this context, means that if process X is guaranteed > a timeslice of Y out of a total timeslice of Z, with a > margin of error M, then no matter how time is divided > up, how busy/free the system is, whether the system > voluntarily hands back control, etc, it WILL occupy > that amount of the available time, give or take only > the margin of error, nothing more. > > This can be fun when clustering. You don't hear of > hard real-time clusters for a reason, and the reason > has nothing to do with a lack of need. What does > real-time mean for openMosix? It means that where such > state information exists, openMosix needs to copy it. > It means that processes need to migrate according to a > packing algorithm that maximizes the chances of always > meeting the timeslice goals. If real-time networking > is also being used, then you also need the packing > algorithm to consider network latencies. > > Real-time does not mean the fastest solution, which is > what most SSI strategies look for. Nearest neighbors > and all that. You will occupy a certain length of time > even if the transfer takes less, so you must place at > the greatest distance that the transfers will still > meet deadlines even under the greatest-expected > network loads. > > Last, but by no means least, if the project is to go > unmaintained, does anyone object to it being listed on > the Unmaintained Free Projects pages? > > Jonathan > > P.S. Ok, I lied, that wasn't last, this is. If there > are people from this list going to the Open Source > conference in Portland, Oregon, next week, I suggest > we arrange some sort of mini meetup. Accepting that > Moshe has found nobody with the time and ability to > run the project at this time, I propose that those of > us who know gifted coders work on finding some, and > that those who have contacts in the moneyed world see > if the project can be sponsored somehow to the point > where a team of developers can hack it to perfection > on a viable salaried basis. > > > > > ____________________________________________________________________________________ > Need a vacation? Get great deals > to amazing places on Yahoo! Travel. > http://travel.yahoo.com/ > > ------------------------------------------------------------------------- > This SF.net email is sponsored by DB2 Express > Download DB2 Express C - the FREE version of DB2 express and take > control of your XML. No limits. Just data. Click to get it now. > http://sourceforge.net/powerbar/db2/ > _______________________________________________ > openMosix-devel mailing list > openMosix-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org > <mailto:openMosix-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org> > https://lists.sourceforge.net/lists/listinfo/openmosix-devel > <https://lists.sourceforge.net/lists/listinfo/openmosix-devel> > > > > > -- > Wes Wagner > > Join a libertarian network of person-to-person lending on Prosper: > > http://www.prosper.com/groups/group_home.aspx?group_short_name=freelibertarians&referrer=AiriusTorpora&utm_source=referrer-AiriusTorpora&utm_medium=referral-link&utm_content=join_my_group-160x33&utm_campaign=referrals-group > <http://www.prosper.com/groups/group_home.aspx?group_short_name=freelibertarians&referrer=AiriusTorpora&utm_source=referrer-AiriusTorpora&utm_medium=referral-link&utm_content=join_my_group-160x33&utm_campaign=referrals-group> > > ------------------------------------------------------------------------ > > ------------------------------------------------------------------------- > This SF.net email is sponsored by DB2 Express > Download DB2 Express C - the FREE version of DB2 express and take > control of your XML. No limits. Just data. Click to get it now. > http://sourceforge.net/powerbar/db2/ > ------------------------------------------------------------------------ > > _______________________________________________ > openMosix-devel mailing list > openMosix-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org > https://lists.sourceforge.net/lists/listinfo/openmosix-devel > -- www.openQRM.org - Keeps your Data-Center Up and Running Matt's blog - http://mattinaction.blogspot.com/ ------------------------------------------------------------------------- This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/