Re: End-of-line or just carriage-return
"Wes Wagner" <[email protected]> Thu, 19 Jul 2007 06:58:51 -0700
| Newsgroups | gmane.linux.cluster.openmosix.devel |
|---|---|
| Message-ID | <[email protected]> |
--===============0764913141== Content-Type: multipart/alternative; boundary="----=_Part_63871_8682337.1184853531287" ------=_Part_63871_8682337.1184853531287 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline Any objections to a Tuesday 5:30pm meeting? We can call ourselves a Birds of a Feather group and hash out some of these details about openmosix. -Wes On 7/19/07, Matt Rechenburg <[email protected]> wrote: > > Hi Wes, > > Wes Wagner wrote: > > I live in Portland, Oregon and if you are willing to discuss a > > continuation of the open mosix project in another form I would be very > > happy to meet with you and anyone else here who is interested. > > :) good, i am interested. > > > many thanks + all the best, > > Matt > > > > > -Wes Wagner > > > > > > On 7/18/07, *Jonathan Day* <imipak-/[email protected] > > <mailto:imipak-/[email protected]>> wrote: > > > > First off, I'd like to thank Moshe Bar and all the > > other programmers involved in openMosix - I firmly > > believe that it is the best of the SSI solutions out > > there at present. > > > > I also believe that SSI is going to be more important, > > not less, on the next generation of machines. The next > > version of PCI Express supports multiple masters on a > > single network of PCI switches. However, QoS seems to > > be limited to bandwidth reservation for a master to be > > established at setup. There is no way of dynamically > > configuring the next generation PCI-e switches > > according to varying demands. > > > > How does this impact SSI? Well, if you can't move the > > allocation, move the process. If you can migrate > > between masters, you can load-balance a PCI-e system > > in a way that the hardware does not support (by > > design). > > > > There is also the case that programs are usually > > smaller than the data sets being manipulated. In a > > general-purpose cluster, it is sometimes the case that > > shifting the program image is vastly less expensive > > than shifting the data. SIMD would seem to be the > > perfect market for this, because you're guaranteed to > > be running the same code on all nodes, but each node > > will have its own data - which it would therefore need > > to collect, rather than having a central collection > > point. > > > > DSM is a problem - and a problem I have worked hard on > > for the last few years. Part of the problem is that > > when you copy to N nodes using a unicast delivery > > system, you must transmit the data N times. That is > > slow. NACK-Oriented Reliable Multicast would reduce > > the copies needed, cutting bandwidth requirements and > > the total elapsed time between start of first packet > > to end of last packet. > > > > RDMA (Remote Direct Memory Access) is also an > > interesting technology, allowing one node to copy from > > local memory to remote memory, or remote memory to > > local memory, in a single operation without > > intervention by the kernel on either side. However, > > most machines are not provided with RNICs or > > Infiniband cards. Pity. > > > > A project of mine has been to extend RDMA to support > > multicasting - something the specs don't currently > > allow for - so that you can transfer to multiple > > memories in one go. Devising a protocol that the RDMA > > groups can sign on to is proving difficult. There are > > a large number of corner cases and the more I fix, the > > more I find. If RDMA proves impossible, then any > > zero-copy kernel-bypass protocol would work in > > principle. This would avoid having specialized > > hardware but you'd need an emulation layer to provide > > compatibility to the bulk of interesting software. > > > > Multicast with zero-copy does not totally solve the > > problem of DSM, but it reduces some of the issues. You > > shed a whole bunch of context switches, stack > > latencies, buffer copies, backoffs caused by switch > > flooding, etc. It's hard to quantify, but my > > calculations repeatedly show that the latency is > > roughly 5.6 microseconds plus the per hop latency on > > the longest path for every 4K, regardless of the > > number of nodes actually delivered to. > > > > Another networking technology I have been taking a > > close look at is Active Messaging. There's a project - > > GAMMA - that implements AM on Linux. Again, the idea > > is to reduce the wasted cycles involved. AM looks very > > interesting, but the implementation seems to be very > > card-specific and doesn't play nice with other > > protocols. To use AM in a general-purpose cluster > > would need a fair amount of work put into it. > > > > TIPC is also worthy of examination, as anything that > > reduces what a clustering solution needs to do to > > cluster transparently is a Good Thing. > > > > Ok, I've now completely flogged, beaten and trampled > > this poor dead horse into the ground, I'll mutilate a > > few other things, the first of which is the SSI > > concept. > > > > Strictly speaking, openMosix does not need to be SSI. > > Indeed, it would make some sense to use the IBCS > > patches to load and start applications for a range of > > Intel-based *nixes and then migrate the process space > > to an instance of the actual OS it is intended for. We > > now have a multi system image clustering technology. > > > > The second issue is with real-time support. Real-time, > > in this context, means that if process X is guaranteed > > a timeslice of Y out of a total timeslice of Z, with a > > margin of error M, then no matter how time is divided > > up, how busy/free the system is, whether the system > > voluntarily hands back control, etc, it WILL occupy > > that amount of the available time, give or take only > > the margin of error, nothing more. > > > > This can be fun when clustering. You don't hear of > > hard real-time clusters for a reason, and the reason > > has nothing to do with a lack of need. What does > > real-time mean for openMosix? It means that where such > > state information exists, openMosix needs to copy it. > > It means that processes need to migrate according to a > > packing algorithm that maximizes the chances of always > > meeting the timeslice goals. If real-time networking > > is also being used, then you also need the packing > > algorithm to consider network latencies. > > > > Real-time does not mean the fastest solution, which is > > what most SSI strategies look for. Nearest neighbors > > and all that. You will occupy a certain length of time > > even if the transfer takes less, so you must place at > > the greatest distance that the transfers will still > > meet deadlines even under the greatest-expected > > network loads. > > > > Last, but by no means least, if the project is to go > > unmaintained, does anyone object to it being listed on > > the Unmaintained Free Projects pages? > > > > Jonathan > > > > P.S. Ok, I lied, that wasn't last, this is. If there > > are people from this list going to the Open Source > > conference in Portland, Oregon, next week, I suggest > > we arrange some sort of mini meetup. Accepting that > > Moshe has found nobody with the time and ability to > > run the project at this time, I propose that those of > > us who know gifted coders work on finding some, and > > that those who have contacts in the moneyed world see > > if the project can be sponsored somehow to the point > > where a team of developers can hack it to perfection > > on a viable salaried basis. > > > > > > > > > > > ____________________________________________________________________________________ > > Need a vacation? Get great deals > > to amazing places on Yahoo! Travel. > > http://travel.yahoo.com/ > > > > > ------------------------------------------------------------------------- > > This SF.net email is sponsored by DB2 Express > > Download DB2 Express C - the FREE version of DB2 express and take > > control of your XML. No limits. Just data. Click to get it now. > > http://sourceforge.net/powerbar/db2/ > > _______________________________________________ > > openMosix-devel mailing list > > openMosix-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org > > <mailto:openMosix-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org> > > https://lists.sourceforge.net/lists/listinfo/openmosix-devel > > <https://lists.sourceforge.net/lists/listinfo/openmosix-devel> > > > > > > > > > > -- > > Wes Wagner > > > > Join a libertarian network of person-to-person lending on Prosper: > > > > > http://www.prosper.com/groups/group_home.aspx?group_short_name=freelibertarians&referrer=AiriusTorpora&utm_source=referrer-AiriusTorpora&utm_medium=referral-link&utm_content=join_my_group-160x33&utm_campaign=referrals-group > > < > http://www.prosper.com/groups/group_home.aspx?group_short_name=freelibertarians&referrer=AiriusTorpora&utm_source=referrer-AiriusTorpora&utm_medium=referral-link&utm_content=join_my_group-160x33&utm_campaign=referrals-group > > > > > > ------------------------------------------------------------------------ > > > > > ------------------------------------------------------------------------- > > This SF.net email is sponsored by DB2 Express > > Download DB2 Express C - the FREE version of DB2 express and take > > control of your XML. No limits. Just data. Click to get it now. > > http://sourceforge.net/powerbar/db2/ > > ------------------------------------------------------------------------ > > > > _______________________________________________ > > openMosix-devel mailing list > > openMosix-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org > > https://lists.sourceforge.net/lists/listinfo/openmosix-devel > > > > > -- > www.openQRM.org > - Keeps your Data-Center Up and Running > > Matt's blog - http://mattinaction.blogspot.com/ > > > > -- Wes Wagner Join a libertarian network of person-to-person lending on Prosper: http://www.prosper.com/groups/group_home.aspx?group_short_name=freelibertarians&referrer=AiriusTorpora&utm_source=referrer-AiriusTorpora&utm_medium=referral-link&utm_content=join_my_group-160x33&utm_campaign=referrals-group ------=_Part_63871_8682337.1184853531287 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline Any objections to a Tuesday 5:30pm meeting? We can call ourselves a Birds of a Feather group and hash out some of these details about openmosix.<br><br>-Wes<br><br><br><div><span class="gmail_quote">On 7/19/07, <b class="gmail_sendername"> Matt Rechenburg</b> <<a href="mailto:[email protected]">[email protected]</a>> wrote:</span><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"> Hi Wes,<br><br>Wes Wagner wrote:<br>> I live in Portland, Oregon and if you are willing to discuss a<br>> continuation of the open mosix project in another form I would be very<br>> happy to meet with you and anyone else here who is interested. <br><br>:) good, i am interested.<br><br><br>many thanks + all the best,<br><br>Matt<br><br>><br>> -Wes Wagner<br>><br>><br>> On 7/18/07, *Jonathan Day* <<a href="mailto:imipak-/[email protected]">imipak-/[email protected] </a><br>> <mailto:<a href="mailto:imipak-/[email protected]">imipak-/[email protected]</a>>> wrote:<br>><br>> First off, I'd like to thank Moshe Bar and all the<br>> other programmers involved in openMosix - I firmly <br>> believe that it is the best of the SSI solutions out<br>> there at present.<br>><br>> I also believe that SSI is going to be more important,<br>> not less, on the next generation of machines. The next <br>> version of PCI Express supports multiple masters on a<br>> single network of PCI switches. However, QoS seems to<br>> be limited to bandwidth reservation for a master to be<br>> established at setup. There is no way of dynamically <br>> configuring the next generation PCI-e switches<br>> according to varying demands.<br>><br>> How does this impact SSI? Well, if you can't move the<br>> allocation, move the process. If you can migrate <br>> between masters, you can load-balance a PCI-e system<br>> in a way that the hardware does not support (by<br>> design).<br>><br>> There is also the case that programs are usually<br>> smaller than the data sets being manipulated. In a <br>> general-purpose cluster, it is sometimes the case that<br>> shifting the program image is vastly less expensive<br>> than shifting the data. SIMD would seem to be the<br>> perfect market for this, because you're guaranteed to <br>> be running the same code on all nodes, but each node<br>> will have its own data - which it would therefore need<br>> to collect, rather than having a central collection<br>> point.<br>> <br>> DSM is a problem - and a problem I have worked hard on<br>> for the last few years. Part of the problem is that<br>> when you copy to N nodes using a unicast delivery<br>> system, you must transmit the data N times. That is <br>> slow. NACK-Oriented Reliable Multicast would reduce<br>> the copies needed, cutting bandwidth requirements and<br>> the total elapsed time between start of first packet<br>> to end of last packet. <br>><br>> RDMA (Remote Direct Memory Access) is also an<br>> interesting technology, allowing one node to copy from<br>> local memory to remote memory, or remote memory to<br>> local memory, in a single operation without <br>> intervention by the kernel on either side. However,<br>> most machines are not provided with RNICs or<br>> Infiniband cards. Pity.<br>><br>> A project of mine has been to extend RDMA to support <br>> multicasting - something the specs don't currently<br>> allow for - so that you can transfer to multiple<br>> memories in one go. Devising a protocol that the RDMA<br>> groups can sign on to is proving difficult. There are <br>> a large number of corner cases and the more I fix, the<br>> more I find. If RDMA proves impossible, then any<br>> zero-copy kernel-bypass protocol would work in<br>> principle. This would avoid having specialized <br>> hardware but you'd need an emulation layer to provide<br>> compatibility to the bulk of interesting software.<br>><br>> Multicast with zero-copy does not totally solve the<br>> problem of DSM, but it reduces some of the issues. You <br>> shed a whole bunch of context switches, stack<br>> latencies, buffer copies, backoffs caused by switch<br>> flooding, etc. It's hard to quantify, but my<br>> calculations repeatedly show that the latency is <br>> roughly 5.6 microseconds plus the per hop latency on<br>> the longest path for every 4K, regardless of the<br>> number of nodes actually delivered to.<br>><br>> Another networking technology I have been taking a <br>> close look at is Active Messaging. There's a project -<br>> GAMMA - that implements AM on Linux. Again, the idea<br>> is to reduce the wasted cycles involved. AM looks very<br>> interesting, but the implementation seems to be very <br>> card-specific and doesn't play nice with other<br>> protocols. To use AM in a general-purpose cluster<br>> would need a fair amount of work put into it.<br>><br>> TIPC is also worthy of examination, as anything that <br>> reduces what a clustering solution needs to do to<br>> cluster transparently is a Good Thing.<br>><br>> Ok, I've now completely flogged, beaten and trampled<br>> this poor dead horse into the ground, I'll mutilate a <br>> few other things, the first of which is the SSI<br>> concept.<br>><br>> Strictly speaking, openMosix does not need to be SSI.<br>> Indeed, it would make some sense to use the IBCS<br>> patches to load and start applications for a range of <br>> Intel-based *nixes and then migrate the process space<br>> to an instance of the actual OS it is intended for. We<br>> now have a multi system image clustering technology.<br>><br>> The second issue is with real-time support. Real-time, <br>> in this context, means that if process X is guaranteed<br>> a timeslice of Y out of a total timeslice of Z, with a<br>> margin of error M, then no matter how time is divided<br>> up, how busy/free the system is, whether the system <br>> voluntarily hands back control, etc, it WILL occupy<br>> that amount of the available time, give or take only<br>> the margin of error, nothing more.<br>><br>> This can be fun when clustering. You don't hear of <br>> hard real-time clusters for a reason, and the reason<br>> has nothing to do with a lack of need. What does<br>> real-time mean for openMosix? It means that where such<br>> state information exists, openMosix needs to copy it. <br>> It means that processes need to migrate according to a<br>> packing algorithm that maximizes the chances of always<br>> meeting the timeslice goals. If real-time networking<br>> is also being used, then you also need the packing <br>> algorithm to consider network latencies.<br>><br>> Real-time does not mean the fastest solution, which is<br>> what most SSI strategies look for. Nearest neighbors<br>> and all that. You will occupy a certain length of time <br>> even if the transfer takes less, so you must place at<br>> the greatest distance that the transfers will still<br>> meet deadlines even under the greatest-expected<br>> network loads.<br> ><br>> Last, but by no means least, if the project is to go<br>> unmaintained, does anyone object to it being listed on<br>> the Unmaintained Free Projects pages?<br>><br>> Jonathan<br>> <br>> P.S. Ok, I lied, that wasn't last, this is. If there<br>> are people from this list going to the Open Source<br>> conference in Portland, Oregon, next week, I suggest<br>> we arrange some sort of mini meetup. Accepting that <br>> Moshe has found nobody with the time and ability to<br>> run the project at this time, I propose that those of<br>> us who know gifted coders work on finding some, and<br>> that those who have contacts in the moneyed world see <br>> if the project can be sponsored somehow to the point<br>> where a team of developers can hack it to perfection<br>> on a viable salaried basis.<br>><br>><br>><br>><br>> ____________________________________________________________________________________ <br>> Need a vacation? Get great deals<br>> to amazing places on Yahoo! Travel.<br>> <a href="http://travel.yahoo.com/">http://travel.yahoo.com/</a><br>><br>> ------------------------------------------------------------------------- <br>> This SF.net email is sponsored by DB2 Express<br>> Download DB2 Express C - the FREE version of DB2 express and take<br>> control of your XML. No limits. Just data. Click to get it now.<br>> <a href="http://sourceforge.net/powerbar/db2/">http://sourceforge.net/powerbar/db2/</a><br>> _______________________________________________<br>> openMosix-devel mailing list<br>> <a href="mailto:openMosix-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org"> openMosix-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org</a><br>> <mailto:<a href="mailto:openMosix-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org">openMosix-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org</a>><br>> <a href="https://lists.sourceforge.net/lists/listinfo/openmosix-devel"> https://lists.sourceforge.net/lists/listinfo/openmosix-devel</a><br>> <<a href="https://lists.sourceforge.net/lists/listinfo/openmosix-devel">https://lists.sourceforge.net/lists/listinfo/openmosix-devel</a>><br> ><br>><br>><br>><br>> --<br>> Wes Wagner<br>><br>> Join a libertarian network of person-to-person lending on Prosper:<br>><br>> <a href="http://www.prosper.com/groups/group_home.aspx?group_short_name=freelibertarians&referrer=AiriusTorpora&utm_source=referrer-AiriusTorpora&utm_medium=referral-link&utm_content=join_my_group-160x33&utm_campaign=referrals-group"> http://www.prosper.com/groups/group_home.aspx?group_short_name=freelibertarians&referrer=AiriusTorpora&utm_source=referrer-AiriusTorpora&utm_medium=referral-link&utm_content=join_my_group-160x33&utm_campaign=referrals-group </a><br>> <<a href="http://www.prosper.com/groups/group_home.aspx?group_short_name=freelibertarians&referrer=AiriusTorpora&utm_source=referrer-AiriusTorpora&utm_medium=referral-link&utm_content=join_my_group-160x33&utm_campaign=referrals-group"> http://www.prosper.com/groups/group_home.aspx?group_short_name=freelibertarians&referrer=AiriusTorpora&utm_source=referrer-AiriusTorpora&utm_medium=referral-link&utm_content=join_my_group-160x33&utm_campaign=referrals-group </a>><br>><br>> ------------------------------------------------------------------------<br>><br>> -------------------------------------------------------------------------<br>> This SF.net email is sponsored by DB2 Express <br>> Download DB2 Express C - the FREE version of DB2 express and take<br>> control of your XML. No limits. Just data. Click to get it now.<br>> <a href="http://sourceforge.net/powerbar/db2/">http://sourceforge.net/powerbar/db2/ </a><br>> ------------------------------------------------------------------------<br>><br>> _______________________________________________<br>> openMosix-devel mailing list<br>> <a href="mailto:openMosix-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org"> openMosix-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org</a><br>> <a href="https://lists.sourceforge.net/lists/listinfo/openmosix-devel">https://lists.sourceforge.net/lists/listinfo/openmosix-devel</a><br>><br><br><br>--<br><a href="http://www.openQRM.org"> www.openQRM.org</a><br>- Keeps your Data-Center Up and Running<br><br>Matt's blog - <a href="http://mattinaction.blogspot.com/">http://mattinaction.blogspot.com/</a><br><br><br><br></blockquote></div><br><br clear="all"> <br>-- <br>Wes Wagner<br><br>Join a libertarian network of person-to-person lending on Prosper:<br> <br><a href="http://www.prosper.com/groups/group_home.aspx?group_short_name=freelibertarians&referrer=AiriusTorpora&utm_source=referrer-AiriusTorpora&utm_medium=referral-link&utm_content=join_my_group-160x33&utm_campaign=referrals-group"> http://www.prosper.com/groups/group_home.aspx?group_short_name=freelibertarians&referrer=AiriusTorpora&utm_source=referrer-AiriusTorpora&utm_medium=referral-link&utm_content=join_my_group-160x33&utm_campaign=referrals-group </a> ------=_Part_63871_8682337.1184853531287-- --===============0764913141== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline ------------------------------------------------------------------------- This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/ --===============0764913141== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ openMosix-devel mailing list openMosix-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org https://lists.sourceforge.net/lists/listinfo/openmosix-devel --===============0764913141==--