Re: [Openmosix-Devel] Recommendations
David Brodbeck <[email protected]> Wed, 18 Jul 2007 10:12:21 -0700
| Newsgroups | gmane.linux.cluster.openmosix.general |
|---|---|
| Message-ID | <[email protected]> |
--===============1634385218== Content-Type: multipart/alternative; boundary=Apple-Mail-1-313111400 --Apple-Mail-1-313111400 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed (Large CC list trimmed down) On Jul 18, 2007, at 7:24 AM, Wes Wagner wrote: > What we are using openmosix for is genetic programming, which is > cpu intensive, not quite so memory intensive, and produces lots of > single threaded processes since genetic programing is inherently > tons more simple serialized rather than multi-threaded. It is > better to have one thread per organism and just increase your > population sizes to achieve the results you want. > > Now one could load all this into a queuing program - and run it via > remote shell, but the other problem is that your processes run for > a long time. This presents real problems if you want to take a node > offline near immediately, but you don't want to abort the run. Have you looked at Condor? It uses a batch-and-queue mechanism, but if you link your binaries with its libraries it allows checkpointing so processes can be gracefully moved to other nodes. You don't have to rewrite your code, just recompile. On the down side, it's free but not currently open source, if that matters to you. Unlike OpenMosix, though, it's maintained and under active development. We're currently planning a move from OpenMosix to Condor. More and more of our software is written in Java these days, and running Java on OpenMosix presents insurmountable difficulties. We've also been experiencing problems with Python processes segfaulting during migration. As the list of languages I had to tell people to avoid grew, I got more and more pressure from users to find something else. David Brodbeck Information Technology Specialist 3 Computational Linguistics University of Washington --Apple-Mail-1-313111400 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=ISO-8859-1 <HTML><BODY style=3D"word-wrap: break-word; -khtml-nbsp-mode: space; = -khtml-line-break: after-white-space; ">(Large CC list trimmed = down)<DIV><BR><DIV><DIV>On Jul 18, 2007, at 7:24 AM, Wes Wagner = wrote:</DIV><BR class=3D"Apple-interchange-newline"><BLOCKQUOTE = type=3D"cite">What we are using openmosix for is genetic programming, = which is cpu intensive, not quite so memory intensive, and produces lots = of single threaded processes since genetic programing is inherently tons = more simple serialized rather than multi-threaded. It is better to have = one thread per organism and just increase your population sizes to = achieve the results you want. <BR><BR>Now one could load all this into a = queuing program - and run it via remote shell, but the other problem is = that your processes run for a long time. This presents real problems if = you want to take a node offline near immediately, but you don't want to = abort the run.<BR></BLOCKQUOTE><DIV><BR = class=3D"khtml-block-placeholder"></DIV><DIV>Have you looked at Condor?=A0= It uses a batch-and-queue mechanism, but if you link your binaries with = its libraries it allows checkpointing so processes can be gracefully = moved to other nodes.=A0 You don't have to rewrite your code, just = recompile.=A0 On the down side, it's free but not currently open source, = if that matters to you.=A0 Unlike OpenMosix, though, it's maintained and = under active development.</DIV><DIV><BR = class=3D"khtml-block-placeholder"></DIV><DIV>We're currently planning a = move from OpenMosix to Condor.=A0 More and more of our software is = written in Java these days, and running Java on OpenMosix presents = insurmountable difficulties.=A0 We've also been experiencing problems = with Python processes segfaulting during migration.=A0 As the list of = languages I had to tell people to avoid grew, I got more and more = pressure from users to find something else.</DIV><DIV><BR = class=3D"khtml-block-placeholder"></DIV></DIV><BR><DIV> <SPAN = class=3D"Apple-style-span" style=3D"border-collapse: separate; = border-spacing: 0px 0px; color: rgb(0, 0, 0); font-family: Helvetica; = font-size: 12px; font-style: normal; font-variant: normal; font-weight: = normal; letter-spacing: normal; line-height: normal; text-align: auto; = -khtml-text-decorations-in-effect: none; text-indent: 0px; = -apple-text-size-adjust: auto; text-transform: none; orphans: 2; = white-space: normal; widows: 2; word-spacing: 0px; "><SPAN = class=3D"Apple-style-span" style=3D"border-collapse: separate; = border-spacing: 0px 0px; color: rgb(0, 0, 0); font-family: Helvetica; = font-size: 12px; font-style: normal; font-variant: normal; font-weight: = normal; letter-spacing: normal; line-height: normal; text-align: auto; = -khtml-text-decorations-in-effect: none; text-indent: 0px; = -apple-text-size-adjust: auto; text-transform: none; orphans: 2; = white-space: normal; widows: 2; word-spacing: 0px; "><DIV>David = Brodbeck</DIV><DIV>Information Technology Specialist = 3</DIV><DIV>Computational Linguistics</DIV><DIV>University of = Washington</DIV><BR class=3D"Apple-interchange-newline"></SPAN></SPAN> = </DIV><BR></DIV></BODY></HTML>= --Apple-Mail-1-313111400-- --===============1634385218== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline ------------------------------------------------------------------------- This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/ --===============1634385218== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ openMosix-general mailing list openMosix-general-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org https://lists.sourceforge.net/lists/listinfo/openmosix-general --===============1634385218==--