Re: Looking for people interested in factoring.
"John L. Bass" <jbass-/[email protected]> Mon, 22 Mar 2010 20:56:32 -0600
| Newsgroups | gmane.comp.distributed-net.hardware |
|---|---|
| Message-ID | <[email protected]> |
This is a multi-part message in MIME format. --===============1592093812968291665== Content-Type: multipart/alternative; boundary="------------020709030901090108080003" This is a multi-part message in MIME format. --------------020709030901090108080003 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit On 03/22/2010 06:12 PM, Martin K wrote: > John, > I'm interested in helping. I'm a bit more of a hardware and low-level > 'expert' rather than a computer scientist. I also have no involvement > with d.net if that's what you're looking for specifically. I am, > however; looking for a project to work on that isn't writing 8 bit > assembly code or dealing with ultrasonic communication. > If you have something I can read up on, let me know. > - > Martin K > Hi Martin, This one is unfortunately more of a computer science problem. I wrote hardware simply because I didn't find any other general d.net mailing list. My two clusters are 85 P3 1.26GHz cpus (512k caches), and 34 Itanium 1.6GHz cpus (6, 9, 24M caches), with about 130GB of Dram total in various amounts per machine, all sharing a 5TB SAN (plus local disks) with both Fibre channel and gigabit ethernet switches. My last test in 2006 took about 60 hours to exhaust the resources on a majority of the cluster nodes, with some active load balancing. I think the next re-write will reduce working set requirements by a factor of about 3, but to complete the RSA-706 project in reasonable time is likely to require a cluster about 5 times my size. Depending on the CPU and FSB there is a performance penalty of a factor of 10-100X for cache misses -- DRAM is MUCH slower than the CPU, so having enough CPU cache to hold the aggregate working set is important. Literally the difference between 1 week, and 30 weeks, for the same working set and computation, possibly a lot worse after communications costs. So the d.net usefulness is possible access to many hundreds of cpu caches, with a possible highly distributed communications architecture that might avoid single point network bottlenecks. Otherwise I need to live inside my clusters resources, where communications are a LOT faster. The biggest problem I see, is that the work units are not independent, and the problem is likely to require a communications network between nodes with an average fan out of about 4-5 connections per node. I haven't yet thought of a low bandwidth way to have completely independent work assignments like RC5 project had. This algorithm doesn't partition as nicely as RC5 or the sieve/matrix factoring algorithms, in terms of node independence ... it's unfortunately better suited for local clusters with large CPU caches and gigabit ethernet using an MPI communications model. On the plus side, is that it scales very well inside local clusters with fast networking switches. Pushed onto the open internet, the number of nodes will have to scale up to obtain the equivalent aggregate bandwidth. Factoring big numbers isn't "just" easy ... something like the remaining RC5 problems. John --------------020709030901090108080003 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> <html> <head> <meta content="text/html; charset=ISO-8859-1" http-equiv="Content-Type"> </head> <body text="#000000" bgcolor="#ffffff"> On 03/22/2010 06:12 PM, Martin K wrote: <blockquote cite="mid:[email protected]" type="cite"> <pre wrap="">John, I'm interested in helping. I'm a bit more of a hardware and low-level 'expert' rather than a computer scientist. I also have no involvement with d.net if that's what you're looking for specifically. I am, however; looking for a project to work on that isn't writing 8 bit assembly code or dealing with ultrasonic communication. If you have something I can read up on, let me know. - Martin K </pre> </blockquote> <font face="Fixed"><br> Hi Martin,<br> <br> This one is unfortunately more of a computer science problem. I wrote hardware simply because I didn't find any other general d.net mailing list.<br> <br> My two clusters are 85 P3 1.26GHz cpus (512k caches), and 34 Itanium 1.6GHz cpus (6, 9, 24M caches), with about 130GB of Dram total in various amounts per machine, all sharing a 5TB SAN (plus local disks) with both Fibre channel and gigabit ethernet switches.<br> <br> My last test in 2006 took about 60 hours to exhaust the resources on a majority of the cluster nodes, with some active load balancing.<br> <br> I think the next re-write will reduce working set requirements by a factor of about 3, but to complete the RSA-706 project in reasonable time is likely to require a cluster about 5 times my size. Depending on the CPU and FSB there is a performance penalty of a factor of 10-100X for cache misses -- DRAM is MUCH slower than the CPU, so having enough CPU cache to hold the aggregate working set is important. Literally the difference between 1 week, and 30 weeks, for the same working set and computation, possibly a lot worse after communications costs. So the d.net usefulness is possible access to many hundreds of cpu caches, with a possible highly distributed communications architecture that might avoid single point network bottlenecks. Otherwise I need to live inside my clusters resources, where communications are a LOT faster.<br> <br> The biggest problem I see, is that the work units are not independent, and the problem is likely to require a communications network between nodes with an average fan out of about 4-5 connections per node. I haven't yet thought of a low bandwidth way to have completely independent work assignments like RC5 project had.<br> <br> This algorithm doesn't partition as nicely as RC5 or the sieve/matrix factoring algorithms, in terms of node independence ... it's unfortunately better suited for local clusters with large CPU caches and gigabit ethernet using an MPI communications model. On the plus side, is that it scales very well inside local clusters with fast networking switches. Pushed onto the open internet, the number of nodes will have to scale up to obtain the equivalent aggregate bandwidth.<br> <br> Factoring big numbers isn't "just" easy ... something like the remaining RC5 problems.<br> <br> John<br> </font> </body> </html> --------------020709030901090108080003-- --===============1592093812968291665== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Hardware mailing list Hardware-Ra3b/QYEcJ3d140v2zMXi0fjHoOT/h/[email protected] http://lists.distributed.net/mailman/listinfo/hardware --===============1592093812968291665==--