Re: Re[2]: Comparison of high-level concurrency models?

MapReduce to some extent is best for "embarrassingly parallel" problems, it seems to me.  More general distributed computing problems might need more sophisticated strategies, though perhaps at some level of abstraction they could still be thought of as MapReduce.

The physics simulations I did in grad school did a very simple partitioning of the physical dimensions the computation grid was laid out on, having somewhat large sub-regions that were each owned by a processor and which would only communicate outside that processor to other processors to exchange information about what was happening at the mutual boundaries between adjacent sub-regions.

We used explicit message passing for the communication (in our case the MPI library, though PVM was also talked about [but discarded because it was proprietary and not available for all platforms]), but there are other ways to go, like the write-only memory scheme used by the DEC machine that AltaVista originally ran on (each processor had a segment of memory that was write-only to other processors and read-only to itself, which was used for communication).


Al


----- Original Message ----
> From: Tanton Gibbs <[email protected]>
> To: [email protected]
> Sent: Tuesday, May 27, 2008 7:47:28 PM
> Subject: Re: Re[2]: [pragprog] Comparison of high-level concurrency models?
> 
> For distributed workflow, you need (and may have) looked at the
> map-reduce model and the open source hadoop implementation.  It is
> being used by yahoo!, facebook, veoh, and others and is a nice, easy
> to understand distributed model.
> 
> Check out http://hadoop.apache.org
> 
> On Tue, May 27, 2008 at 5:10 PM, Gregg Irwin wrote:
> > Hi John,
> >
> >>> I'm looking for a comparison of high-level concurrency models such as
> >>> Actors, Tuplespace, Software Transactional Memory, and others that I
> >>> may not have heard of.
> >
> > JC> And so, since I have burnt and crispy fingers, from playing with
> > JC> concurrency....
> >
> > JC> The only solutions on that I personally will believe in are ones which
> > JC> are mind bogglingly simple... and impose harsh, but simplifying
> > JC> restrictions on the programmers.
> >
> > This being the prag list, and hoping to save my own fingers from as
> > much crispiness as possible, what would be the "harsh, but simplifying
> > restrictions" you would impose, or what models/lanugages worked for
> > you (or didn't)? My context is more distributed workflow oriented than
> > pure HPC, but any thoughts based on your experience would be most
> > welcome.
> >
> > Thanks,
> >
> > --Gregg