Re: OpenSSI & Infiniband

----- Original Message ----- 

> Unfortunately for Quadrics and Myrinet both, their days look numbered,
> and their applications more and more fringe. It's fairly obvious they
> are on the decline now that IB is here.
>
> I can probably help with getting OSSI up again w/ IB. I've been wanting
> to do this for several years, and I think I'll be having some time to
> spend coming up. Let me know how I can help, what others have done thus
> far, etc. Get me up to speed on it's current status.
> Thanks,
> Chris Barry

I'm not so sure of that.

Both quadrics and infiniband are pretty expensive solutions for highend 
networks.
Myri on other hand is real cheap and dominating market right now.

Quadrics is probably together with infiniband scaling very well to large 
networks,
yet quadrics has a very fast one way pingpong latency.

On paper Myri has a very good one too, yet of course scales not so well when 
all
nodes are broadcasting small messages at the same time.

Infinibands one way pingpong latency really is ugly for a highend network 
card.

You just can't use that network for short message software that needs quick 
latency.

So quadrics has the advantages of both, but also a higher price than both 
other.
Price of quadrics is real ugly at card level.

Where the network (which is most expensive at big networks) is similar 
priced,
1 card is soon $1000 a piece at quadrics versus $500 for myri.

Additional it has a shared memory library that's real nice using the shmem 
library.

Yet if you look to price per node it's not that much of a huge price 
compared to the cost of a node itself.

A highend cluster that you'll see with a default outdated P4 dual Xeon 3 Ghz 
"prescott core" that uni's order
soon you're thinking in 5000-6000 dollar a node and adding another 1500 
dollar a node for
a network is not that much of a big pain in that case.

On other hand look who's talking, i'm trying to build some cheapo cluster 
now.

For highend networks bigger than 2 nodes always the problem is: "how to get 
a cheap switch".

So in the end myri outsells them all as it's cheaper a node simply.
Most scientists wouldn't be able to tell their grandmother what network is 
faster than the other.
Basically things are so technical that every manufacturer has a story of his 
own and in the end
what happens is that the company building the cluster will put in of course 
the cheapest highend
network, as they can earn more a node in that case.

That's how real world life works simplistically said.

Bit sad, at the older cards infiniband has 128MB cache, quadrics has 64MB 
cache, Myri has most cards like 4MB cache.

Quite a gap.

Yes there is 8MB versions but those were $1500 a card or so and probably 
some newer card by now released with even more for > $1k.

What will happen of course real soon is that windows will take over which 
sucks. The highend company who manages to get supported by
windows in kernel of windows cluster, is going to sell major league and will 
by means of a low price per port obviously doom all other
manufacturers real soon.

Vincent