Re: ssi on podcast
Roger Tsang <[email protected]> Sun, 2 Sep 2012 23:08:51 -0400
| Newsgroups | gmane.linux.cluster.ssic.user |
|---|---|
| Message-ID | <CALTdT_OdSCWaoZ+9PBF0wgFDDJawWLtSGv-pHvd=J=eE4Jk-KA@mail.gmail.com> |
--===============3576866052058881425==
Content-Type: multipart/alternative; boundary=485b397dd301069b3a04c8c37686
--485b397dd301069b3a04c8c37686
Content-Type: text/plain; charset=ISO-8859-1
FYI: http://wiki.openssi.org/go/Features
OpenSSI has been known to support Infiniband (IP over IB). Stan Smith has
used OpenSSI with Infiniband. However our CVS code respository at
SourceForge contains a new optimization for CFS over TCP/IP - an internal
CFS read/write cache but this optimization can be disabled for Infiniband.
SHM is not affected.
OpenSSI provides a single IPC namespace. If I remember correctly, the
single namespace means shared memory segments are accessible cluster-wide
(see below). These SHM segments do migrate but only during process
migration of the owning process. The load-balancing of just SHM segments
(without process migration) is not implemented.
In OpenSSI, procfs provides an extra column: node_num. All IPC entities in
the cluster are visible.
$ cat /proc/sysvipc/shm
key shmid perms size cpid lpid nattch uid gid cuid
cgid atime dtime ctime view node_num
0 1671168 1666 294912 200617 200620 10 0 0
0 0 1346607713 1346607607 1346607603 default 3
0 1572876 1666 294912 69603 69606 9 0 0 0
0 1346607713 1346587446 1346473273 default 1
$ cat /proc/sysvipc/sem
key semid perms nsems uid gid cuid cgid
otime ctime view node_num
0 4947968 666 1 0 0 0 0 1346639170
1346607607 default 3
0 3997698 666 1 0 0 0 0 1346639050
1346587447 default 1
On Sun, Aug 26, 2012 at 5:46 PM, Vincent Diepeveen <[email protected]> wrote:
> Brock,
>
> If we may be honest about SSI in general:
>
> For my 8 node $200 a box cluster having a SSI would be really GREAT.
> And i can't use big enough capital letters for it.
>
> For HPC with big supercomputers it's total nonsense to do things in a
> semi-centralized manner.
>
> Yet where it is interesting to make 1 big supercomputer out of my
> $3200 set of 8 machines
> which eat together by the way around 1.4 kilowatt under full load, so
> that's similar to a
> 8 socket machine of $200k.
>
> Now latest 8 socket machines sure are faster than this 64 cores i
> have here,
> yet realize the price i paid for them.
>
> So it would be as powerful nearly as a 8 socket machine
> given the right software and decent interconnects.
>
> So you want the choice of which interconnect to use, to take that
> yourself.
>
> Let me assure you that built in ethernet cards are not useful to
> combine into a chessmachine. The latency
> is too bad of the cheap ethernet cards (sure solarlare at $1000 a
> card will work, but that's my point exactly).
>
> So where the price of the network is already a problem, you sure
> don't want to pay big bucks for the software.
>
> So software that makes 1 SSI out of it, that IS interesting, provided
> it is cheap. Anything that's $xxx a port is
> nonsense to use. In fact even $xx a node already is nonsense
> considering that infiniband software is for free
> and also networks from the past usually, not all of them, spreaded
> the software for free.
>
> A great latency interconnect was for example Quadrics. I still have
> an old network here. Bandwidth is a joke of it
> compared to todays pci-e cards, but latency still is beating the
> fastest and most expensive TCP solution on the planet.
> Software was for free downloadable. I still have that here somewhere.
>
> OpenSSI was for free, that is what is good of it. Yet look what
> distro's it used to work for. Same with OpenMosix.
> They never really were compatible with HPC networks that usually work
> at different distro's with different kernel requirements.
>
> Fixing that costs money. Open source software is great, yet usually
> it's paid persons who maintain / carry it out.
> Someone needs to pay that maintenance bill. I'm not, and as it
> appears no one is willing to do that.
>
> Yet where having a small SSI using free software, the promise that
> OpenSSI delivered, it appears to be
> the pipedream that never matured. It appears no one is prepared to
> fund some people to build this.
>
> Now that's sad, but it's the reality of the market simply.
>
> Let's just move on instead of giving attention to the few who still
> try to make big bucks based upon something that's
> easy to program for and with, yet which simply always was too expensive.
>
> Price matters.
>
>
>
>
>
>
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Ssic-linux-users mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/ssic-linux-users
>
--485b397dd301069b3a04c8c37686
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
FYI: <a href=3D"http://wiki.openssi.org/go/Features">http://wiki.openssi.or=
g/go/Features</a><br><br>OpenSSI has been known to support Infiniband (IP o=
ver IB).=A0 Stan Smith has used OpenSSI with Infiniband.=A0 However our CVS=
code respository at SourceForge contains a new optimization for CFS over T=
CP/IP - an internal CFS read/write cache but this optimization can be disab=
led for Infiniband.=A0 SHM is not affected.<br>
<br>OpenSSI provides a single IPC namespace.=A0 If I remember correctly, th=
e single namespace means shared memory segments are accessible cluster-wide=
(see below).=A0 These SHM segments do migrate but only during process migr=
ation of the owning process.=A0 The load-balancing of just SHM segments (wi=
thout process migration) is not implemented.<br>
<br>In OpenSSI, procfs provides an extra column: node_num.=A0 All IPC entit=
ies in the cluster are visible.<br><br>$ cat /proc/sysvipc/shm<br>=A0=A0 ke=
y=A0=A0=A0=A0=A0 shmid perms=A0=A0=A0=A0=A0=A0 size=A0 cpid=A0 lpid nattch=
=A0=A0 uid=A0=A0 gid=A0 cuid=A0 cgid=A0=A0=A0=A0=A0 atime=A0=A0=A0=A0=A0 dt=
ime=A0=A0=A0=A0=A0 ctime=A0=A0=A0=A0=A0=A0 view=A0=A0 node_num<br>
=A0=A0=A0=A0=A0=A0=A0=A0 0=A0=A0=A0 1671168=A0 1666=A0=A0=A0=A0 294912 2006=
17 200620=A0=A0=A0=A0 10=A0=A0=A0=A0 0=A0=A0=A0=A0 0=A0=A0=A0=A0 0=A0=A0 0 =
1346607713 1346607607 1346607603=A0=A0=A0 default=A0=A0=A0=A0=A0=A0=A0=A0=
=A0 3<br>=A0=A0=A0=A0=A0=A0=A0=A0 0=A0=A0=A0 1572876=A0 1666=A0=A0=A0=A0 29=
4912 69603 69606=A0=A0=A0=A0=A0 9=A0=A0=A0=A0 0=A0=A0=A0=A0 0=A0=A0=A0=A0 0=
0 1346607713 1346587446 1346473273=A0=A0=A0 default=A0=A0=A0=A0=A0=A0=A0=
=A0=A0 1<br>
<br>$ cat /proc/sysvipc/sem<br>=A0=A0=A0=A0=A0=A0 key=A0=A0=A0=A0=A0 semid =
perms=A0=A0=A0=A0=A0 nsems=A0=A0 uid=A0=A0 gid=A0 cuid=A0 cgid=A0=A0=A0=A0=
=A0 otime=A0=A0=A0=A0=A0 ctime=A0=A0=A0=A0=A0=A0 view=A0=A0=A0 node_num<br>=
=A0=A0=A0=A0=A0=A0=A0=A0 0=A0=A0=A0 4947968=A0=A0 666=A0=A0=A0=A0=A0=A0=A0=
=A0=A0 1=A0=A0=A0=A0 0=A0=A0=A0=A0 0=A0=A0=A0=A0 0=A0=A0=A0=A0 0 1346639170=
1346607607=A0=A0=A0 default=A0=A0=A0=A0=A0=A0=A0=A0=A0 3<br>
=A0=A0=A0=A0=A0=A0=A0=A0 0=A0=A0=A0 3997698=A0=A0 666=A0=A0=A0=A0=A0=A0=A0=
=A0=A0 1=A0=A0=A0=A0 0=A0=A0=A0=A0 0=A0=A0=A0=A0 0=A0=A0=A0=A0 0 1346639050=
1346587447=A0=A0=A0 default=A0=A0=A0=A0=A0=A0=A0=A0=A0 1<br><br><br><br><d=
iv class=3D"gmail_quote">On Sun, Aug 26, 2012 at 5:46 PM, Vincent Diepeveen=
<span dir=3D"ltr"><<a href=3D"mailto:[email protected]" target=3D"_blank">=
[email protected]</a>></span> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">Brock,<br>
<br>
If we may be honest about SSI in general:<br>
<br>
For my 8 node $200 a box cluster having a SSI would be really GREAT.<br>
And i can't use big enough capital letters for it.<br>
<br>
For HPC with big supercomputers it's total nonsense to do things in a<b=
r>
semi-centralized manner.<br>
<br>
Yet where it is interesting to make 1 big supercomputer out of my<br>
$3200 set of 8 machines<br>
which eat together by the way around 1.4 kilowatt under full load, so<br>
that's similar to a<br>
8 socket machine of $200k.<br>
<br>
Now latest 8 socket machines sure are faster than this 64 cores i<br>
have here,<br>
yet realize the price i paid for them.<br>
<br>
So it would be as powerful nearly as a 8 socket machine<br>
given the right software and decent interconnects.<br>
<br>
So you want the choice of which interconnect to use, to take that<br>
yourself.<br>
<br>
Let me assure you that built in ethernet cards are not useful to<br>
combine into a chessmachine. The latency<br>
is too bad of the cheap ethernet cards (sure solarlare at $1000 a<br>
card will work, but that's my point exactly).<br>
<br>
So where the price of the network is already a problem, you sure<br>
don't want to pay big bucks for the software.<br>
<br>
So software that makes 1 SSI out of it, that IS interesting, provided<br>
it is cheap. Anything that's $xxx a port is<br>
nonsense to use. In fact even $xx a node already is nonsense<br>
considering that infiniband software is for free<br>
and also networks from the past usually, not all of them, spreaded<br>
the software for free.<br>
<br>
A great latency interconnect was for example Quadrics. I still have<br>
an old network here. Bandwidth is a joke of it<br>
compared to todays pci-e cards, but latency still is beating the<br>
fastest and most expensive TCP solution on the planet.<br>
Software was for free downloadable. I still have that here somewhere.<br>
<br>
OpenSSI was for free, that is what is good of it. Yet look what<br>
distro's it used to work for. Same with OpenMosix.<br>
They never really were compatible with HPC networks that usually work<br>
at different distro's with different kernel requirements.<br>
<br>
Fixing that costs money. Open source software is great, yet usually<br>
it's paid persons who maintain / carry it out.<br>
Someone needs to pay that maintenance bill. I'm not, and as it<br>
appears no one is willing to do that.<br>
<br>
Yet where having a small SSI using free software, the promise that<br>
OpenSSI delivered, it appears to be<br>
the pipedream that never matured. It appears no one is prepared to<br>
fund some people to build this.<br>
<br>
Now that's sad, but it's the reality of the market simply.<br>
<br>
Let's just move on instead of giving attention to the few who still<br>
try to make big bucks based upon something that's<br>
easy to program for and with, yet which simply always was too expensive.<br=
>
<br>
Price matters.<br>
<div class=3D"HOEnZb"><div class=3D"h5"><br>
<br>
<br>
<br>
<br>
<br>
---------------------------------------------------------------------------=
---<br>
Live Security Virtual Conference<br>
Exclusive live event will cover all the ways today's security and<br>
threat landscape has changed and how IT managers can respond. Discussions<b=
r>
will include endpoint security, mobile security and the latest in malware<b=
r>
threats. <a href=3D"http://www.accelacomm.com/jaw/sfrnl04242012/114/5012226=
3/" target=3D"_blank">http://www.accelacomm.com/jaw/sfrnl04242012/114/50122=
263/</a><br>
_______________________________________________<br>
Ssic-linux-users mailing list<br>
<a href=3D"mailto:[email protected]">Ssic-linux-users@=
lists.sourceforge.net</a><br>
<a href=3D"https://lists.sourceforge.net/lists/listinfo/ssic-linux-users" t=
arget=3D"_blank">https://lists.sourceforge.net/lists/listinfo/ssic-linux-us=
ers</a><br>
</div></div></blockquote></div><br>
--485b397dd301069b3a04c8c37686--
--===============3576866052058881425==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
--===============3576866052058881425==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
_______________________________________________
Ssic-linux-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ssic-linux-users
--===============3576866052058881425==--