Re: OpenSSI Project Status
"Stan C Smith" <[email protected]> Sun, 7 Aug 2016 11:17:13 -0700
| Newsgroups | gmane.linux.cluster.ssic.devel |
|---|---|
| Message-ID | <[email protected]> |
This is a multipart message in MIME format. --===============6625671879866027496== Content-Type: multipart/alternative; boundary="----=_NextPart_000_011C_01D1F09D.3FD39E60" Content-Language: en-us This is a multipart message in MIME format. ------=_NextPart_000_011C_01D1F09D.3FD39E60 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable OpenSSI is a very interesting technology for its time: Locus Computing, = Intel, Transmeta and HP labs all gave the SSI (Single-System-Image) code = base a chance. =20 The challenges which held back OpenSSI acceptance are: =C2=B7 Fear of SSI model when OpenSSI RPCs have no = =E2=80=98real=E2=80=99 failure/recovery mechanism; if interconnect fails = RPC hangs; that said the predecessor code of OpenSSI powered Intel = Paragon supercomputers @ 2000+ nodes, in addition to the TFLOPS machine = service/IO nodes, large IBM and Transmeta systems. OpenSSI can shine = when your interconnect is reliable. =C2=B7 Issues upstreaming Linux kernel patches: o CFS filesystem was disliked by the Linux community; context specific = symlinks were disliked by Linus; not clear on other CFS aspects which = were disliked. o rfork(), rexec(), rvfork() were abused by Linux community as the = implementation was built on RPCs which had to no clean failure recovery. o Linux-2.6.18 + some x64 work is approximately were OpenSSI faded = away; bringing OpenSSI up to current Linux kernel standards would be a = large task. o OpenSSI was based on RHEL 4.x and Debian xxx, updates to sys config = as OpenSSI is somewhat disruptive w.r.t. use of context specific = symlinks which make the booting of a Linux distro work=E2=80=A6 = =E2=80=98/etc =3D=3D /node-num/etc=E2=80=99 thus creating a global /etc = which namei resolves to a accessing node specific path. =C2=B7 Node discovery is based on Ethernet broadcast capabilities = which not all high performance fabric supported in the node boot phase. = For the InfiniBand interconnect implementation I had to use Ethernet as = the boot/node discovery fabric. OpenSSI in various flavors was developed = and functional in the era before high performance interconnects were = widely available outside of commercial system builder designs. =20 Once the industry gets to an on-chip reliable network/interconnect = (reliable as processor registers) OpenSSI concepts could become a viable = option. The prevailing industry trend is to cluster #=E2=80=99s of independent = nodes/systems (private root FS) which all mount a DFS (Lustre or = GFS=E2=80=A6.) to form a cluster; isolating the reliability concerns to = a specific node and the cluster wide application. =20 Stan. =20 From: Luke Hindman [mailto:[email protected]]=20 Sent: Thursday, August 04, 2016 4:46 PM To: [email protected] Subject: [SSI-devel] OpenSSI Project Status =20 Hi, =20 A friend and I were just talking about OpenSSI and we're thinking it = might be interesting to explore this technology again. I am curious if = any of the OpenSSI developers are still around and subscribed to this = list. It would be interesting to know why this project died and whether = they feel this approach to clustering might still be useful in 2016. =20 I would appreciate any thoughts or comments. =20 Thanks, =20 --Luke ------=_NextPart_000_011C_01D1F09D.3FD39E60 Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: quoted-printable <html xmlns:v=3D"urn:schemas-microsoft-com:vml" = xmlns:o=3D"urn:schemas-microsoft-com:office:office" = xmlns:w=3D"urn:schemas-microsoft-com:office:word" = xmlns:m=3D"http://schemas.microsoft.com/office/2004/12/omml" = xmlns=3D"http://www.w3.org/TR/REC-html40"><head><meta = http-equiv=3DContent-Type content=3D"text/html; charset=3Dutf-8"><meta = name=3DGenerator content=3D"Microsoft Word 14 (filtered = medium)"><style><!-- /* Font Definitions */ @font-face {font-family:Wingdings; panose-1:5 0 0 0 0 0 0 0 0 0;} @font-face {font-family:Wingdings; panose-1:5 0 0 0 0 0 0 0 0 0;} @font-face {font-family:Calibri; panose-1:2 15 5 2 2 2 4 3 2 4;} @font-face {font-family:Tahoma; panose-1:2 11 6 4 3 5 4 4 2 4;} @font-face {font-family:"Comic Sans MS"; panose-1:3 15 7 2 3 3 2 2 2 4;} /* Style Definitions */ p.MsoNormal, li.MsoNormal, div.MsoNormal {margin:0in; margin-bottom:.0001pt; font-size:12.0pt; font-family:"Times New Roman","serif";} a:link, span.MsoHyperlink {mso-style-priority:99; color:#0563C1; text-decoration:underline;} a:visited, span.MsoHyperlinkFollowed {mso-style-priority:99; color:#954F72; text-decoration:underline;} p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph {mso-style-priority:34; margin-top:0in; margin-right:0in; margin-bottom:0in; margin-left:.5in; margin-bottom:.0001pt; font-size:12.0pt; font-family:"Times New Roman","serif";} span.EmailStyle17 {mso-style-type:personal-reply; font-family:"Calibri","sans-serif"; color:#44546A;} .MsoChpDefault {mso-style-type:export-only; font-family:"Calibri","sans-serif";} @page WordSection1 {size:8.5in 11.0in; margin:1.0in 1.0in 1.0in 1.0in;} div.WordSection1 {page:WordSection1;} /* List Definitions */ @list l0 {mso-list-id:99686509; mso-list-type:hybrid; mso-list-template-ids:-1418692474 67698689 67698691 67698693 67698689 = 67698691 67698693 67698689 67698691 67698693;} @list l0:level1 {mso-level-number-format:bullet; mso-level-text:=EF=82=B7; mso-level-tab-stop:none; mso-level-number-position:left; text-indent:-.25in; font-family:Symbol;} @list l0:level2 {mso-level-number-format:bullet; mso-level-text:o; mso-level-tab-stop:none; mso-level-number-position:left; text-indent:-.25in; font-family:"Courier New";} @list l0:level3 {mso-level-number-format:bullet; mso-level-text:=EF=82=A7; mso-level-tab-stop:none; mso-level-number-position:left; text-indent:-.25in; font-family:Wingdings;} @list l0:level4 {mso-level-number-format:bullet; mso-level-text:=EF=82=B7; mso-level-tab-stop:none; mso-level-number-position:left; text-indent:-.25in; font-family:Symbol;} @list l0:level5 {mso-level-number-format:bullet; mso-level-text:o; mso-level-tab-stop:none; mso-level-number-position:left; text-indent:-.25in; font-family:"Courier New";} @list l0:level6 {mso-level-number-format:bullet; mso-level-text:=EF=82=A7; mso-level-tab-stop:none; mso-level-number-position:left; text-indent:-.25in; font-family:Wingdings;} @list l0:level7 {mso-level-number-format:bullet; mso-level-text:=EF=82=B7; mso-level-tab-stop:none; mso-level-number-position:left; text-indent:-.25in; font-family:Symbol;} @list l0:level8 {mso-level-number-format:bullet; mso-level-text:o; mso-level-tab-stop:none; mso-level-number-position:left; text-indent:-.25in; font-family:"Courier New";} @list l0:level9 {mso-level-number-format:bullet; mso-level-text:=EF=82=A7; mso-level-tab-stop:none; mso-level-number-position:left; text-indent:-.25in; font-family:Wingdings;} ol {margin-bottom:0in;} ul {margin-bottom:0in;} --></style><!--[if gte mso 9]><xml> <o:shapedefaults v:ext=3D"edit" spidmax=3D"1026" /> </xml><![endif]--><!--[if gte mso 9]><xml> <o:shapelayout v:ext=3D"edit"> <o:idmap v:ext=3D"edit" data=3D"1" /> </o:shapelayout></xml><![endif]--></head><body lang=3DEN-US = link=3D"#0563C1" vlink=3D"#954F72"><div class=3DWordSection1><p = class=3DMsoNormal><span = style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif";color:#44546= A'>OpenSSI is a very interesting technology for its time: Locus = Computing, Intel, Transmeta and HP labs all gave the SSI = (Single-System-Image) code base a chance.<o:p></o:p></span></p><p = class=3DMsoNormal><span = style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif";color:#44546= A'><o:p> </o:p></span></p><p class=3DMsoNormal><span = style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif";color:#44546= A'>The challenges which held back OpenSSI acceptance = are:<o:p></o:p></span></p><p class=3DMsoListParagraph = style=3D'text-indent:-.25in;mso-list:l0 level1 lfo1'><![if = !supportLists]><span = style=3D'font-size:11.0pt;font-family:Symbol;color:#44546A'><span = style=3D'mso-list:Ignore'>=C2=B7<span style=3D'font:7.0pt "Times New = Roman"'> = </span></span></span><![endif]><span = style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif";color:#44546= A'>Fear of SSI model when OpenSSI RPCs have no =E2=80=98real=E2=80=99 = failure/recovery mechanism; if interconnect fails RPC hangs; that said = the predecessor code of OpenSSI powered Intel Paragon supercomputers @ = 2000+ nodes, in addition to the TFLOPS machine service/IO nodes, large = IBM and Transmeta systems. =C2=A0OpenSSI can shine when your = interconnect is reliable.<br><br><o:p></o:p></span></p><p = class=3DMsoListParagraph style=3D'text-indent:-.25in;mso-list:l0 level1 = lfo1'><![if !supportLists]><span = style=3D'font-size:11.0pt;font-family:Symbol;color:#44546A'><span = style=3D'mso-list:Ignore'>=C2=B7<span style=3D'font:7.0pt "Times New = Roman"'> = </span></span></span><![endif]><span = style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif";color:#44546= A'>Issues upstreaming Linux kernel patches:<o:p></o:p></span></p><p = class=3DMsoListParagraph = style=3D'margin-left:1.0in;text-indent:-.25in;mso-list:l0 level2 = lfo1'><![if !supportLists]><span = style=3D'font-size:11.0pt;font-family:"Courier New";color:#44546A'><span = style=3D'mso-list:Ignore'>o<span style=3D'font:7.0pt "Times New = Roman"'> </span></span></span><![endif]><span = style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif";color:#44546= A'>CFS filesystem was disliked by the Linux community; context specific = symlinks were disliked by Linus; not clear on other CFS aspects which = were disliked.<o:p></o:p></span></p><p class=3DMsoListParagraph = style=3D'margin-left:1.0in;text-indent:-.25in;mso-list:l0 level2 = lfo1'><![if !supportLists]><span = style=3D'font-size:11.0pt;font-family:"Courier New";color:#44546A'><span = style=3D'mso-list:Ignore'>o<span style=3D'font:7.0pt "Times New = Roman"'> </span></span></span><![endif]><span = style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif";color:#44546= A'>rfork(), rexec(), rvfork() were abused by Linux community as the = implementation was built on RPCs which had to no clean failure = recovery.<o:p></o:p></span></p><p class=3DMsoListParagraph = style=3D'margin-left:1.0in;text-indent:-.25in;mso-list:l0 level2 = lfo1'><![if !supportLists]><span = style=3D'font-size:11.0pt;font-family:"Courier New";color:#44546A'><span = style=3D'mso-list:Ignore'>o<span style=3D'font:7.0pt "Times New = Roman"'> </span></span></span><![endif]><span = style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif";color:#44546= A'>Linux-2.6.18 + some x64 work is approximately were OpenSSI faded = away; bringing OpenSSI up to current Linux kernel standards would be a = large task.<o:p></o:p></span></p><p class=3DMsoListParagraph = style=3D'margin-left:1.0in;text-indent:-.25in;mso-list:l0 level2 = lfo1'><![if !supportLists]><span = style=3D'font-size:11.0pt;font-family:"Courier New";color:#44546A'><span = style=3D'mso-list:Ignore'>o<span style=3D'font:7.0pt "Times New = Roman"'> </span></span></span><![endif]><span = style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif";color:#44546= A'>OpenSSI was based on RHEL 4.x and Debian xxx, updates to sys config = as OpenSSI is somewhat disruptive w.r.t. use of context specific = symlinks which make the booting of a Linux distro work=E2=80=A6 = =E2=80=98/etc =3D=3D /node-num/etc=E2=80=99 thus creating a global /etc = which namei resolves to a accessing node specific = path.<br><br><o:p></o:p></span></p><p class=3DMsoListParagraph = style=3D'text-indent:-.25in;mso-list:l0 level1 lfo1'><![if = !supportLists]><span = style=3D'font-size:11.0pt;font-family:Symbol;color:#44546A'><span = style=3D'mso-list:Ignore'>=C2=B7<span style=3D'font:7.0pt "Times New = Roman"'> = </span></span></span><![endif]><span = style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif";color:#44546= A'>Node discovery is based on Ethernet broadcast capabilities which not = all high performance fabric supported in the node boot phase. For the = InfiniBand interconnect implementation I had to use Ethernet as the = boot/node discovery fabric. OpenSSI in various flavors was developed and = functional in the era before high performance interconnects were widely = available outside of commercial system builder = designs.<o:p></o:p></span></p><p class=3DMsoNormal><span = style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif";color:#44546= A'><o:p> </o:p></span></p><p class=3DMsoNormal><span = style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif";color:#44546= A'>Once the industry gets to an on-chip reliable network/interconnect = (reliable as processor registers) OpenSSI concepts could become a viable = option.<o:p></o:p></span></p><p class=3DMsoNormal><span = style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif";color:#44546= A'>The prevailing industry trend is to cluster #=E2=80=99s of = independent nodes/systems (private root FS) which all mount a DFS = (Lustre or GFS=E2=80=A6.) to form a cluster; isolating the reliability = concerns to a specific node and the cluster wide = application.<o:p></o:p></span></p><p class=3DMsoNormal><span = style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif";color:#44546= A'><o:p> </o:p></span></p><p class=3DMsoNormal><span = style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif";color:#44546= A'>Stan.<o:p></o:p></span></p><p class=3DMsoNormal><span = style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif";color:#44546= A'><o:p> </o:p></span></p><p class=3DMsoNormal><b><span = style=3D'font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span>= </b><span style=3D'font-size:10.0pt;font-family:"Tahoma","sans-serif"'> = Luke Hindman [mailto:[email protected]] <br><b>Sent:</b> = Thursday, August 04, 2016 4:46 PM<br><b>To:</b> = [email protected]<br><b>Subject:</b> [SSI-devel] = OpenSSI Project Status<o:p></o:p></span></p><p = class=3DMsoNormal><o:p> </o:p></p><div><div><p = class=3DMsoNormal><span style=3D'font-family:"Comic Sans = MS"'>Hi,<o:p></o:p></span></p></div><div><p class=3DMsoNormal><span = style=3D'font-family:"Comic Sans = MS"'><o:p> </o:p></span></p></div><div><p class=3DMsoNormal><span = style=3D'font-family:"Comic Sans MS"'>A friend and I were just talking = about OpenSSI and we're thinking it might be interesting to explore this = technology again. I am curious if any of the OpenSSI developers = are still around and subscribed to this list. It would be = interesting to know why this project died and whether they feel this = approach to clustering might still be useful in = 2016.<o:p></o:p></span></p></div><div><p class=3DMsoNormal><span = style=3D'font-family:"Comic Sans = MS"'><o:p> </o:p></span></p></div><div><p class=3DMsoNormal><span = style=3D'font-family:"Comic Sans MS"'>I would appreciate any thoughts or = comments.<o:p></o:p></span></p></div><div><p class=3DMsoNormal><span = style=3D'font-family:"Comic Sans = MS"'><o:p> </o:p></span></p></div><div><p class=3DMsoNormal><span = style=3D'font-family:"Comic Sans = MS"'>Thanks,<o:p></o:p></span></p></div><div><p class=3DMsoNormal><span = style=3D'font-family:"Comic Sans = MS"'><o:p> </o:p></span></p></div><div><p class=3DMsoNormal><span = style=3D'font-family:"Comic Sans = MS"'>--Luke<o:p></o:p></span></p></div></div></div></body></html> ------=_NextPart_000_011C_01D1F09D.3FD39E60-- --===============6625671879866027496== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline ------------------------------------------------------------------------------ --===============6625671879866027496== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ ssic-linux-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/ssic-linux-devel --===============6625671879866027496==--