Re: Opinions wanted: Is smartfrog the right framework for my osgi-based system?
"Stanley C. Burnitt" <[email protected]> Tue, 13 Jul 2010 15:53:34 -0300
| Newsgroups | gmane.comp.java.smartfrog.user |
|---|---|
| Message-ID | <[email protected]> |
--===============3993772954258725528== Content-Type: multipart/alternative; boundary=001636832f32e09c1f048b4962d0 --001636832f32e09c1f048b4962d0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Thank you for your time, Julio. I should have studied and run more of the examples before sending that msg to the users' mailing list. The SF Dynamic Webserver example looks very useful. If it can deploy, configure, run, and undeploy apache servers, I believe it can be customized and extended to meet most of my requirements. Regards, Stan 2010/7/13 Guijarro, Julio <[email protected]> > Hi Stanley, > > > > See my answers below > > > > *From:* Stanley C. Burnitt [mailto:[email protected]] > *Sent:* 13 July 2010 05:34 > *To:* smartfrog-users-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org > *Subject:* [Smartfrog-users] Opinions wanted: Is smartfrog the right > framework for my osgi-based system? > > > > I am one day into learning how to use smartfrog on my home network > (fedora12 + mac osx platforms). So far so good. > > But I want to know how --* or if *-- smartfrog should be used to install > and configure non-jar artifacts such as db server distributions, hadoop > distributions, solr distributions, and standalone eclipse-osgi products, = on > remote linux servers running smartfrog daemons. > > If I am not misinterpreting the docs, I *could*: > > 1. Use "cluster ssh" to install a jdk or jvm, smartfrog, and start the > sf-daemon on each target linux host. > > That is one option, especially if all the machines need to be configured = at > the same time. You could also use SF to install the JDK and SF in other > machines or to have a standard boot image that you use in every machine w= ith > SF configured to boot automatically. > > > 2. Use smartfrog to deploy and run bash scripts which download and insta= ll > db distributions (via yum), edit db configuration files, ceate db users= / > schemas / tables, and issue startup & shutdown commands. > > That is one option, the other is to write special component wrappers to > manage the lifecycle of your individual app. Also, there is a SF componen= t > that allows you to use any ANT task directly to manipulate files, > directories, apps=E2=80=A6 > > > 3. Use smartfrog to deploy and run bash scripts which install > cloudera.repo files, then install cloudera's hadoop / hbase / zookeeper > distributions (via yum), edit configuration files, and issue startup & > shutdown commands. > > Steve has been working on some especial components to deploy and manage > Hadoop but he is using Apache=E2=80=99s distro. He is on holiday for two = weeks and > could give you an update on it once he is back. > > > 4. Use smartfrog to deploy and configure a solr cluster. (There is no y= um > pkg for solr, to my knowledge. Can I use smartfrog to deploy & run a bas= h > script which scp's, extracts, and configures gzipped solr distributions o= n a > cluster of solr servers?) > > There are components for SSH and SCP, you can also write components that > get the file from a webserver for example and perform the operation local= ly. > For example, you deploy a component in a machine that then downloads a > package locally, unzips it and run the app. I would advise you to try the > Dynamic Webserver example included with SF. It does exactly what you > describe here and you could reuse most of it. > > Check: > > > http://smartfrog.svn.sourceforge.net/viewvc/smartfrog/trunk/core/smartfro= g/docs/sfDynamicWebServerExample.pdf?revision=3D8357 > > And: > > http://wiki.smartfrog.org/wiki/display/sf/Dynamic+Web+Server > > > > > 5. Deploy and manage the dependencies and lifecycles of all of my system= 's > OSGi components -- standalone product launchers on top of dozens of plugi= ns. > > Does this seem reasonable? My entire system is implemented in java, and > for this reason I chose to look at smartfrog first, before bcfg2 or puppe= t. > But this is an OSGi based system, not J2EE, and I will be deploying (then > extracting) gzipped eclipse products, not jars. > Also, the OSGi bundles are started from native product launchers -- again= , > not java jars. > > With the stable release of SF you have to manage your OSGi components usi= ng > either Java and Scripts but there is a version of SF that has full > integration with OSGi (SF uses it but it also allows you to describe an > manage the entire environment but is not stable and has not yet been used= in > production) The intention is to merge the branch into the trunk soon but = we > do not know how long it is going to take to merge the code and test it > properly. > > > And I intend to run system / scalability tests on EC2. > > I see smartfrog's advantages when all the java components are up and > running, but is smartfrog the right tool for managing the remote > installation and configuration of all these subsystems? I don't know... = I > would really appreciate some feedback. > > SF is about deployment, configuration and managing but it is a framework > and needs the right components to show its potential. We use SF for all t= he > things that you have described but to really take advantage of SF sometim= es > you need to add your own components to manage the lifecycle of the app th= at > you want to deploy, configure and manage (this can be done by calling nat= ive > apps also, just see the webserver example). Also, you need to think what = is > the right architectural patterns for you app and model them into descript= ion > templates that you can then re-use for multiple deployments or to > adapt/adjust your deployment. > > I would recommend you to first define a simple PoC to prove that the key > things that you need are available to you, to test key risk issues and to > learn how the system can help you. Don=E2=80=99t hesitate to ask for help= if you > encounter any problems. > > > Thanks, > Stan > > Best regards, > > > > Julio G > --001636832f32e09c1f048b4962d0 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Thank you for your time, Julio.=C2=A0 I should have studied and run more of= the examples before sending that msg to the users' mailing list. <br>T= he SF Dynamic Webserver example looks very useful.=C2=A0 If it can deploy, = configure, run, and undeploy apache servers, I believe it can be customized= and extended to meet most of my requirements.=C2=A0=C2=A0 <br> <br>Regards,<br>Stan<br><br><br><div class=3D"gmail_quote">2010/7/13 Guijar= ro, Julio <span dir=3D"ltr"><<a href=3D"mailto:[email protected]">ju= [email protected]</a>></span><br><blockquote class=3D"gmail_quote" sty= le=3D"margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204);= padding-left: 1ex;"> <div link=3D"blue" vlink=3D"purple" lang=3D"EN-GB"><div><p class=3D"MsoNorm= al"><span style=3D"font-size: 11pt; color: rgb(31, 73, 125);">Hi Stanley,</= span></p><p class=3D"MsoNormal"><span style=3D"font-size: 11pt; color: rgb(= 31, 73, 125);">=C2=A0</span></p> <p class=3D"MsoNormal"><span style=3D"font-size: 11pt; color: rgb(31, 73, 1= 25);">See my answers below</span></p><p class=3D"MsoNormal"><span style=3D"= font-size: 11pt; color: rgb(31, 73, 125);">=C2=A0</span></p><p class=3D"Mso= Normal"><b><span style=3D"font-size: 10pt;" lang=3D"EN-US">From:</span></b>= <span style=3D"font-size: 10pt;" lang=3D"EN-US"> Stanley C. Burnitt [mailto= :<a href=3D"mailto:[email protected]" target=3D"_blank">[email protected]</a>] <b= r> <b>Sent:</b> 13 July 2010 05:34<br><b>To:</b> <a href=3D"mailto:smartfrog-u= [email protected]" target=3D"_blank">[email protected]= orge.net</a><br><b>Subject:</b> [Smartfrog-users] Opinions wanted: Is smart= frog the right framework for my osgi-based system?</span></p> <div class=3D"im"><p class=3D"MsoNormal">=C2=A0</p><p class=3D"MsoNormal" s= tyle=3D"margin-bottom: 12pt;"><span style=3D"font-size: 13.5pt;">I am one d= ay into learning how to use smartfrog on my home network (fedora12 + mac os= x platforms).=C2=A0=C2=A0 So far so good.<br> <br>But I want to know how --<i> or if </i>-- smartfrog should be used to i= nstall and configure non-jar artifacts such as db server distributions, had= oop distributions, solr distributions, and standalone eclipse-osgi products= ,=C2=A0 on remote linux servers running smartfrog daemons.<br> <br>If I am not misinterpreting the docs, I <i>could</i>:<br><br>1.=C2=A0 U= se "cluster ssh" to install a jdk or jvm, smartfrog, and start th= e sf-daemon on each target linux host.<span style=3D"color: rgb(31, 73, 125= );"></span></span></p> </div><p class=3D"MsoNormal" style=3D"margin-bottom: 12pt;"><span style=3D"= font-size: 11pt; color: rgb(31, 73, 125);">That is one option, especially i= f all the machines need to be configured at the same time. You could also u= se SF to install the JDK and SF in other machines or to have a standard boo= t image that you use in every machine with SF configured to boot automatica= lly.</span></p> <div class=3D"im"><p class=3D"MsoNormal" style=3D"margin-bottom: 12pt;"><sp= an style=3D"font-size: 13.5pt;"><br>2.=C2=A0 Use smartfrog to deploy and ru= n bash scripts which download and install db distributions (via yum),=C2=A0= edit db configuration files,=C2=A0 ceate db users / schemas / tables, and = issue startup & shutdown commands.<span style=3D"color: rgb(31, 73, 125= );"></span></span></p> </div><p class=3D"MsoNormal" style=3D"margin-bottom: 12pt;"><span style=3D"= font-size: 11pt; color: rgb(31, 73, 125);">That is one option, the other is= to write special component wrappers to manage the lifecycle of your indivi= dual app. Also, there is a SF component that allows you to use any ANT task= directly to manipulate files, directories, apps=E2=80=A6</span></p> <div class=3D"im"><span style=3D"font-size: 13.5pt;"><br><br>3.=C2=A0 Use s= martfrog to deploy and run bash scripts which install cloudera.repo files, = then install cloudera's hadoop / hbase / zookeeper distributions (via y= um),=C2=A0 edit configuration files, and issue startup & shutdown comma= nds.<span style=3D"color: rgb(31, 73, 125);"></span></span></div> <p class=3D"MsoNormal" style=3D"margin-bottom: 12pt;"><span style=3D"font-s= ize: 11pt; color: rgb(31, 73, 125);">Steve has been working on some especia= l components to deploy and manage Hadoop but he is using Apache=E2=80=99s d= istro. He is on holiday for two weeks and could give you an update on it on= ce he is back.</span></p> <div class=3D"im"><br><br><span style=3D"font-size: 13.5pt;">4.=C2=A0 Use s= martfrog to deploy and configure a solr cluster.=C2=A0 (There is no yum pkg= for solr, to my knowledge.=C2=A0 Can I use smartfrog to deploy & run a= bash script which scp's, extracts, and configures gzipped solr distrib= utions on a cluster of solr servers?)<br> <br><span style=3D"color: rgb(31, 73, 125);"></span></span></div><p class= =3D"MsoNormal" style=3D"margin-bottom: 12pt;"><span style=3D"font-size: 11p= t; color: rgb(31, 73, 125);">There are components for SSH and SCP, you can = also write components that get the file from a webserver for example and pe= rform the operation locally. For example, you deploy a component in a machi= ne that then downloads a package locally, unzips it and run the app. I woul= d advise you to try the Dynamic Webserver example included with SF. It does= exactly what you describe here and you could reuse most of it.</span></p> <p class=3D"MsoNormal" style=3D"margin-bottom: 12pt;"><span style=3D"font-s= ize: 11pt; color: rgb(31, 73, 125);">Check: </span></p><p class=3D"MsoNorma= l" style=3D"margin-bottom: 12pt;"><span style=3D"font-size: 11pt; color: rg= b(31, 73, 125);"><a href=3D"http://smartfrog.svn.sourceforge.net/viewvc/sma= rtfrog/trunk/core/smartfrog/docs/sfDynamicWebServerExample.pdf?revision=3D8= 357" target=3D"_blank">http://smartfrog.svn.sourceforge.net/viewvc/smartfro= g/trunk/core/smartfrog/docs/sfDynamicWebServerExample.pdf?revision=3D8357</= a></span></p> <p class=3D"MsoNormal" style=3D"margin-bottom: 12pt;"><span style=3D"font-s= ize: 11pt; color: rgb(31, 73, 125);">And: </span></p><p class=3D"MsoNormal"= style=3D"margin-bottom: 12pt;"><span style=3D"font-size: 11pt; color: rgb(= 31, 73, 125);"><a href=3D"http://wiki.smartfrog.org/wiki/display/sf/Dynamic= +Web+Server" target=3D"_blank">http://wiki.smartfrog.org/wiki/display/sf/Dy= namic+Web+Server</a> </span></p> <div class=3D"im"><p class=3D"MsoNormal" style=3D"margin-bottom: 12pt;"><sp= an style=3D"font-size: 11pt; color: rgb(31, 73, 125);">=C2=A0</span></p><p = class=3D"MsoNormal" style=3D"margin-bottom: 12pt;"><span style=3D"font-size= : 13.5pt;"><br> 5.=C2=A0 Deploy and manage the dependencies and lifecycles of all of my sys= tem's OSGi components -- standalone product launchers on top of dozens = of plugins.<br><br>Does this seem reasonable?=C2=A0 My entire system is imp= lemented in java, and for this reason I chose to look at smartfrog first, b= efore bcfg2 or puppet.=C2=A0 But this is an OSGi based system, not J2EE, an= d I will be deploying (then extracting) gzipped eclipse products, not jars.= <br> Also, the OSGi bundles are started from native product launchers -- again, = not java jars.<span style=3D"color: rgb(31, 73, 125);"></span></span></p></= div><p class=3D"MsoNormal" style=3D"margin-bottom: 12pt;"><span style=3D"fo= nt-size: 11pt; color: rgb(31, 73, 125);">With the stable release of SF you = have to manage your OSGi components using either Java and Scripts but there= is a version of SF that has full integration with OSGi (SF uses it but it = also allows you to describe an manage the entire environment but is not sta= ble and has not yet been used in production) The intention is to merge the = branch into the trunk soon but we do not know how long it is going to take = to merge the code and test it properly.</span></p> <div class=3D"im"><span style=3D"font-size: 13.5pt;"><br><br>And I intend t= o run system / scalability tests on EC2.<br><br>I see smartfrog's advan= tages when all the java components are up and running, but is smartfrog the= right tool for managing the remote installation and configuration of all t= hese subsystems?=C2=A0 I don't know... I would really appreciate some f= eedback.<br> <br><span style=3D"color: rgb(31, 73, 125);"></span></span></div><p class= =3D"MsoNormal" style=3D"margin-bottom: 12pt;"><span style=3D"font-size: 11p= t; color: rgb(31, 73, 125);">SF is about deployment, configuration and mana= ging but it is a framework and needs the right components to show its poten= tial. We use SF for all the things that you have described but to really ta= ke advantage of SF sometimes you need to add your own components to manage = the lifecycle of the app that you want to deploy, configure and manage (thi= s can be done by calling native apps also, just see the webserver example).= Also, you need to think what is the right architectural patterns for you a= pp and model them into description templates that you can then re-use for m= ultiple deployments or to adapt/adjust your deployment.</span></p> <p class=3D"MsoNormal" style=3D"margin-bottom: 12pt;"><span style=3D"font-s= ize: 11pt; color: rgb(31, 73, 125);">I would recommend you to first define = a simple PoC to prove that the key things that you need are available to yo= u, to test key risk issues and to learn how the system can help you. Don=E2= =80=99t hesitate to ask for help if you encounter any problems. </span></p> <p class=3D"MsoNormal" style=3D"margin-bottom: 12pt;"><span style=3D"font-s= ize: 13.5pt;"><br>Thanks,</span> <span style=3D"font-size: 13.5pt;"><br>Sta= n<span style=3D"color: rgb(31, 73, 125);"></span></span></p><p class=3D"Mso= Normal" style=3D"margin-bottom: 12pt;"> <span style=3D"font-size: 11pt; color: rgb(31, 73, 125);">Best regards,</sp= an></p><p class=3D"MsoNormal" style=3D"margin-bottom: 12pt;"><span style=3D= "font-size: 11pt; color: rgb(31, 73, 125);">=C2=A0</span></p><p class=3D"Ms= oNormal" style=3D"margin-bottom: 12pt;"> <span style=3D"font-size: 11pt; color: rgb(31, 73, 125);">Julio G</span></p= ></div></div></blockquote></div><br> --001636832f32e09c1f048b4962d0-- --===============3993772954258725528== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline ------------------------------------------------------------------------------ This SF.net email is sponsored by Sprint What will you do first with EVO, the first 4G phone? Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first --===============3993772954258725528== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Smartfrog-users mailing list Smartfrog-users-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org https://lists.sourceforge.net/lists/listinfo/smartfrog-users --===============3993772954258725528==--