Re: Reggie
Per Edlund <[email protected]> Tue, 30 Jun 2009 13:34:23 +0200
| Newsgroups | gmane.comp.java.sun.jini |
|---|---|
| Message-ID | <[email protected]> |
--0016364592e83c542a046d8f30d2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Could be, don't really know. I don't have the abillity to test this out, bu= t we now have more surveillance of the jvm status of the webapps and making manual restarts when memory becomes too close to max. After this we haven't had any issues at all. It seamed like reggie ran out of thread's trying to notify (don't know how many it used), and this caused a big queue of jobs and in that q there wher= e other jobs that would timeout as well witch led to new services wouldn't ge= t the serviceId and stuff like that. /Per On Fri, Jun 26, 2009 at 2:39 PM, Gregg Wonderly <[email protected]> wrote: > > On Jun 26, 2009, at 3:29 AM, Per Edlund wrote: > > As it seams, this issue is related to client machines. If a client machin= e > get's memoty issues, and reggie is trying to notify the client (which use= s a > servicechache listening on all service events), reggie chokes when the > client can't reply to reggie. > > > Could this also be an issue for clients with firewalls up which can't > accept connections for the notifications, and the notification connection= s > timeout? I've seen some strange things from time to time on more restric= ted > networks and not really been able to pinpoint the issues. Many times, it > seems like discovery degenerates into a really slow and painful process. = An > outbound call, initiating a TCP connection which must timeout before > anything can be done to eliminate the call context, can be quite long liv= ed, > on the order of minutes. I wonder if this would cause other > discovery/notification to be delayed and thus build up more held context = in > the reggie JVM and elsewhere. > > Gregg Wonderly > > Fixiing the memory issues on the client get's rid of the behaviour on > reggie > > Just thouth I should tell > > Cheers > /Per > > On Wed, May 27, 2009 at 3:47 PM, Juan Ramirez <[email protected]>wrote= : > >> Hi, Peter. Reggie does queue up discovery requests and event >> notifications and services these with threads from the same pool. It is >> possible that there is a large number of event notifications to be deliv= ered >> to the LookupCache instances thus delaying the processing of discovery >> requests. You could set Reggie's logging level to FINEST to see what Re= ggie >> is doing at the time it appears to "choke". There is additional info on >> this scenario in the following CRs. >> >> https://issues.apache.org/jira/browse/RIVER-52 >> https://issues.apache.org/jira/browse/RIVER-197 >> >> Hope this helps, >> >> Juan >> >> >> On 05/27/09 07:17, Per Edlund wrote: >> >>> Hi! >>> >>> We'ev been using Jini for quite som time now but havn't upgraded after >>> the 2.1 release. We're currently experiencing some issues with reggie >>> choking the machine and we can't really tell why. The number of jiniser= vices >>> we have is around 60 (these are client's as well, and on top of this we= have >>> webservers which are clients (around 100-200)) , we use the LookupCache= etc, >>> but efery now and then we have to restart the VM running reggie. Doing = a >>> discovery during this time get's no lookup, and services can't be found= . >>> >>> If we look at how services register/unregister, that seam to take an >>> aweful long time when reggie is starting to choke, and out theory is th= at >>> reggie uses some kind of queue which grows more than it can be processe= d. >>> >>> Does any one have an idea on how to dig into this? >>> >>> Env: Linux Debian, Sun Java 1.5&1.6 >>> >>> Kind regards >>> /Per >>> -----------------------------------------------------------------------= --- >>> Getting Started: http://www.jini.org/wiki/Category:Getting_StartedCommu= nity Web Site: >>> http://jini.org jini-users Archive: >>> http://archives.java.sun.com/archives/jini-users.html Unsubscribing: >>> email "signoff JINI-USERS" to [email protected] >>> >> >> > > > -- > ******************************************** > Per Edlund > www.bozoka.com > [email protected] > +46(0)8 545 068 05 > +46(0)703 71 70 02 > ******************************************** > -------------------------------------------------------------------------= - > Getting Started: http://www.jini.org/wiki/Category:Getting_StartedCommuni= ty Web Site: > http://jini.org jini-users Archive: > http://archives.java.sun.com/archives/jini-users.html Unsubscribing: emai= l > "signoff JINI-USERS" to [email protected] > > > --=20 ******************************************** Per Edlund Bozoka.com, [email protected] Web: www.bozoka.com Office: +46(0)8 545 068 05 Mobile:+46(0)703 71 70 02 Address: Tegn=E9rgatan 28, 2tr 113 59 Stockholm Sweden ******************************************** -------------------------------------------------------------------------- Getting Started: http://www.jini.org/wiki/Category:Getting_Started Community Web Site: http://jini.org jini-users Archive: http://archives.java.sun.com/archives/jini-users.html Unsubscribing: email "signoff JINI-USERS" to [email protected] --0016364592e83c542a046d8f30d2 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Could be, don't really know. I don't have the abillity to test this= out, but we now have more surveillance of the jvm status of the webapps an= d making manual restarts when memory becomes too close to max. After this w= e haven't had any issues at all.<br> <br>It seamed like reggie ran out of thread's trying to notify (don'= ;t know how many it used), and this caused a big queue of jobs and in that = q there where other jobs that would timeout as well witch led to new servic= es wouldn't get the serviceId and stuff like that.<br> <br>/Per<br><br><div class=3D"gmail_quote">On Fri, Jun 26, 2009 at 2:39 PM,= Gregg Wonderly <span dir=3D"ltr"><<a href=3D"mailto:[email protected]"= >[email protected]</a>></span> wrote:<br><blockquote class=3D"gmail_quo= te" style=3D"border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt= 0.8ex; padding-left: 1ex;"> <div style=3D""><br><div><div class=3D"im"><div>On Jun 26, 2009, at 3:29 AM= , Per Edlund wrote:</div><blockquote type=3D"cite">As it seams, this issue = is related to client machines. If a client machine get's memoty issues,= and reggie is trying to notify the client (which uses a servicechache list= ening on all service events), reggie chokes when the client can't reply= to reggie.<br> </blockquote><div><br></div></div><div>Could this also be an issue for clie= nts with firewalls up which can't accept connections for the notificati= ons, and the notification connections timeout? =A0I've seen some strang= e things from time to time on more restricted networks and not really been = able to pinpoint the issues. =A0Many times, it seems like discovery degener= ates into a really slow and painful process. =A0An outbound call, initiatin= g a TCP connection which must timeout before anything can be done to elimin= ate the call context, can be quite long lived, on the order of minutes. =A0= I wonder if this would cause other discovery/notification to be delayed and= thus build up more held context in the reggie JVM and elsewhere.</div> <div><br></div><font color=3D"#888888"><div>Gregg Wonderly</div></font><div= ><div></div><div class=3D"h5"><br><blockquote type=3D"cite">Fixiing the mem= ory issues on the client get's rid of the behaviour on reggie<br><br>Ju= st thouth I should tell<br> <br>Cheers<br>/Per<br><br><div class=3D"gmail_quote">On Wed, May 27, 2009 a= t 3:47 PM, Juan Ramirez <span dir=3D"ltr"><<a href=3D"mailto:Juan.Ramire= [email protected]" target=3D"_blank">[email protected]</a>></span> wrote:<br>= <blockquote class=3D"gmail_quote" style=3D"border-left: 1px solid rgb(204,= 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"> Hi, Peter. =A0Reggie does queue up discovery requests and event notificatio= ns and services these with threads from the same pool. =A0It is possible th= at there is a large number of event notifications to be delivered to the Lo= okupCache instances thus delaying the processing of discovery requests. =A0= You could set Reggie's logging level to FINEST to see what Reggie is do= ing at the time it appears to "choke". =A0There is additional inf= o on this scenario in the following CRs.<br> <br> <a href=3D"https://issues.apache.org/jira/browse/RIVER-52" target=3D"= _blank">https://issues.apache.org/jira/browse/RIVER-52</a><br> <a href=3D"h= ttps://issues.apache.org/jira/browse/RIVER-197" target=3D"_blank">https://i= ssues.apache.org/jira/browse/RIVER-197</a><br> <br> Hope this helps,<br> <br> Juan<div><div></div><div><br> <br> On 05/27= /09 07:17, Per Edlund wrote:<br> </div></div><blockquote class=3D"gmail_quo= te" style=3D"border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt= 0.8ex; padding-left: 1ex;"> <div><div></div><div> Hi!<br> <br> We'ev been using Jini for quite som = time now but havn't upgraded after the 2.1 release. We're currently= experiencing some issues with reggie choking the machine and we can't = really tell why. The number of jiniservices we have is around 60 (these are= client's as well, and on top of this we have webservers which are clie= nts (around 100-200)) , we use the LookupCache etc, but efery now and then = we have to restart the VM running reggie. Doing a discovery during this tim= e get's no lookup, and services can't be found.<br> <br> If we look at how services register/unregister, that seam to take an = aweful long time when reggie is starting to choke, and out theory is that r= eggie uses some kind of queue which grows more than it can be processed.<br= > <br> Does any one have an idea on how to dig into this?<br> <br> Env: Linu= x Debian, Sun Java 1.5&1.6<br> <br> Kind regards<br> /Per<br></div></di= v> ------------------------------------------------------------------------= -- Getting Started: <a href=3D"http://www.jini.org/wiki/Category:Getting_St= arted" target=3D"_blank">http://www.jini.org/wiki/Category:Getting_Started<= /a> Community Web Site: <a href=3D"http://jini.org" target=3D"_blank">http:= //jini.org</a> jini-users Archive: <a href=3D"http://archives.java.sun.com/= archives/jini-users.html" target=3D"_blank">http://archives.java.sun.com/ar= chives/jini-users.html</a> Unsubscribing: email "signoff JINI-USERS&qu= ot; to <a href=3D"mailto:[email protected]" target=3D"_blank">listserv@= java.sun.com</a><br> </blockquote> <br> </blockquote></div><br><br clear=3D"all"><br>-- <br>***= *****************************************<br>Per Edlund<br><a href=3D"http:= //www.bozoka.com" target=3D"_blank">www.bozoka.com</a><br><a href=3D"mailto= :[email protected]" target=3D"_blank">[email protected]</a><br> +46(0)8 545 068 05<br>+46(0)703 71 70 02<br>******************************= **************<br> --------------------------------------------------------= ------------------ Getting Started: <a href=3D"http://www.jini.org/wiki= /Category:Getting_Started" target=3D"_blank">http://www.jini.org/wiki/Categ= ory:Getting_Started</a> Community Web Site: <a href=3D"http://jini.org" ta= rget=3D"_blank">http://jini.org</a> jini-users Archive: <a href=3D"http://= archives.java.sun.com/archives/jini-users.html" target=3D"_blank">http://ar= chives.java.sun.com/archives/jini-users.html</a> Unsubscribing: email= "signoff JINI-USERS" to <a href=3D"mailto:[email protected]= " target=3D"_blank">[email protected]</a></blockquote> </div></div></div><br></div></blockquote></div><br><br clear=3D"all"><br>--= <br>********************************************<br><br>Per Edlund Bozoka.= com, <a href=3D"mailto:[email protected]">[email protected]</a><br>Web: =A0 =A0 <= a href=3D"http://www.bozoka.com">www.bozoka.com</a><br> Office: +46(0)8 545 068 05<br>Mobile:+46(0)703 71 70 02<br>Address:<br> =A0= Tegn=E9rgatan 28, 2tr<br> =A0113 59 Stockholm<br> =A0Sweden<br>************= ********************************<br> -------------------------------------------------------------------------- Getting Started: http://www.jini.org/wiki/Category:Getting_Started Community Web Site: http://jini.org jini-users Archive: http://archives.java.sun.com/archives/jini-users.html Unsubscribing: email "signoff JINI-USERS" to [email protected] --0016364592e83c542a046d8f30d2--