Re: Scaling Reggie

Mark Brouwer wrote:
> Dennis Reedy wrote:
>
>> Greetings,
>>
>> A note back to the group that the scalability challenges I
>> encountered at this time seem to be solved. The issue I found was the
>> configuration of a service proxy was done incorrectly, resulting in a
>> codebase of 90 MB (ouch). Once this was fixed, the deployment, with
>> 800+ clients discovering reggie and gaining access to the proxy in
>> question worked out of the box.
>>
>> I'm trying to document the root cause of this issue, and the question
>> I have is why would reggie get an out of memory exception (OOME)? The
>> service was able to register without issue, only when the clients
>> would discover the service would the OOME occur. The memory never
>> seemed to be reclaimed, once each client performed the discovery.
>> Does reggie make a copy of the proxy for each client?
>
>
> Hi Dennis,
>
> The codebase is part of the marshalled objects so that means that in
> case one service registers and you have loads of clients that all need
> to be notified the remote events become rather 'large'. Assuming that
> for each client the remote event is produced and queued I can understand
> that an OOME will eventually happen, depending on the retry strategy it
> seems to be possible for a LUS that such memory is not being reclaimed
> (event scheduled for retry e.g.).
>
> BTW nice denial of service attack such a large codebase ;-) It would be
> nice if Reggie could have a limit on the max size of any object
> registered, although that is probably the case for any object marshalled

This is an interesting issue.  The codebase, being downloaded to reggie, would
cause a new classloader, per client.  Java would then also create a jar index
and do some other things that consume memory to manage access to that codebase
jar.  For my clients, I almost always strive to have them use a
http://localhost:8080/ url for their codebase so that the services (which have
explicit knowledge of the clients 'platform', in general) use a single
classloader for all clients.  You have to be careful about static class data,
but otherwise, I've found this to be a good way to minimize memory use with
large numbers of clients.

Resource management is one of the things that my http://reef.dev.java.net
project tries to deal more explicitly with regarding service lookup.  The
deferred unmarshalling allows class loading to be managed and ordered more
explicitly so that you don't get into problems (as a client discovering 100s of
services) with mass unmarshalling.  Reggie's proxy could, perhaps do some more
explicit control of the notify() calling implementation to make it possible for
the service object to manage the inflow more explicitly.

I'm betting that it's simply the size of the jar indexes and URLClassLoader
counts that are creating the memory issue.  Perhaps those details end up using
non-heap memory which is sized fairly small?  I think that the total numer of
classes that can be loaded is impacted by non-heap memory size too right?

Gregg Wonderly

--------------------------------------------------------------------------
Getting Started:     http://www.jini.org/wiki/Category:Getting_Started
Community Web Site:  http://jini.org
jini-users Archive:  http://archives.java.sun.com/archives/jini-users.html
Unsubscribing:       email "signoff JINI-USERS"  to [email protected]