Re: Outrigger memory leak?

So, the possible fly in the ointment is the harvester container aspect.

Wondering how that interacts with outrigger and how that might be
different from the normal environment you (JM) run your tests in.

Based on talk of not being able to drop-in replace outrigger with blitz,
it sounded like the plugin/container integration was some non-trivial
piece of work - more than just an inert wrapper anyways maybe.

Dan.

John McClain - Sun Microsystems, Inc. wrote:
> Well, I would have hoped that 1000 byte payloads would work. In the past
> I have run plenty of tests with 1000 byte payloads and tens of thousands
> of iterations (probably with bigger heaps though)....
>
> Did the problem crop up in some new space client you are writing, or
> some existing client?
>
> Very strange....
>
> Greg Trasuk wrote:
>> Hi John:
>>
>>       Thank you for the reply.  See answers interspersed. Oddly
>> enough, I've
>> just begun having difficulty replicating the problem on Solaris
>>
>> Cheers,
>>
>> Greg.
>>
>> On Wed, 2007-01-03 at 17:50, John McClain - Sun Microsystems, Inc.
>> wrote:
>>> Hmmm, seems odd - not sure why your test client would result in OoMEs. I
>>> assume your entry is just a Uuid and a byte[]?
>>>
>> My apologies for not including it before -
>> public class SampleEntry implements Serializable,Entry  {
>>
>>     public Uuid uuid=null;
>>
>>     public Object payload=null;
>>
>>
>> }
>>
>>> Given your access patters I think it makes sense that change the
>>> reapingInternal or Priority didn't have any effect. The takes your are
>>> doing should have the save the same effect as the reaping thread.
>>>
>> I haven't dug deep enough in the source code for Outrigger yet; I
>> wondered if the reaping happened solely in the background, or if it
>> would be done when the take() is done.
>>
>>> Random thought - Java will throw OoME for other conditions besides
>>> running out of memory, it could have memory but no block is big enough
>>> to satisfy the requested allocation, it could be out of threads (seems
>>> unlikely that would be the case here though). Don't suppose there is any
>>> sort of useful stack trace from the OoME?
>>>
>> See Exhibit A below.  I don't see anything significant in it, but you
>> might.
>>
>>> What was the smallest payload size you tried? 250KB seems a bit large (I
>>> may just be showing my age - when I was your age all we had was 128
>>> bytes, and WE LIKED IT...)
>> Yeah, yeah, I had to save up for six months to put that second 4K into
>> the OSI Challenger 4P....You tell that to the kids today, they won't
>> believe you.
>>
>> Anyway, the tests looked like this:
>>
>> Payload(bytes)                Iterations to fail      Total
>> throughput(MB)=Payload*Iterations
>> =======================================================================================
>>
>> 1000000                                 197                   197
>>  500000                                 225                   112.5
>>  250000                                1342                   335
>>  100000                                3018                   301.8
>>   50000                                8410                   420.5
>>   25000                                8194                   204.8
>>   10000                               11336                   113.4
>>    5000                               22330                   111.7
>>    2500                               32976                    82.4
>>    1000                               53471                    53.5
>>
>> All tests were run with -Xmx128m.
>>
>>> I could imagine the GC is getting into some
>>> odd state, when you let it sit idle for a short time how long would you
>>> wait? Was the waiting between testSpace calls or the between the write
>>> and take?
>>>
>> My hypothesis was that reaping happened on a background thread and it
>> didn't get to run when the main I/O threads were ping-ponging back and
>> forth.  So I gave it 100mS between testSpace calls; hopefully one or two
>> time slices.  No effect.  Also, stopping the test partway through and
>> letting Outrigger sit for several minutes had no effect on the memory
>> usage (as seen by the Netbeans profiler).
>
>
> --
> John McClain                                    [email protected]
> Sun Microsystems, Inc.
> Burlington, MA
>
> And it is that way today. We are tricked by hope into starting
> companies, beginning books, immigrating to this country and investing
> in telecom networks. The challenges turn out to be tougher than we
> imagined. Our excessive optimism is exposed. New skills are demanded.
> But nothing important was ever begun in a prudential frame of mind.
>
>         - David Brooks
>
> ===========================================================================
> To unsubscribe, send email to [email protected] and include in the body
> of the message "signoff JAVASPACES-USERS".  For general help, send email to
> [email protected] and include in the body of the message "help".
>
> To view past JAVASPACES-USERS postings, please see:
> http://archives.java.sun.com/archives/javaspaces-users.html
>

===========================================================================
To unsubscribe, send email to [email protected] and include in the body
of the message "signoff JAVASPACES-USERS".  For general help, send email to
[email protected] and include in the body of the message "help".

To view past JAVASPACES-USERS postings, please see:
http://archives.java.sun.com/archives/javaspaces-users.html