Re: Watch out for memory leaks with EhCache

On Wed, Dec 11, 2019 at 1:59 PM John Huss <[email protected]> wrote:

>
>
> On Wed, Dec 11, 2019 at 10:59 AM Andrus Adamchik <[email protected]>
> wrote:
>
>>
>>
>> > On Dec 10, 2019, at 9:39 PM, John Huss <[email protected]> wrote:
>> >
>> >> My projects are set up to use soft references using "
>> >> cayenne.server.object_retain_strategy=soft". I wouldn't expect this to
>> >> behave differently than weak under memory pressure, but my unit test
>> with
>> >> soft is not releasing this memory. I'll try running the test with
>> "weak"
>> >> and see if that changes it. And I'll look at your project (thanks for
>> >> that).
>> >>
>> >
>> > To clarify for future readers, we're talking about objects being
>> retained
>> > by this reference path:
>> > QueryCache -> cached_object -> ObjectContext ->
>> > other_objects_that_weren't_cached
>> >
>> > Sorry, I was wrong about the cause here. The persistent objects ARE
>> > released by the context's objectStore (whether using weak or soft). But
>> the
>> > context is still retaining a lot of extra memory. I'm having a hard time
>> > determining the specific cause. It might be the dataRowCache?
>>
>> If you have "Use Shared Cache" unchecked in the modeler, then you get a
>> single dataRowCache per context, so that would definitely explain it. I am
>> using the default - one dataRowCache per stack, shared by all contexts, so
>> that's never a problem.
>>
>
> I've narrowed it down - the "extra" memory being retained by the
> ObjectContext is: entries in ObjectStore.objectMap - but not DataObjects
> themselves (those get cleared), it's just the references to those objects:
> the mapping from ObjectId -> WeakReference. These entries stay present
> after the WeakReference is cleared. All those ObjectIds (though small) add
> up to a significant amount of memory over time. It looks like these are
> supposed to be cleared ReferenceMap.checkReferenceQueue(), which calls
> ReferenceQueue.poll() to find the cleared WeakReferences and remove those
> entries from the ObjectStore's objectMap. However, poll() doesn't ever seem
> to return any results (a cleared WeakReference). If I add in a call to
> ReferenceQueue.remove(5) manually (which will block for five milliseconds
> while it finds cleared references), it does return them and clear that
> memory. I need to read more about ReferenceQueue, but the current
> implementation of checkReferenceQueue() does not appear to be working since
> poll() never returns anything.
>

This is actually a timing problem, not a problem with poll().
checkReferenceQueue() would need to be called *after* the context is done
being used *and* the gc has occurred. As it currently is
checkReferenceQueue() is called while the context is being used, which can
help clear *some* memory that can be freed while in use, but doesn't help
after the context is unused (like when it is just sitting being referenced
by an object in the QueryCache.


>
>>
>> Andrus
>>
>>