getIteratorByQuery + thousands of objects = OutOfMemoryError

"Vasily Ivanov" <[email protected]> Fri, 16 Nov 2007 19:26:13 +1100
Newsgroups gmane.comp.jakarta.ojb.user
Message-ID <[email protected]>
Hello,

We've got a system that needs to update thousands (possibly hundreds
of thousands) of Objects.

After reading OJB FAQ I thought that
PersistenceBroker.getIteratorByQuery(query) would be the best solution
in this case. Query runs fine and returns RsIterator. However the
problems start when I try to iterate and process few thousands of
records eventually I get OutOfMemoryError (out of java heap). I did
profiling and found that QueryReferenceBroker object holds a big
ArrayList called prefetchingListeners that grows rapidly while
iterating. I think that's because OJB keeps all objects to commit when
PersistenceBroker is closed.

What I do is I call the following statement after each iteration:
((PersistenceBrokerInternal)
pb).getReferenceBroker().removePrefetchingListeners();

That helped but that's obviously not the best way of doing it and I
don't know how thread-safe it is.

Is there any more appropriate way of doing that?

Thank you,
Vasily