Re: Better Slide performance on a database, PUT hangs and Lucene creating out of memory troubles
Eirikur Hrafnsson <[email protected]>
| Newsgroups | gmane.comp.jakarta.slide.devel |
|---|---|
| Message-ID | <[email protected]> |
Hi
thanks for the tip. I wasn't using any "asynchron" property but
setting it to false changed nothing. I still get the out of memory.
The stacktrace goes like this:
26 Jul 2006 11:19:19 - org.apache.slide.transaction.SlideTransaction
- WARNING - Prepare failure: Resource manager
com.idega.slide.store.IWSlideStore@1386751 Error code XA_RBBASE in
Transaction 2 xid [B@158539f-[B@1da1a93 in thread TP-Processor1
javax.transaction.xa.XAException
at
org.apache.commons.transaction.util.xa.AbstractXAResource.prepare
(AbstractXAResource.java:122)
at com.idega.slide.store.AbstractSlideStore.prepare
(AbstractSlideStore.java:544)
at org.apache.slide.transaction.SlideTransaction.commit
(SlideTransaction.java:251)
at
org.apache.slide.transaction.SlideTransactionManager.commit
(SlideTransactionManager.java:186)
at org.apache.slide.common.NamespaceAccessTokenImpl.commit
(NamespaceAccessTokenImpl.java:390)
at
org.apache.slide.index.lucene.IndexInitializer.namespaceInitialized
(IndexInitializer.java:152)
at org.apache.slide.event.DomainEvent
$NamespaceInitialized.fireEvent(DomainEvent.java:74)
at org.apache.slide.event.EventDispatcher.fireEvent
(EventDispatcher.java:72)
at org.apache.slide.common.Domain.initNamespace(Domain.java:
929)
at org.apache.slide.common.Domain.init(Domain.java:462)
at org.apache.slide.common.Domain.init(Domain.java:375)
at org.apache.slide.common.Domain.init(Domain.java:338)
at org.apache.slide.webdav.WebdavServlet.init
(WebdavServlet.java:251)
at javax.servlet.GenericServlet.init(GenericServlet.java:211)
at org.apache.slide.webdav.WebdavServlet.init
(WebdavServlet.java:191)
at com.idega.servlet.ServletWrapper.init(ServletWrapper.java:
125)
at com.idega.slide.webdavservlet.WebdavExtendedServlet.init
(WebdavExtendedServlet.java:67)
at org.apache.catalina.core.StandardWrapper.loadServlet
(StandardWrapper.java:1091)
at org.apache.catalina.core.StandardWrapper.allocate
(StandardWrapper.java:750)
at org.apache.catalina.core.StandardWrapperValve.invoke
(StandardWrapperValve.java:130)
at org.apache.catalina.core.StandardContextValve.invoke
(StandardContextValve.java:178)
at org.apache.catalina.core.StandardHostValve.invoke
(StandardHostValve.java:126)
at org.apache.catalina.valves.ErrorReportValve.invoke
(ErrorReportValve.java:105)
at org.apache.catalina.core.StandardEngineValve.invoke
(StandardEngineValve.java:107)
at org.apache.catalina.connector.CoyoteAdapter.service
(CoyoteAdapter.java:148)
at org.apache.jk.server.JkCoyoteHandler.invoke
(JkCoyoteHandler.java:307)
at org.apache.jk.common.HandlerRequest.invoke
(HandlerRequest.java:385)
at org.apache.jk.common.ChannelSocket.invoke
(ChannelSocket.java:748)
at org.apache.jk.common.ChannelSocket.processConnection
(ChannelSocket.java:678)
at org.apache.jk.common.SocketConnection.runIt
(ChannelSocket.java:871)
at org.apache.tomcat.util.threads.ThreadPool
$ControlRunnable.run(ThreadPool.java:684)
at java.lang.Thread.run(Thread.java:534)
26 Jul 2006 11:19:19 - org.apache.slide.common.Domain - ERROR - Error
while commiting initialized properties index for scope / - null
26 Jul 2006 11:19:19 - org.apache.slide.common.Domain - ERROR -
javax.transaction.RollbackException
javax.transaction.RollbackException
at org.apache.slide.transaction.SlideTransaction.commit
(SlideTransaction.java:328)
at
org.apache.slide.transaction.SlideTransactionManager.commit
(SlideTransactionManager.java:186)
at org.apache.slide.common.NamespaceAccessTokenImpl.commit
(NamespaceAccessTokenImpl.java:390)
at
org.apache.slide.index.lucene.IndexInitializer.namespaceInitialized
(IndexInitializer.java:152)
at org.apache.slide.event.DomainEvent
$NamespaceInitialized.fireEvent(DomainEvent.java:74)
at org.apache.slide.event.EventDispatcher.fireEvent
(EventDispatcher.java:72)
at org.apache.slide.common.Domain.initNamespace(Domain.java:
929)
at org.apache.slide.common.Domain.init(Domain.java:462)
at org.apache.slide.common.Domain.init(Domain.java:375)
at org.apache.slide.common.Domain.init(Domain.java:338)
at org.apache.slide.webdav.WebdavServlet.init
(WebdavServlet.java:251)
at javax.servlet.GenericServlet.init(GenericServlet.java:211)
at org.apache.slide.webdav.WebdavServlet.init
(WebdavServlet.java:191)
at com.idega.servlet.ServletWrapper.init(ServletWrapper.java:
125)
at com.idega.slide.webdavservlet.WebdavExtendedServlet.init
(WebdavExtendedServlet.java:67)
at org.apache.catalina.core.StandardWrapper.loadServlet
(StandardWrapper.java:1091)
at org.apache.catalina.core.StandardWrapper.allocate
(StandardWrapper.java:750)
at org.apache.catalina.core.StandardWrapperValve.invoke
(StandardWrapperValve.java:130)
at org.apache.catalina.core.StandardContextValve.invoke
(StandardContextValve.java:178)
at org.apache.catalina.core.StandardHostValve.invoke
(StandardHostValve.java:126)
at org.apache.catalina.valves.ErrorReportValve.invoke
(ErrorReportValve.java:105)
at org.apache.catalina.core.StandardEngineValve.invoke
(StandardEngineValve.java:107)
at org.apache.catalina.connector.CoyoteAdapter.service
(CoyoteAdapter.java:148)
at org.apache.jk.server.JkCoyoteHandler.invoke
(JkCoyoteHandler.java:307)
at org.apache.jk.common.HandlerRequest.invoke
(HandlerRequest.java:385)
at org.apache.jk.common.ChannelSocket.invoke
(ChannelSocket.java:748)
at org.apache.jk.common.ChannelSocket.processConnection
(ChannelSocket.java:678)
at org.apache.jk.common.SocketConnection.runIt
(ChannelSocket.java:871)
at org.apache.tomcat.util.threads.ThreadPool
$ControlRunnable.run(ThreadPool.java:684)
at java.lang.Thread.run(Thread.java:534)
26 Jul 2006 11:19:19 - org.apache.slide.common.Domain - ERROR -
java.lang.OutOfMemoryError
java.lang.OutOfMemoryError
Any further ideas?
-Eiki
On 26.7.2006, at 09:37, Honoré David N. wrote:
>
>
> Eirikur Hrafnsson wrote:
>>
>> Secondly I have a problem with the RE-indexing of a Slide store. I
>> deleted the index and restarted tomcat and the RE-indexing starts.
>> Then the memory usage skyrockets, not by some tens of megabytes
>> but by
>> hundreds of precious MB's at once and then gives me a OUT OF MEMORY
>> error and a failed transaction when it stops at 1500MB of RAM!
>>
>> Has anyone had this problem? First of all, why does the Slide-Lucene
>> index use transactions? Its writing to files in the file system
>> right?
>> And what's with the memory feast?
>>
> Try to desactivate the async mode. Because while indexing (and content
> extraction), Slide keep the binary content of document into memory.
> In synchronized mode slide wait the end of the indexing of one
> document
> before loading the content of another for indexing.
> In Asynchone mode, all "IndexJob" (which reference a
> NodeRevisionContent
> with the binary content in memory) are put in a queue at the same
> time.
> Loading the content of all documents into stream in memory at the same
> time cause your OutOfMemoryError.
>
>> Best regards
>> Eiki
>>
>> ¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
>> ¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
>>
>> Eirikur S. Hrafnsson
>> Chief Software Engineer
>>
>> [email protected]
>>
>> ¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
>> ¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
>>
>> Idega hf. ı Engjavegi 6 ı 104 Reykjavík ı Iceland
>> Tel. +354 554 7557 ı Fax +354 885 7557
>>
>> http://www.idega.com ı [email protected]
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [email protected]
>> For additional commands, e-mail: [email protected]
>>
>>
> --
>
> ----------------------------------------------------------------------
> --
> Honoré David N.
> Projet DISSCO (http://www.meteo.be/DISSCO/)
>
> email: [email protected]
> tel: 02 37 30 530
> ----------------------------------------------------------------------
> --
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [email protected]
> For additional commands, e-mail: [email protected]
>