RE: Deadlock in Cache

Hi Serge,

Thanks for that.  I note these revisions on the following branches for
this change:

4.0 - 14113
4.1 - 14117
TRUNK - 14118

Has this patch been tested against a system setup including JMS cache
replication?

Regards
Brett Randall


-----Original Message-----
From: Serge Huber [mailto:[email protected]] 
Sent: Thursday, 4 May 2006 19:07
To: [email protected]
Subject: Re: Deadlock in Cache


Hi Brett,

I have committed modifications to the cache system so that we don't lock

everything on a cache flush. This will be in the next release of the 
4.0.x and 4.1.x versions.

Regards,
Serge Huber.

Brett Randall wrote:
>
> We have a production site suffering the same intermittent deadlock 
> issue identified in http://www.jahia.net/jira/browse/JAHIA-639 . This 
> is a Jahia 4.0.5 system. The result is a completely deadlocked Jahia 
> engine, no rendering, no authoring.
>
> I'm enquiring whether this bug is likely to receive any attention in 
> the near future, given the severity.
>
> Stack traces below - I will also update the Jira.
>
> Found one Java-level deadlock:
>
> =============================
>
> "TP-Processor70593":
>
> waiting to lock monitor 0x080cc0bc (object 0x92ffe878, a 
> org.jahia.services.cache.Cache),
>
> which is held by "TP-Processor70118"
>
> "TP-Processor70118":
>
> waiting to lock monitor 0x080cc0f4 (object 0x92ffe8a8, a 
> org.jahia.services.cache.Cache),
>
> which is held by "TP-Processor70593"
>
> Java stack information for the threads listed above:
>
> ===================================================
>
> "TP-Processor70593":
>
> at org.jahia.services.cache.Cache.putCacheEntry(Cache.java:280)
>
> - waiting to lock <0x92ffe878> (a org.jahia.services.cache.Cache)
>
> at org.jahia.services.cache.Cache.put(Cache.java:259)
>
> at org.jahia.services.cache.Cache.put(Cache.java:235)
>
> at 
>
org.jahia.registries.JahiaContainerDefinitionsRegistry.loadAllDefinition
s(JahiaContainerDefinitionsRegistry.java:106)
>
> - locked <0x92ffc790> (a 
> org.jahia.registries.JahiaContainerDefinitionsRegistry)
>
> at 
>
org.jahia.registries.JahiaContainerDefinitionsRegistry.onCacheFlush(Jahi
aContainerDefinitionsRegistry.java:301)
>
> at org.jahia.services.cache.Cache.flush(Cache.java:442)
>
> - locked <0x92ffe8a8> (a org.jahia.services.cache.Cache)
>
> at org.jahia.services.cache.Cache.flush(Cache.java:412)
>
> at 
>
org.jahia.services.cache.CacheFactory.flushAllCaches(CacheFactory.java:3
56)
>
> - locked <0x92ffe8a8> (a org.jahia.services.cache.Cache)
>
> - locked <0x92c46d70> (a org.jahia.services.cache.CacheFactory) at 
>
org.jahia.admin.status.ManageStatus.processSettings(ManageStatus.java:19
0)
>
> at 
>
org.jahia.admin.status.ManageStatus.userRequestDispatcher(ManageStatus.j
ava:111)
>
> at org.jahia.admin.status.ManageStatus.<init>(ManageStatus.java:91)
>
> at 
>
org.jahia.bin.JahiaAdministration.userRequestDispatcher(JahiaAdministrat
ion.java:374)
>
> at
org.jahia.bin.JahiaAdministration.service(JahiaAdministration.java:245)
>
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:809)
>
> at 
>
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Applica
tionFilterChain.java:200)
>
> <snip>
>
> "TP-Processor70118":
>
> at org.jahia.services.cache.Cache.putCacheEntry(Cache.java:280)
>
> - waiting to lock <0x92ffe8a8> (a org.jahia.services.cache.Cache)
>
> at org.jahia.services.cache.Cache.put(Cache.java:259)
>
> at org.jahia.services.cache.Cache.put(Cache.java:235)
>
> at 
>
org.jahia.registries.JahiaContainerDefinitionsRegistry.loadDefinitionByI
D(JahiaContainerDefinitionsRegistry.java:122)
>
> at 
>
org.jahia.registries.JahiaContainerDefinitionsRegistry.getDefinition(Jah
iaContainerDefinitionsRegistry.java:162)
>
> - locked <0x92ffe878> (a org.jahia.services.cache.Cache)
>
> at 
>
org.jahia.data.containers.JahiaContainerStructure.<init>(JahiaContainerS
tructure.java:75)
>
> at 
>
org.jahia.services.containers.JahiaContainerStructuresDB.db_load_contain
er_structure(JahiaContainerStructuresDB.java:88)
>
> at 
>
org.jahia.services.containers.JahiaContainerDefinitionsDB.db_load_contai
ner_definition(JahiaContainerDefinitionsDB.java:124)
>
> at 
>
org.jahia.services.containers.JahiaContainersBaseService.loadContainerDe
finition(JahiaContainersBaseService.java:2053)
>
> at 
>
org.jahia.registries.JahiaContainerDefinitionsRegistry.loadDefinitionByI
D(JahiaContainerDefinitionsRegistry.java:115)
>
> at 
>
org.jahia.registries.JahiaContainerDefinitionsRegistry.getDefinition(Jah
iaContainerDefinitionsRegistry.java:162)
>
> - locked <0x92ffe878> (a org.jahia.services.cache.Cache)
>
> at 
>
org.jahia.data.containers.JahiaContainerStructure.<init>(JahiaContainerS
tructure.java:75)
>
> at 
>
org.jahia.services.containers.JahiaContainerStructuresDB.db_load_contain
er_structure(JahiaContainerStructuresDB.java:88)
>
> at 
>
org.jahia.services.containers.JahiaContainerDefinitionsDB.db_load_contai
ner_definition(JahiaContainerDefinitionsDB.java:124)
>
> at 
>
org.jahia.services.containers.JahiaContainersBaseService.loadContainerDe
finition(JahiaContainersBaseService.java:2053)
>
> at 
>
org.jahia.registries.JahiaContainerDefinitionsRegistry.loadDefinitionByI
D(JahiaContainerDefinitionsRegistry.java:115)
>
> at 
>
org.jahia.registries.JahiaContainerDefinitionsRegistry.getDefinition(Jah
iaContainerDefinitionsRegistry.java:162)
>
> - locked <0x92ffe878> (a org.jahia.services.cache.Cache)
>
> at 
>
org.jahia.engines.containerlistproperties.ContainerListProperties_Engine
.renderLink(ContainerListProperties_Engine.java:113)
>
> at org.jahia.gui.GuiBean.drawUrlCheckWriteAccess(GuiBean.java:720)
>
> at
org.jahia.gui.GuiBean.drawContainerListPropertiesUrl(GuiBean.java:488)
>
> at 
>
org.jahia.gui.HTMLToolBox.drawContainerListPropertiesLauncher(HTMLToolBo
x.java:536)
>
> at 
>
org.jahia.gui.HTMLToolBox.drawContainerListPropertiesLauncher(HTMLToolBo
x.java:515)
>
> - locked <0xa1f16b80> (a org.jahia.gui.HTMLToolBox)
>
> at 
>
org.jahia.taglibs.url.ContainerListPropertiesURLTag.getJSPopupURL(Contai
nerListPropertiesURLTag.java:69)
>
> at 
>
org.jahia.taglibs.url.ContainerListPropertiesURLTag.getIdObject(Containe
rListPropertiesURLTag.java:43)
>
> at
org.jahia.taglibs.url.AbstractURLTag.doStartTag(AbstractURLTag.java:91)
>
> at org.apache.jsp.columns_jsp._jspService(columns_jsp.java:3858)
>
> at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:92)
>
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:809)
>
> at 
>
org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.ja
va:162)
>
> at 
>
org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:240)
>
> at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:187)
>
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:809)
>
> at 
>
org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatc
her.java:627)
>
> at 
>
org.apache.catalina.core.ApplicationDispatcher.doInclude(ApplicationDisp
atcher.java:518)
>
> at 
>
org.apache.catalina.core.ApplicationDispatcher.include(ApplicationDispat
cher.java:448)
>
> at 
>
org.apache.jasper.runtime.JspRuntimeLibrary.include(JspRuntimeLibrary.ja
va:768)
>
> at 
>
org.apache.jsp.general_columns_jsp._jspService(general_columns_jsp.java:
1895)
>
> at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:92)
>
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:809)
>
> at 
>
org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.ja
va:162)
>
> at 
>
org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:240)
>
> at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:187)
>
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:809)
>
> at 
>
org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatc
her.java:627)
>
> at 
>
org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDisp
atcher.java:382)
>
> at 
>
org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispat
cher.java:306)
>
> at
org.jahia.engines.EngineRenderer.renderCore(EngineRenderer.java:182)
>
> at org.jahia.engines.EngineRenderer.render(EngineRenderer.java:78)
>
> at
org.jahia.engines.core.Core_Engine.processCore(Core_Engine.java:186)
>
> at
org.jahia.engines.core.Core_Engine.handleActions(Core_Engine.java:107)
>
> at 
>
org.jahia.operations.OperationManager.handleOperations(OperationManager.
java:279)
>
> at org.jahia.bin.JahiaAction.execute(JahiaAction.java:50)
>
> at 
>
org.apache.struts.action.RequestProcessor.processActionPerform(RequestPr
ocessor.java:484)
>
> at 
>
org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:
274)
>
> at
org.apache.struts.action.ActionServlet.process(ActionServlet.java:1482)
>
> at org.jahia.bin.Jahia.process(Jahia.java:1522)
>
> at org.jahia.bin.Jahia.service(Jahia.java:846)
>
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:809)
>
> at 
>
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Applica
tionFilterChain.java:200)
>
> <snip>
>
> Found 1 deadlock.
>
> Thanks
>
> Brett Randall
>