Re: Exceptions due to freemarker's use of non-thread-safe DOM library

"Well, I think that a DOM implementation that has thread-safety issues when used for *read-only* access is very unfortunate design. "

Agreed... this is flat out ridiculous.  I've had quite a few other thread safety issues in the past due to this, but was always able to add a sync block around it in my code to prevent the issue.  Now with this one, I can't put a big fat sync block around the whole call to the servlet, and I can't easily extend your classes - the pieces that need the sync blocks are in the middle of large blocks or even private methods, so I'd essentially have to copy and paste the whole thing. Plus I don't even know exactly where all I'd have to put them, potentially a few dozen places.

This is the default DOM implementation with the jdk... I'm getting the feeling from your replies that you always use a different DOM implementation that doesn't have this problem?  = )  This is really a major issue - I've got an entire multi-threaded web application built around this library that is totally not thread safe, even for reads!  I'm thinking the smart thing to do would be to change the library, but I can't find one that thread safe by design.

"I don't think we'd need to introduce synchronization in our library to accommodate an implementation artifact of a particular data model in a completely unrelated library."

I'll also agree with that.  But I figured since this is the default jdk lib, freemarker should be happily compatible with that.  It's really not freemarker's fault, I'd put the fault on their design decision to be not thread safe.  It seems like everyone in general is quick to say 'we are not going to be thread safe, you deal with it', and myself included.  Xerces passed it to me, I'm trying to pass it to you, and you're passing it back.  =)  ah

If there's not a thread safe lib out there, I'll try to get something going with the cglib proxy.  Good suggestion.  Is there a better approach to sharing an xml document to multiple threads?

Thanks to you both for your prompt replies.



From: Attila Szegedi [mailto:[email protected]]
Sent: Friday, June 03, 2011 2:24 PM
To: FreeMarker-user
Subject: Re: [FreeMarker-user] Exceptions due to freemarker's use of non-thread-safe DOM library


I understand they're doing gradual parsing and thus have internal state, but that should really be an option you can turn off, as it's just an optimization that doesn't necessarily always apply.
You can, but it
The correct way to fix this, assuming you want to keep using this particular DOM implementation, is to create your own proxy classes that implement the DOM interfaces, wrap and delegate to the underlying DOM objects, synchronizing the operations on them. You can use CGLIB to create these proxies really quickly. Alternatively, you can come up with a wrapping/delegating+synchronizing freemarker.ext.xml.NodeModel implementation too.

I don't think we'd need to introduce synchronization in our library to accommodate an implementation artifact of a particular data model in a completely unrelated library.
I'll agree with that.  This is just the 'out of the box' behavior with the jdk however.  Clearly the issue is not with freemarker, but since this is the default


Attila.

On Jun 3, 2011, at 9:04 AM, Newman, John W wrote:


Hello,

In production under load, occasionally we'll get this stack trace from a template (using freemarker servlet).

java.lang.NullPointerException:
(no message)
at org.apache.xerces.dom.ParentNode.nodeListItem(Unknown Source)
at org.apache.xerces.dom.ParentNode.item(Unknown Source)
at freemarker.ext.dom.NodeListModel.<init>(NodeListModel.java:89)
at freemarker.ext.dom.NodeModel.getChildNodes(NodeModel.java:302)
at freemarker.ext.dom.ElementModel.get(ElementModel.java:124)
at freemarker.core.Dot._getAsTemplateModel(Dot.java:76)
at freemarker.core.Expression.getAsTemplateModel(Expression.java:89)
at freemarker.core.IteratorBlock.accept(IteratorBlock.java:94)
at freemarker.core.Environment.visit(Environment.java:209)
[ a few dozen stack frames visit/accept/macro/ifblock from template itself ]
at freemarker.core.Environment.visit(Environment.java:209)
at freemarker.core.Environment.process(Environment.java:189)
at freemarker.template.Template.process(Template.java:237)
at freemarker.ext.servlet.FreemarkerServlet.process(FreemarkerServlet.java:458)
at freemarker.ext.servlet.FreemarkerServlet.doGet(FreemarkerServlet.java:397)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:690)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)


Our application uses XML elements that are sometimes cloned from another element.  The freemarker templates do a lot of various xml operations, ${element.@Attribute}<mailto:$%7belement.@Attribute%7d> , ${element[xpath]}, etc.  things like that that normally works perfectly fine.  In investigating the stack trace, it really looks like https://issues.apache.org/jira/browse/XERCESJ-727

Please read through that, essentially the org.apache.xerces.dom library is deliberately not thread safe by design.  [This is news to me, a bit unexpected...]  Apparently these element implementations maintain a local cache of what has already been parsed through, and if a second thread jumps in it will end up stepping on this shared cache causing undefined behavior.  So the burden falls on the caller to synchronize on the elements before doing certain operations. (I'm not sure exactly which).

I've looked through the freemarker xml package a bit, and I didn't find any sync blocks.  I tried isolating the problem in there and getting a good repeatable test case, but unfortunately I don't understand your library well enough to do so.  The best test case I can provide you is attached - it's basically an adaptation of what was posted in that issue.  I think the freemarker library needs to add a few sync blocks here and there around these xml operations, but unfortunately I can't say exactly where, possibly quite a few places.  And hopefully it's not one big wide block on the outer method that would kill performance.  Please look at this and let me know what you think.  I was hoping to just get a patch to send, but after several hours I'm not having much luck.  A simple pojo test case is attached.

Thanks,
John

------------------------------------------------------------------------------
Simplify data backup and recovery for your virtual environment with vRanger.
Installation's a snap, and flexible recovery options mean your data is safe,
secure and there when you need it. Discover what all the cheering's about.
Get your free trial download today. 
http://p.sf.net/sfu/quest-dev2dev2

_______________________________________________
FreeMarker-user mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/freemarker-user