Re: recovering potentially corrupt data

Rohit Mathur <[email protected]> Tue, 16 May 2006 16:39:48 -0500
Newsgroups gmane.text.xml.xindice.user
Message-ID <[email protected]>
Hi Todd,

OK, i'll tell the users about this. They arent going to be happy.
But at least i can pass the buck onto you now :)

No, seriously though, it probably was an unclean tomcat shutdown that 
caused this.

thanks for all your help.

regards,

Rohit

Todd Byrne wrote:

>Thanks that helps allot but it appears you have truncated database files
>which isn't going to be recoverable. I am going to come up with a fix to
>better represent this to the user but the data in that collection is
>pretty much lost.
>
>So the way I would try to recover the rest of the data is to
>1) shutdown tomcat.
>2) Go to /software/jakarta-tomcat-5.0.30/webapps/xindice/WEB-INF/
>3) remove "db/globalJobs1/globalJobs1.tbl"
>4) startup tomcat
>5)run $XINDICE_HOME/xindice.sh export -v -c
>xmldb:xindice://localhost:8080/db -n db -f backup
>
>If you get any more errors just find the collection name and delete
>corresponding ".tbl" file.  Then just repeat the steps again until it
>runs cleanly. Take note of which files you delete or make a back up by
>copying the entire db directory somewhere else.
>
>This problem usually happens with a unclean shutdown of tomcat, power
>failure that time of thing.
>
>Hope this helps. If you have any questions let me know.
>
>Todd
>
>
>Rohit Mathur wrote:
>  
>
>>Hi Todd
>>
>>Below is the exception i see in catalina.out . However, i get this
>>exception when i try to access the globalJobs1 through the db browser on
>>my internet browser. I have been executing the export command using the
>>core xindice install (i.e. not the webapp install)
>>I think its the same error though, occurs when parsing through the
>>colleciton.
>>
>>I tried to run xindice.sh in the webapps/xindice/WEB-INF dir for
>>exporting, but couldnt get it to work.
>>
>>after setting JAVA_HOME,
>>
>>export XINDICE_HOME=/software/jakarta-tomcat-5.0.30/webapps/xindice/WEB-INF/
>>[root@mathur WEB-INF]# ./xindice.sh export -v -c 
>>xmldb:xindice://localhost:8080/db -n globalJobs1 -f globalJobs1
>>bash: ./xindice.sh: Permission denied
>>
>>and after changing permissions
>>
>>[root@mathur WEB-INF]# ./xindice.sh export -v -c 
>>xmldb:xindice://localhost:8080/db -n globalJobs1 -f globalJobs1
>>: bad interpreter: No such file or directory
>>
>>regards,
>>
>>Rohit
>>
>>hope this is good enough
>>
>>May 16, 2006 2:56:40 PM org.apache.xindice.core.filer.BTree getBTreeNode
>>WARNING: Ignored exception
>>java.io.EOFException
>>        at java.io.DataInputStream.readShort(DataInputStream.java:377)
>>        at
>>org.apache.xindice.core.filer.BTree$BTreeNode.read(BTree.java:495)
>>        at org.apache.xindice.core.filer.BTree.getBTreeNode(BTree.java:372)
>>        at org.apache.xindice.core.filer.BTree.access$800(BTree.java:71)
>>        at
>>org.apache.xindice.core.filer.BTree$BTreeNode.getChildNode(BTree.java:536)
>>        at
>>org.apache.xindice.core.filer.BTree$BTreeNode.query(BTree.java:937)
>>        at
>>org.apache.xindice.core.filer.BTree$BTreeNode.query(BTree.java:937)
>>        at org.apache.xindice.core.filer.BTree.query(BTree.java:221)
>>        at
>>org.apache.xindice.core.filer.BTreeFiler$BTreeFilerRecordSet.<init>(BTreeFiler.java:191)
>>        at
>>org.apache.xindice.core.filer.BTreeFiler.getRecordSet(BTreeFiler.java:174)
>>        at
>>org.apache.xindice.core.Collection.listDocuments(Collection.java:1007)
>>        at
>>org.apache.xindice.server.UglyBrowser.getHierarchy(UglyBrowser.java:157)
>>        at org.apache.xindice.server.UglyBrowser.doGet(UglyBrowser.java:78)
>>        at
>>org.apache.xindice.server.XindiceServlet.doGet(XindiceServlet.java:82)
>>        at javax.servlet.http.HttpServlet.service(HttpServlet.java:689)
>>        at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
>>        at
>>org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:252
>>)
>>        at
>>org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
>>        at
>>org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:214)
>>        at
>>org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104)
>>        at
>>org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520)
>>        at
>>org.apache.catalina.core.StandardContextValve.invokeInternal(StandardContextValve.java:198)
>>        at
>>org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:152)
>>        at
>>org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104)
>>        at
>>org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520)
>>        at
>>org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:137)
>>        at
>>org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104)
>>        at
>>org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:118)
>>        at
>>org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:102)
>>        at
>>org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520)
>>        at
>>org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
>>        at
>>org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104)
>>        at
>>org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520)
>>        at
>>org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:929)
>>        at
>>org.apache.coyote.tomcat5.CoyoteAdapter.service(CoyoteAdapter.java:160)
>>        at
>>org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:799)
>>        at
>>org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.processConnection(Http11Protocol
>>.java:705)
>>        at
>>org.apache.tomcat.util.net.TcpWorkerThread.runIt(PoolTcpEndpoint.java:577)
>>        at
>>org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684)
>>        at java.lang.Thread.run(Thread.java:534)
>>
>>Todd Byrne wrote:
>>
>>    
>>
>>>Hi Rohit,
>>>
>>>So I need you to run that export command again and send me the expection
>>>thats going to be printed in your catalina.log . It should look like :
>>>
>>>Ignored exception
>>>(stack trace)
>>>
>>>If I can get that stack trace maybe I can write up a patch and you can
>>>still recover your data.
>>>
>>>Todd
>>>
>>>Rohit Mathur wrote:
>>> 
>>>
>>>      
>>>
>>>>Hi Todd,
>>>>
>>>>sorry about that last message, since i saw the error i was expecting, i
>>>>didnt think to check the export command syntax.
>>>>
>>>>   
>>>>
>>>>        
>>>>
>>>>>xindice.sh export -v -c  xmldb:xindice://localhost:8080/db -n
>>>>>     
>>>>>
>>>>>          
>>>>>
>>>>globalJobs1 -f globalJobs1
>>>>trying to register database
>>>>
>>>>Creating directory globalJobs1/db
>>>>Extracting 0 files from xmldb:xindice://localhost:8080/db
>>>>Creating directory globalJobs1/db/products
>>>>Extracting 0 files from xmldb:xindice://localhost:8080/db/products
>>>>...............
>>>>Creating directory globalJobs1/db/site
>>>>Extracting 1 files from xmldb:xindice://localhost:8080/db/site
>>>>Creating directory globalJobs1/db/jobs
>>>>Extracting 0 files from xmldb:xindice://localhost:8080/db/jobs
>>>>Creating directory globalJobs1/db/jobs/globalJobs1
>>>>XMLDB Exception 0: Failed to execute command 'ListDocuments' on server:
>>>>http://localhost:8080/xindice/,message: java.lang.Exception:
>>>>java.lang.NullPointerException
>>>>org.xmldb.api.base.XMLDBException: Failed to execute command
>>>>'ListDocuments' on server: http://localhost:8080/xindice/, message:
>>>>java.lang.Exception: java.lang.NullPointerException
>>>>       at
>>>>org.apache.xindice.client.xmldb.xmlrpc.CollectionImpl.runRemoteCommand(CollectionImpl.java:124)
>>>>       at
>>>>org.apache.xindice.client.xmldb.xmlrpc.CollectionImpl.listResources(CollectionImpl.java:488)
>>>>       .....      
>>>>Caused by: org.apache.xmlrpc.XmlRpcException: java.lang.Exception:
>>>>java.lang.NullPointerException
>>>>       at org.apache.xmlrpc.XmlRpcClient$Worker.execute(XmlRpcClient.java)
>>>>       ... 8 more
>>>>[[email protected]: java.lang.Exception:
>>>>java.lang.NullPointerException
>>>>        ....
>>>>       at
>>>>org.apache.xindice.tools.command.ExportTree.process(ExportTree.java:148)
>>>>       at
>>>>org.apache.xindice.tools.command.ExportTree.execute(ExportTree.java:71)
>>>>       at org.apache.xindice.tools.XMLTools.execute(XMLTools.java:293)
>>>>       at org.apache.xindice.tools.XMLTools.process(XMLTools.java:171)
>>>>       at org.apache.xindice.tools.XMLTools.main(XMLTools.java:82)
>>>>
>>>>and listing the document is where i encountered the null pointer error
>>>>in the first place.
>>>>I am able to list other documents/collections in this db
>>>>
>>>>   
>>>>
>>>>        
>>>>
>>>>>xindice.sh ld -v -c  xmldb:xindice://localhost:8080/db/jobs/globalJobs1
>>>>>     
>>>>>
>>>>>          
>>>>>
>>>>trying to register database
>>>>XMLDB Exception 0: Failed to execute command 'ListDocuments' on server:
>>>>http://localhost:8080/xindice/,message: java.lang.Exception:
>>>>java.lang.NullPointerException
>>>>org.xmldb.api.base.XMLDBException: Failed to execute command
>>>>'ListDocuments' on server: http://localhost:8080/xindice/, message:
>>>>java.lang.Exception: java.lang.NullPointerException
>>>>       at
>>>>org.apache.xindice.client.xmldb.xmlrpc.CollectionImpl.runRemoteCommand(CollectionImpl.java:124)
>>>>       at
>>>>org.apache.xindice.client.xmldb.xmlrpc.CollectionImpl.listResources(CollectionImpl.java:488)
>>>>       at
>>>>org.apache.xindice.tools.command.ListCollectionDocuments.execute(ListCollectionDocuments.java:55)
>>>>       at org.apache.xindice.tools.XMLTools.execute(XMLTools.java:293)
>>>>       at org.apache.xindice.tools.XMLTools.process(XMLTools.java:171)
>>>>       at org.apache.xindice.tools.XMLTools.main(XMLTools.java:82)
>>>>Caused by: org.apache.xmlrpc.XmlRpcException: java.lang.Exception:
>>>>java.lang.NullPointerException
>>>>       at org.apache.xmlrpc.XmlRpcClient$Worker.execute(XmlRpcClient.java)
>>>>       at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java)
>>>>       at
>>>>org.apache.xindice.client.xmldb.xmlrpc.CollectionImpl.runRemoteCommand(CollectionImpl.java:112)
>>>>       ... 5 more
>>>>[[email protected]: java.lang.Exception:
>>>>java.lang.NullPointerException
>>>>       at org.apache.xmlrpc.XmlRpcClient$Worker.execute(XmlRpcClient.java)
>>>>       at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java)
>>>>       at
>>>>org.apache.xindice.client.xmldb.xmlrpc.CollectionImpl.runRemoteCommand(CollectionImpl.java:112)
>>>>       at
>>>>org.apache.xindice.client.xmldb.xmlrpc.CollectionImpl.listResources(CollectionImpl.java:488)
>>>>       at
>>>>org.apache.xindice.tools.command.ListCollectionDocuments.execute(ListCollectionDocuments.java:55)
>>>>       at org.apache.xindice.tools.XMLTools.execute(XMLTools.java:293)
>>>>       at org.apache.xindice.tools.XMLTools.process(XMLTools.java:171)
>>>>       at org.apache.xindice.tools.XMLTools.main(XMLTools.java:82)
>>>>
>>>>hope this helps/is fixable
>>>>
>>>>thanks a lot for your quick responses. i'm new to this project, and i
>>>>havent worked with xindice before so i'm all at sea here.
>>>>
>>>>Rohit
>>>>
>>>>Todd Byrne wrote:
>>>>
>>>>   
>>>>
>>>>        
>>>>
>>>>>I wasn't quite clear but try this:
>>>>>
>>>>>xindice.sh export -v -c  xmldb:xindice://localhost:8080/db -n
>>>>>globalJobs1 -f globalJobs1
>>>>>
>>>>>Also try listing the documents from database and retriving them one by one.
>>>>>
>>>>>$XINDICE_HOME/bin/xindice ld -c
>>>>>xmldb:xindice://localhost:8080/db/globalJobs1
>>>>>
>>>>>Todd
>>>>>
>>>>>
>>>>>Rohit Mathur wrote:
>>>>>
>>>>>
>>>>>     
>>>>>
>>>>>          
>>>>>
>>>>>>Hi Todd,
>>>>>>
>>>>>>So the original db was running off an application that was a wrapper
>>>>>>around xindice, which is why i didnt want to send details.
>>>>>>I have since copied the db over to another machine where I'm running the
>>>>>>xindice 1.1b4.
>>>>>>
>>>>>>I tried what you recommended on a subset of the db (i.e. db/jobs).
>>>>>>Here is the error i get
>>>>>>
>>>>>>$ xindice.sh export -v -c  xmldb:xindice://localhost:8080/db/jobs -n
>>>>>>test -f test
>>>>>>trying to register database
>>>>>>
>>>>>>Creating directory test/jobs
>>>>>>Extracting 0 files from xmldb:xindice://localhost:8080/db/jobs
>>>>>>Creating directory test/jobs/globalJobs1
>>>>>>XMLDB Exception 0: Failed to execute command 'ListDocuments' on server:
>>>>>>http://localhost:8080/xindice/,message: java.lang.Exception:
>>>>>>java.lang.NullPointerException
>>>>>>org.xmldb.api.base.XMLDBException: Failed to execute command
>>>>>>'ListDocuments' on server: http://localhost:8080/xindice/, message:
>>>>>>java.lang.Exception: java.lang.NullPointerException
>>>>>>      at
>>>>>>org.apache.xindice.client.xmldb.xmlrpc.CollectionImpl.runRemoteCommand(CollectionImpl.java:124)
>>>>>>      at
>>>>>>org.apache.xindice.client.xmldb.xmlrpc.CollectionImpl.listResources(CollectionImpl.java:488)
>>>>>>      at
>>>>>>org.apache.xindice.tools.command.ExportTree.process(ExportTree.java:111)
>>>>>>      at
>>>>>>org.apache.xindice.tools.command.ExportTree.process(ExportTree.java:148)
>>>>>>      at
>>>>>>org.apache.xindice.tools.command.ExportTree.execute(ExportTree.java:71)
>>>>>>      at org.apache.xindice.tools.XMLTools.execute(XMLTools.java:293)
>>>>>>      at org.apache.xindice.tools.XMLTools.process(XMLTools.java:171)
>>>>>>      at org.apache.xindice.tools.XMLTools.main(XMLTools.java:82)
>>>>>>Caused by: org.apache.xmlrpc.XmlRpcException: java.lang.Exception:
>>>>>>java.lang.NullPointerException
>>>>>>      at org.apache.xmlrpc.XmlRpcClient$Worker.execute(XmlRpcClient.java)
>>>>>>      at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java)
>>>>>>      at
>>>>>>org.apache.xindice.client.xmldb.xmlrpc.CollectionImpl.runRemoteCommand(CollectionImpl.java:112)
>>>>>>      ... 7 more
>>>>>>[[email protected]: java.lang.Exception:
>>>>>>java.lang.NullPointerException
>>>>>>      at org.apache.xmlrpc.XmlRpcClient$Worker.execute(XmlRpcClient.java)
>>>>>>      at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java)
>>>>>>      at
>>>>>>org.apache.xindice.client.xmldb.xmlrpc.CollectionImpl.runRemoteCommand(CollectionImpl.java:112)
>>>>>>      at
>>>>>>org.apache.xindice.client.xmldb.xmlrpc.CollectionImpl.listResources(CollectionImpl.java:488)
>>>>>>      at
>>>>>>org.apache.xindice.tools.command.ExportTree.process(ExportTree.java:111)
>>>>>>      at
>>>>>>org.apache.xindice.tools.command.ExportTree.process(ExportTree.java:148)
>>>>>>      at
>>>>>>org.apache.xindice.tools.command.ExportTree.execute(ExportTree.java:71)
>>>>>>      at org.apache.xindice.tools.XMLTools.execute(XMLTools.java:293)
>>>>>>      at org.apache.xindice.tools.XMLTools.process(XMLTools.java:171)
>>>>>>      at org.apache.xindice.tools.XMLTools.main(XMLTools.java:82)
>>>>>>
>>>>>>thanks,
>>>>>>
>>>>>>Rohit
>>>>>>
>>>>>>P.S. I am able to add/remove collections from db/jobs, but not from
>>>>>>db/globalJobs1
>>>>>>
>>>>>>[email protected] wrote:
>>>>>>
>>>>>>  
>>>>>>
>>>>>>       
>>>>>>
>>>>>>            
>>>>>>
>>>>>>>You could try running  xindice command line tool like this to export the 
>>>>>>>data and then reload it.
>>>>>>>
>>>>>>>$XINDICE_HOME/bin/xindice export -c  xmldb:xindice://localhost:8080/db/ -n test -f test
>>>>>>>
>>>>>>>Maybe I can work with you to narrow down where this null pointer is coming 
>>>>>>>from. Are you using the latest release?
>>>>>>>
>>>>>>>Todd
>>>>>>>
>>>>>>>
>>>>>>>On Fri, 12 May 2006, Rohit Mathur wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>    
>>>>>>>
>>>>>>>         
>>>>>>>
>>>>>>>              
>>>>>>>
>>>>>>>>Hi,
>>>>>>>>
>>>>>>>>Is there any utility out there that can recover the data from a collection.
>>>>>>>>
>>>>>>>>Is it at all possible to do this at all?
>>>>>>>>
>>>>>>>>One particular collection has "gone bad". The rest of the db is working 
>>>>>>>>great.
>>>>>>>>I am unable to query it or create a collection within it.
>>>>>>>>
>>>>>>>>I get a null pointer exception from the data.
>>>>>>>>
>>>>>>>>thanks
>>>>>>>>
>>>>>>>>Rohit
>>>>>>>>
>>>>>>>>java.lang.NullPointerException at 
>>>>>>>>org.apache.xindice.core.filer.BTree$BTreeNode.query(BTree.java:937) at 
>>>>>>>>org.apache.xindice.core.filer.BTree$BTreeNode.query(BTree.java:937) at 
>>>>>>>>org.apache.xindice.core.filer.BTree.query(BTree.java:221) at 
>>>>>>>>org.apache.xindice.core.filer.BTreeFiler$BTreeFilerRecordSet.(BTreeFiler.java:191) 
>>>>>>>>at 
>>>>>>>>org.apache.xindice.core.filer.BTreeFiler.getRecordSet(BTreeFiler.java:174) 
>>>>>>>>at 
>>>>>>>>org.apache.xindice.core.Collection.listDocuments(Collection.java:1007) 
>>>>>>>>at 
>>>>>>>>org.apache.xindice.server.UglyBrowser.getHierarchy(UglyBrowser.java:157) 
>>>>>>>>at org.apache.xindice.server.UglyBrowser.doGet(UglyBrowser.java:78) at 
>>>>>>>>org.apache.xindice.server.XindiceServlet.doGet(XindiceServlet.java:82) 
>>>>>>>>at javax.servlet.http.HttpServlet.service(HttpServlet.java:740) at 
>>>>>>>>javax.servlet.http.HttpServlet.service(HttpServlet.java:853) at 
>>>>>>>>org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:247) 
>>>>>>>>at 
>>>>>>>>org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) 
>>>>>>>>at 
>>>>>>>>org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:260) 
>>>>>>>>at 
>>>>>>>>org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:643) 
>>>>>>>>at 
>>>>>>>>org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480) 
>>>>>>>>at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995) 
>>>>>>>>at 
>>>>>>>>org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) 
>>>>>>>>at 
>>>>>>>>org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:643) 
>>>>>>>>at 
>>>>>>>>org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480) 
>>>>>>>>at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995) 
>>>>>>>>at 
>>>>>>>>org.apache.catalina.core.StandardContext.invoke(StandardContext.java:2415) 
>>>>>>>>at 
>>>>>>>>org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:180) 
>>>>>>>>at 
>>>>>>>>org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:643) 
>>>>>>>>at 
>>>>>>>>org.apache.catalina.valves.ErrorDispatcherValve.invoke(ErrorDispatcherValve.java:170) 
>>>>>>>>at 
>>>>>>>>org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:641) 
>>>>>>>>at 
>>>>>>>>org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:172) 
>>>>>>>>at 
>>>>>>>>org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:641) 
>>>>>>>>at 
>>>>>>>>org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:509) 
>>>>>>>>at 
>>>>>>>>org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:641) 
>>>>>>>>at 
>>>>>>>>org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480) 
>>>>>>>>at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995) 
>>>>>>>>at 
>>>>>>>>org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:174) 
>>>>>>>>at 
>>>>>>>>org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:643) 
>>>>>>>>at 
>>>>>>>>org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480) 
>>>>>>>>at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995) 
>>>>>>>>at 
>>>>>>>>org.apache.coyote.tomcat4.CoyoteAdapter.service(CoyoteAdapter.java:223) 
>>>>>>>>at 
>>>>>>>>org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:432) 
>>>>>>>>at 
>>>>>>>>org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.processConnection(Http11Protocol.java:386) 
>>>>>>>>at 
>>>>>>>>org.apache.tomcat.util.net.TcpWorkerThread.runIt(PoolTcpEndpoint.java:534) 
>>>>>>>>at 
>>>>>>>>org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:530) 
>>>>>>>>at java.lang.Thread.run(Thread.java:534)
>>>>>>>>
>>>>>>>> 
>>>>>>>>
>>>>>>>>      
>>>>>>>>
>>>>>>>>           
>>>>>>>>
>>>>>>>>                
>>>>>>>>
>>>>>>>    
>>>>>>>
>>>>>>>         
>>>>>>>
>>>>>>>              
>>>>>>>