Re: recovering potentially corrupt data
Rohit Mathur <[email protected]> Tue, 16 May 2006 16:39:48 -0500
| Newsgroups | gmane.text.xml.xindice.user |
|---|---|
| Message-ID | <[email protected]> |
Hi Todd, OK, i'll tell the users about this. They arent going to be happy. But at least i can pass the buck onto you now :) No, seriously though, it probably was an unclean tomcat shutdown that caused this. thanks for all your help. regards, Rohit Todd Byrne wrote: >Thanks that helps allot but it appears you have truncated database files >which isn't going to be recoverable. I am going to come up with a fix to >better represent this to the user but the data in that collection is >pretty much lost. > >So the way I would try to recover the rest of the data is to >1) shutdown tomcat. >2) Go to /software/jakarta-tomcat-5.0.30/webapps/xindice/WEB-INF/ >3) remove "db/globalJobs1/globalJobs1.tbl" >4) startup tomcat >5)run $XINDICE_HOME/xindice.sh export -v -c >xmldb:xindice://localhost:8080/db -n db -f backup > >If you get any more errors just find the collection name and delete >corresponding ".tbl" file. Then just repeat the steps again until it >runs cleanly. Take note of which files you delete or make a back up by >copying the entire db directory somewhere else. > >This problem usually happens with a unclean shutdown of tomcat, power >failure that time of thing. > >Hope this helps. If you have any questions let me know. > >Todd > > >Rohit Mathur wrote: > > >>Hi Todd >> >>Below is the exception i see in catalina.out . However, i get this >>exception when i try to access the globalJobs1 through the db browser on >>my internet browser. I have been executing the export command using the >>core xindice install (i.e. not the webapp install) >>I think its the same error though, occurs when parsing through the >>colleciton. >> >>I tried to run xindice.sh in the webapps/xindice/WEB-INF dir for >>exporting, but couldnt get it to work. >> >>after setting JAVA_HOME, >> >>export XINDICE_HOME=/software/jakarta-tomcat-5.0.30/webapps/xindice/WEB-INF/ >>[root@mathur WEB-INF]# ./xindice.sh export -v -c >>xmldb:xindice://localhost:8080/db -n globalJobs1 -f globalJobs1 >>bash: ./xindice.sh: Permission denied >> >>and after changing permissions >> >>[root@mathur WEB-INF]# ./xindice.sh export -v -c >>xmldb:xindice://localhost:8080/db -n globalJobs1 -f globalJobs1 >>: bad interpreter: No such file or directory >> >>regards, >> >>Rohit >> >>hope this is good enough >> >>May 16, 2006 2:56:40 PM org.apache.xindice.core.filer.BTree getBTreeNode >>WARNING: Ignored exception >>java.io.EOFException >> at java.io.DataInputStream.readShort(DataInputStream.java:377) >> at >>org.apache.xindice.core.filer.BTree$BTreeNode.read(BTree.java:495) >> at org.apache.xindice.core.filer.BTree.getBTreeNode(BTree.java:372) >> at org.apache.xindice.core.filer.BTree.access$800(BTree.java:71) >> at >>org.apache.xindice.core.filer.BTree$BTreeNode.getChildNode(BTree.java:536) >> at >>org.apache.xindice.core.filer.BTree$BTreeNode.query(BTree.java:937) >> at >>org.apache.xindice.core.filer.BTree$BTreeNode.query(BTree.java:937) >> at org.apache.xindice.core.filer.BTree.query(BTree.java:221) >> at >>org.apache.xindice.core.filer.BTreeFiler$BTreeFilerRecordSet.<init>(BTreeFiler.java:191) >> at >>org.apache.xindice.core.filer.BTreeFiler.getRecordSet(BTreeFiler.java:174) >> at >>org.apache.xindice.core.Collection.listDocuments(Collection.java:1007) >> at >>org.apache.xindice.server.UglyBrowser.getHierarchy(UglyBrowser.java:157) >> at org.apache.xindice.server.UglyBrowser.doGet(UglyBrowser.java:78) >> at >>org.apache.xindice.server.XindiceServlet.doGet(XindiceServlet.java:82) >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:689) >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:802) >> at >>org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:252 >>) >> at >>org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173) >> at >>org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:214) >> at >>org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104) >> at >>org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520) >> at >>org.apache.catalina.core.StandardContextValve.invokeInternal(StandardContextValve.java:198) >> at >>org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:152) >> at >>org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104) >> at >>org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520) >> at >>org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:137) >> at >>org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104) >> at >>org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:118) >> at >>org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:102) >> at >>org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520) >> at >>org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) >> at >>org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104) >> at >>org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520) >> at >>org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:929) >> at >>org.apache.coyote.tomcat5.CoyoteAdapter.service(CoyoteAdapter.java:160) >> at >>org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:799) >> at >>org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.processConnection(Http11Protocol >>.java:705) >> at >>org.apache.tomcat.util.net.TcpWorkerThread.runIt(PoolTcpEndpoint.java:577) >> at >>org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684) >> at java.lang.Thread.run(Thread.java:534) >> >>Todd Byrne wrote: >> >> >> >>>Hi Rohit, >>> >>>So I need you to run that export command again and send me the expection >>>thats going to be printed in your catalina.log . It should look like : >>> >>>Ignored exception >>>(stack trace) >>> >>>If I can get that stack trace maybe I can write up a patch and you can >>>still recover your data. >>> >>>Todd >>> >>>Rohit Mathur wrote: >>> >>> >>> >>> >>>>Hi Todd, >>>> >>>>sorry about that last message, since i saw the error i was expecting, i >>>>didnt think to check the export command syntax. >>>> >>>> >>>> >>>> >>>> >>>>>xindice.sh export -v -c xmldb:xindice://localhost:8080/db -n >>>>> >>>>> >>>>> >>>>> >>>>globalJobs1 -f globalJobs1 >>>>trying to register database >>>> >>>>Creating directory globalJobs1/db >>>>Extracting 0 files from xmldb:xindice://localhost:8080/db >>>>Creating directory globalJobs1/db/products >>>>Extracting 0 files from xmldb:xindice://localhost:8080/db/products >>>>............... >>>>Creating directory globalJobs1/db/site >>>>Extracting 1 files from xmldb:xindice://localhost:8080/db/site >>>>Creating directory globalJobs1/db/jobs >>>>Extracting 0 files from xmldb:xindice://localhost:8080/db/jobs >>>>Creating directory globalJobs1/db/jobs/globalJobs1 >>>>XMLDB Exception 0: Failed to execute command 'ListDocuments' on server: >>>>http://localhost:8080/xindice/,message: java.lang.Exception: >>>>java.lang.NullPointerException >>>>org.xmldb.api.base.XMLDBException: Failed to execute command >>>>'ListDocuments' on server: http://localhost:8080/xindice/, message: >>>>java.lang.Exception: java.lang.NullPointerException >>>> at >>>>org.apache.xindice.client.xmldb.xmlrpc.CollectionImpl.runRemoteCommand(CollectionImpl.java:124) >>>> at >>>>org.apache.xindice.client.xmldb.xmlrpc.CollectionImpl.listResources(CollectionImpl.java:488) >>>> ..... >>>>Caused by: org.apache.xmlrpc.XmlRpcException: java.lang.Exception: >>>>java.lang.NullPointerException >>>> at org.apache.xmlrpc.XmlRpcClient$Worker.execute(XmlRpcClient.java) >>>> ... 8 more >>>>[[email protected]: java.lang.Exception: >>>>java.lang.NullPointerException >>>> .... >>>> at >>>>org.apache.xindice.tools.command.ExportTree.process(ExportTree.java:148) >>>> at >>>>org.apache.xindice.tools.command.ExportTree.execute(ExportTree.java:71) >>>> at org.apache.xindice.tools.XMLTools.execute(XMLTools.java:293) >>>> at org.apache.xindice.tools.XMLTools.process(XMLTools.java:171) >>>> at org.apache.xindice.tools.XMLTools.main(XMLTools.java:82) >>>> >>>>and listing the document is where i encountered the null pointer error >>>>in the first place. >>>>I am able to list other documents/collections in this db >>>> >>>> >>>> >>>> >>>> >>>>>xindice.sh ld -v -c xmldb:xindice://localhost:8080/db/jobs/globalJobs1 >>>>> >>>>> >>>>> >>>>> >>>>trying to register database >>>>XMLDB Exception 0: Failed to execute command 'ListDocuments' on server: >>>>http://localhost:8080/xindice/,message: java.lang.Exception: >>>>java.lang.NullPointerException >>>>org.xmldb.api.base.XMLDBException: Failed to execute command >>>>'ListDocuments' on server: http://localhost:8080/xindice/, message: >>>>java.lang.Exception: java.lang.NullPointerException >>>> at >>>>org.apache.xindice.client.xmldb.xmlrpc.CollectionImpl.runRemoteCommand(CollectionImpl.java:124) >>>> at >>>>org.apache.xindice.client.xmldb.xmlrpc.CollectionImpl.listResources(CollectionImpl.java:488) >>>> at >>>>org.apache.xindice.tools.command.ListCollectionDocuments.execute(ListCollectionDocuments.java:55) >>>> at org.apache.xindice.tools.XMLTools.execute(XMLTools.java:293) >>>> at org.apache.xindice.tools.XMLTools.process(XMLTools.java:171) >>>> at org.apache.xindice.tools.XMLTools.main(XMLTools.java:82) >>>>Caused by: org.apache.xmlrpc.XmlRpcException: java.lang.Exception: >>>>java.lang.NullPointerException >>>> at org.apache.xmlrpc.XmlRpcClient$Worker.execute(XmlRpcClient.java) >>>> at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java) >>>> at >>>>org.apache.xindice.client.xmldb.xmlrpc.CollectionImpl.runRemoteCommand(CollectionImpl.java:112) >>>> ... 5 more >>>>[[email protected]: java.lang.Exception: >>>>java.lang.NullPointerException >>>> at org.apache.xmlrpc.XmlRpcClient$Worker.execute(XmlRpcClient.java) >>>> at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java) >>>> at >>>>org.apache.xindice.client.xmldb.xmlrpc.CollectionImpl.runRemoteCommand(CollectionImpl.java:112) >>>> at >>>>org.apache.xindice.client.xmldb.xmlrpc.CollectionImpl.listResources(CollectionImpl.java:488) >>>> at >>>>org.apache.xindice.tools.command.ListCollectionDocuments.execute(ListCollectionDocuments.java:55) >>>> at org.apache.xindice.tools.XMLTools.execute(XMLTools.java:293) >>>> at org.apache.xindice.tools.XMLTools.process(XMLTools.java:171) >>>> at org.apache.xindice.tools.XMLTools.main(XMLTools.java:82) >>>> >>>>hope this helps/is fixable >>>> >>>>thanks a lot for your quick responses. i'm new to this project, and i >>>>havent worked with xindice before so i'm all at sea here. >>>> >>>>Rohit >>>> >>>>Todd Byrne wrote: >>>> >>>> >>>> >>>> >>>> >>>>>I wasn't quite clear but try this: >>>>> >>>>>xindice.sh export -v -c xmldb:xindice://localhost:8080/db -n >>>>>globalJobs1 -f globalJobs1 >>>>> >>>>>Also try listing the documents from database and retriving them one by one. >>>>> >>>>>$XINDICE_HOME/bin/xindice ld -c >>>>>xmldb:xindice://localhost:8080/db/globalJobs1 >>>>> >>>>>Todd >>>>> >>>>> >>>>>Rohit Mathur wrote: >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>>>Hi Todd, >>>>>> >>>>>>So the original db was running off an application that was a wrapper >>>>>>around xindice, which is why i didnt want to send details. >>>>>>I have since copied the db over to another machine where I'm running the >>>>>>xindice 1.1b4. >>>>>> >>>>>>I tried what you recommended on a subset of the db (i.e. db/jobs). >>>>>>Here is the error i get >>>>>> >>>>>>$ xindice.sh export -v -c xmldb:xindice://localhost:8080/db/jobs -n >>>>>>test -f test >>>>>>trying to register database >>>>>> >>>>>>Creating directory test/jobs >>>>>>Extracting 0 files from xmldb:xindice://localhost:8080/db/jobs >>>>>>Creating directory test/jobs/globalJobs1 >>>>>>XMLDB Exception 0: Failed to execute command 'ListDocuments' on server: >>>>>>http://localhost:8080/xindice/,message: java.lang.Exception: >>>>>>java.lang.NullPointerException >>>>>>org.xmldb.api.base.XMLDBException: Failed to execute command >>>>>>'ListDocuments' on server: http://localhost:8080/xindice/, message: >>>>>>java.lang.Exception: java.lang.NullPointerException >>>>>> at >>>>>>org.apache.xindice.client.xmldb.xmlrpc.CollectionImpl.runRemoteCommand(CollectionImpl.java:124) >>>>>> at >>>>>>org.apache.xindice.client.xmldb.xmlrpc.CollectionImpl.listResources(CollectionImpl.java:488) >>>>>> at >>>>>>org.apache.xindice.tools.command.ExportTree.process(ExportTree.java:111) >>>>>> at >>>>>>org.apache.xindice.tools.command.ExportTree.process(ExportTree.java:148) >>>>>> at >>>>>>org.apache.xindice.tools.command.ExportTree.execute(ExportTree.java:71) >>>>>> at org.apache.xindice.tools.XMLTools.execute(XMLTools.java:293) >>>>>> at org.apache.xindice.tools.XMLTools.process(XMLTools.java:171) >>>>>> at org.apache.xindice.tools.XMLTools.main(XMLTools.java:82) >>>>>>Caused by: org.apache.xmlrpc.XmlRpcException: java.lang.Exception: >>>>>>java.lang.NullPointerException >>>>>> at org.apache.xmlrpc.XmlRpcClient$Worker.execute(XmlRpcClient.java) >>>>>> at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java) >>>>>> at >>>>>>org.apache.xindice.client.xmldb.xmlrpc.CollectionImpl.runRemoteCommand(CollectionImpl.java:112) >>>>>> ... 7 more >>>>>>[[email protected]: java.lang.Exception: >>>>>>java.lang.NullPointerException >>>>>> at org.apache.xmlrpc.XmlRpcClient$Worker.execute(XmlRpcClient.java) >>>>>> at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java) >>>>>> at >>>>>>org.apache.xindice.client.xmldb.xmlrpc.CollectionImpl.runRemoteCommand(CollectionImpl.java:112) >>>>>> at >>>>>>org.apache.xindice.client.xmldb.xmlrpc.CollectionImpl.listResources(CollectionImpl.java:488) >>>>>> at >>>>>>org.apache.xindice.tools.command.ExportTree.process(ExportTree.java:111) >>>>>> at >>>>>>org.apache.xindice.tools.command.ExportTree.process(ExportTree.java:148) >>>>>> at >>>>>>org.apache.xindice.tools.command.ExportTree.execute(ExportTree.java:71) >>>>>> at org.apache.xindice.tools.XMLTools.execute(XMLTools.java:293) >>>>>> at org.apache.xindice.tools.XMLTools.process(XMLTools.java:171) >>>>>> at org.apache.xindice.tools.XMLTools.main(XMLTools.java:82) >>>>>> >>>>>>thanks, >>>>>> >>>>>>Rohit >>>>>> >>>>>>P.S. I am able to add/remove collections from db/jobs, but not from >>>>>>db/globalJobs1 >>>>>> >>>>>>[email protected] wrote: >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>>You could try running xindice command line tool like this to export the >>>>>>>data and then reload it. >>>>>>> >>>>>>>$XINDICE_HOME/bin/xindice export -c xmldb:xindice://localhost:8080/db/ -n test -f test >>>>>>> >>>>>>>Maybe I can work with you to narrow down where this null pointer is coming >>>>>>>from. Are you using the latest release? >>>>>>> >>>>>>>Todd >>>>>>> >>>>>>> >>>>>>>On Fri, 12 May 2006, Rohit Mathur wrote: >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>>Hi, >>>>>>>> >>>>>>>>Is there any utility out there that can recover the data from a collection. >>>>>>>> >>>>>>>>Is it at all possible to do this at all? >>>>>>>> >>>>>>>>One particular collection has "gone bad". The rest of the db is working >>>>>>>>great. >>>>>>>>I am unable to query it or create a collection within it. >>>>>>>> >>>>>>>>I get a null pointer exception from the data. >>>>>>>> >>>>>>>>thanks >>>>>>>> >>>>>>>>Rohit >>>>>>>> >>>>>>>>java.lang.NullPointerException at >>>>>>>>org.apache.xindice.core.filer.BTree$BTreeNode.query(BTree.java:937) at >>>>>>>>org.apache.xindice.core.filer.BTree$BTreeNode.query(BTree.java:937) at >>>>>>>>org.apache.xindice.core.filer.BTree.query(BTree.java:221) at >>>>>>>>org.apache.xindice.core.filer.BTreeFiler$BTreeFilerRecordSet.(BTreeFiler.java:191) >>>>>>>>at >>>>>>>>org.apache.xindice.core.filer.BTreeFiler.getRecordSet(BTreeFiler.java:174) >>>>>>>>at >>>>>>>>org.apache.xindice.core.Collection.listDocuments(Collection.java:1007) >>>>>>>>at >>>>>>>>org.apache.xindice.server.UglyBrowser.getHierarchy(UglyBrowser.java:157) >>>>>>>>at org.apache.xindice.server.UglyBrowser.doGet(UglyBrowser.java:78) at >>>>>>>>org.apache.xindice.server.XindiceServlet.doGet(XindiceServlet.java:82) >>>>>>>>at javax.servlet.http.HttpServlet.service(HttpServlet.java:740) at >>>>>>>>javax.servlet.http.HttpServlet.service(HttpServlet.java:853) at >>>>>>>>org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:247) >>>>>>>>at >>>>>>>>org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) >>>>>>>>at >>>>>>>>org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:260) >>>>>>>>at >>>>>>>>org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:643) >>>>>>>>at >>>>>>>>org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480) >>>>>>>>at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995) >>>>>>>>at >>>>>>>>org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) >>>>>>>>at >>>>>>>>org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:643) >>>>>>>>at >>>>>>>>org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480) >>>>>>>>at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995) >>>>>>>>at >>>>>>>>org.apache.catalina.core.StandardContext.invoke(StandardContext.java:2415) >>>>>>>>at >>>>>>>>org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:180) >>>>>>>>at >>>>>>>>org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:643) >>>>>>>>at >>>>>>>>org.apache.catalina.valves.ErrorDispatcherValve.invoke(ErrorDispatcherValve.java:170) >>>>>>>>at >>>>>>>>org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:641) >>>>>>>>at >>>>>>>>org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:172) >>>>>>>>at >>>>>>>>org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:641) >>>>>>>>at >>>>>>>>org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:509) >>>>>>>>at >>>>>>>>org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:641) >>>>>>>>at >>>>>>>>org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480) >>>>>>>>at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995) >>>>>>>>at >>>>>>>>org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:174) >>>>>>>>at >>>>>>>>org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:643) >>>>>>>>at >>>>>>>>org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480) >>>>>>>>at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995) >>>>>>>>at >>>>>>>>org.apache.coyote.tomcat4.CoyoteAdapter.service(CoyoteAdapter.java:223) >>>>>>>>at >>>>>>>>org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:432) >>>>>>>>at >>>>>>>>org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.processConnection(Http11Protocol.java:386) >>>>>>>>at >>>>>>>>org.apache.tomcat.util.net.TcpWorkerThread.runIt(PoolTcpEndpoint.java:534) >>>>>>>>at >>>>>>>>org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:530) >>>>>>>>at java.lang.Thread.run(Thread.java:534) >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>