Re: "Collection /db/system/security/exist/accounts cannot be created."
Michael Westbay <[email protected]> Fri, 25 Jul 2025 09:35:41 +0900
| Newsgroups | gmane.text.xml.exist |
|---|---|
| Message-ID | <CAJ6MVuNmGnoH=CGzoeTkHCe1WoXVZZUgfM1g7rdoLibzS=iE1w@mail.gmail.com> |
--===============8530102589614200701== Content-Type: multipart/alternative; boundary="000000000000b89e30063ab62069" --000000000000b89e30063ab62069 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi Tony, Okay, the "Failed to open database" message concerns me. under /usr/local/src/exist3-data do you have the following files? -rw-rw-r--. 1 exist exist 1451 Jul 25 08:57 00000000d6.log -rw-rw-r--. 1 exist exist 13991936 Jul 24 22:40 collections.dbx -rw-r--r--. 1 exist exist 1515 Jul 24 22:38 counters -rw-rw-r--. 1 exist exist 16 Jul 25 08:57 dbx_dir.lck -rw-rw-r--. 1 exist exist 6785642496 Jul 24 22:38 dom.dbx drwxrwxr-x. 15 exist exist 4096 Sep 27 2020 expathrepo drwxrwxr-x. 3 exist exist 132 May 20 12:14 export drwxrwxr-x. 3 exist exist 16 Apr 3 2018 fs drwxrwxr-x. 2 exist exist 6 Jul 21 20:47 fs.journal -rw-rw-r--. 1 exist exist 140 Apr 16 2018 jmxservlet.token -rw-rw-r--. 1 exist exist 16 Jul 25 08:57 journal.lck drwxrwxr-x. 2 exist exist 103 Jul 23 2022 lucene -rw-rw-r--. 1 exist exist 4636672 May 22 21:46 ngram.dbx drwxrwxr-x. 2 exist exist 8192 Jul 25 08:44 range -rw-rw-r--. 1 exist exist 120 Apr 9 2018 README -rw-rw-r--. 1 exist exist 257 Apr 9 2018 RECOVERY -rw-------. 1 exist exist 11 Mar 8 10:20 restxq.registry drwxrwxr-x. 2 exist exist 42 Jul 23 2022 sanity -rw-rw-r--. 1 exist exist 8192 Jul 23 2022 sort.dbx -rw-rw-r--. 1 exist exist 6046371840 Jul 24 22:40 structure.dbx -rw-rw-r--. 1 exist exist 63342 Jul 21 20:35 symbols.dbx -rw-rw-r--. 1 exist exist 46583808 Jul 24 22:40 values.dbx The dom.dbx file should be very large. That's where all of your data is at. If this file is broken, then the data isn't recoverable. The RECOVERY file states: RECOVERY =3D=3D=3D=3D=3D=3D=3D=3D If the index got corrupted, this is how to recover: - stop database - remove all ".log" transaction log files - remove all ".lck" lock files - remove all ".dbx" files *EXCEPT dom.dbx, collections.dbx and symbols.dbx* - start database I really don't like doing it without a backup. So first, make a tarball of /usr/local/src/exist3-data as a backup of the raw data then start the recovery process above. And once the database is up and running again, run: cd /usr/local/src/exist3 bin/backup.sh --dir ~/tmp/2025-07-25-backup --backup /db And this is my script modified for your setup that I run daily via crontab: #!/bin/sh unset JAVA_OPTS export JAVA_HOME=3D/usr export EXIST_HOME=3D/usr/local/src/exist3 export TARGET=3D/tmp export DATE=3D`date +"%Y-%m-%d"` export PATH=3D$JAVA_HOME/bin:/usr/local/bin:/usr/bin:/bin cd $EXIST_HOME bin/backup.sh --backup /db --dir $TARGET/full-$DATE -u "admin" -p "password= " I thought it was supposed to get the username and password from the $EXIST_HOME/backup.properties file, but when I tested it by hand just now without the -u/-p parameters it failed. Not sure why. Anyway, daily backups going forward would be a very good idea. Take care. 2025=E5=B9=B47=E6=9C=8824=E6=97=A5(=E6=9C=A8) 23:21 Tony Graham <tgraham@an= tenna.co.jp>: > Michael, > > Thanks for following through with this. > > On 23/07/2025 09:00, Michael Westbay wrote: > > Okay. Let's first find EXIST_HOME by finding where the conf.xml file is= . > > > > find /usr/local -name conf.xml > > It found multiple, but the ones of interest are: > > /usr/local/src/exist3/installer/conf.xml > /usr/local/src/exist3/conf.xml > > > If /usr/local is EXIST_HOME, then run.sh will be in /usr/local/bin alon= g > > with all of the other eXist executables. > > The only things in '/usr/local/bin' are 'tracd' and 'trac-admin' for use > with Trac. (Still the best issue tracking/wiki system.) > > > This conf.xml file will then specify where the data directory is: > > '/usr/local/src/exist3/conf.xml' has: > > <db-connection cacheSize=3D"256M" checkMaxCacheSize=3D"true" > collectionCache=3D"64M" database=3D"native" > files=3D"/usr/local/src/exist3-data" pageSize=3D"4096" > nodesBuffer=3D"1000" cacheShrinkThreshold=3D"10000" > doc-ids=3D"default" minDiskSpace=3D"128M"> > > > You could also search for run.sh the same way: > > > > find /usr/local -name run.sh > > As before, I don't have a 'run.sh' (unless you count 'run.sh' for > Antenna House Formatter). > > Running the Java installer as a non-root user has never given me a > 'run.sh'. > > I downloaded the 'run.sh' [1] for that eXist-db version from GitHub, > edited it to use the same Java alternative as used for the database, ran > it, and got: > > ---- > cd "/usr/local/src/exist3/" ; ./bin/run.sh org.exist.backup.ExportMain > -x -d /mnt/tmp > ERROR: Failed to open database: > org.exist.storage.BrokerPoolServiceException: org.exist.EXistException: > java.io.IOException: Collection /db/system/security/exist/accounts > cannot be created. > ---- > > If this can be solved, the problem then becomes that we don't know how > many other collections will have similar problems, so I'm not hopeful. > > Regards, > > > Tony Graham. > -- > Senior Architect > XML Division > Antenna House, Inc. > ---- > Skerries, Ireland > [email protected] > > [1] > > https://raw.githubusercontent.com/eXist-db/exist/refs/tags/eXist-3.0.RC2/= bin/run.sh > > > _______________________________________________ > Exist-open mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/exist-open > --=20 Michael Westbay Writer/System Administrator http://www.japanesebaseball.com/ --000000000000b89e30063ab62069 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div>Hi Tony,</div><div><br></div><div>Okay, the "Fai= led to open database" message concerns me.</div><div><br></div><div>un= der /usr/local/src/exist3-data do you have the following files?</div><div><= br></div><div style=3D"margin-left:40px">-rw-rw-r--. =C2=A01 exist exist=C2= =A0 =C2=A0 =C2=A0 =C2=A01451 Jul 25 08:57 00000000d6.log<br>-rw-rw-r--. =C2= =A01 exist exist=C2=A0 =C2=A013991936 Jul 24 22:40 collections.dbx<br>-rw-r= --r--. =C2=A01 exist exist=C2=A0 =C2=A0 =C2=A0 =C2=A01515 Jul 24 22:38 coun= ters</div><div style=3D"margin-left:40px">-rw-rw-r--. =C2=A01 exist exist= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A016 Jul 25 08:57 dbx_dir.lck<br>-rw-rw-r--= . =C2=A01 exist exist 6785642496 Jul 24 22:38 dom.dbx<br>drwxrwxr-x. 15 exi= st exist=C2=A0 =C2=A0 =C2=A0 =C2=A04096 Sep 27 =C2=A02020 expathrepo<br>drw= xrwxr-x. =C2=A03 exist exist=C2=A0 =C2=A0 =C2=A0 =C2=A0 132 May 20 12:14 ex= port<br>drwxrwxr-x. =C2=A03 exist exist=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A016= Apr =C2=A03 =C2=A02018 fs<br>drwxrwxr-x. =C2=A02 exist exist=C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 6 Jul 21 20:47 fs.journal<br>-rw-rw-r--. =C2=A01 exist= exist=C2=A0 =C2=A0 =C2=A0 =C2=A0 140 Apr 16 =C2=A02018 jmxservlet.token</d= iv><div style=3D"margin-left:40px">-rw-rw-r--. =C2=A01 exist exist=C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A016 Jul 25 08:57 journal.lck<br>drwxrwxr-x. =C2= =A02 exist exist=C2=A0 =C2=A0 =C2=A0 =C2=A0 103 Jul 23 =C2=A02022 lucene<br= >-rw-rw-r--. =C2=A01 exist exist=C2=A0 =C2=A0 4636672 May 22 21:46 ngram.db= x<br>drwxrwxr-x. =C2=A02 exist exist=C2=A0 =C2=A0 =C2=A0 =C2=A08192 Jul 25 = 08:44 range<br>-rw-rw-r--. =C2=A01 exist exist=C2=A0 =C2=A0 =C2=A0 =C2=A0 1= 20 Apr =C2=A09 =C2=A02018 README<br>-rw-rw-r--. =C2=A01 exist=C2=A0exist=C2= =A0 =C2=A0 =C2=A0 =C2=A0 257 Apr =C2=A09 =C2=A02018 RECOVERY<br>-rw-------.= =C2=A01 exist=C2=A0exist=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A011 Mar =C2=A08 1= 0:20 restxq.registry</div><div style=3D"margin-left:40px">drwxrwxr-x. =C2= =A02 exist exist=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A042 Jul 23 =C2=A02022 sani= ty<br>-rw-rw-r--. =C2=A01 exist exist=C2=A0 =C2=A0 =C2=A0 =C2=A08192 Jul 23= =C2=A02022 sort.dbx<br>-rw-rw-r--. =C2=A01 exist=C2=A0exist 6046371840 Jul= 24 22:40 structure.dbx<br>-rw-rw-r--. =C2=A01 exist exist=C2=A0 =C2=A0 =C2= =A0 63342 Jul 21 20:35 symbols.dbx<br>-rw-rw-r--. =C2=A01 exist exist=C2=A0= =C2=A046583808 Jul 24 22:40 values.dbx</div><div><br></div><div>The dom.db= x file should be very large. That's where all of your data is at. If th= is file is</div><div>broken, then the data isn't recoverable.=C2=A0</di= v><div><br></div><div>The RECOVERY file states:</div><div><br></div><div st= yle=3D"margin-left:40px">RECOVERY<br>=3D=3D=3D=3D=3D=3D=3D=3D<br>If the ind= ex got corrupted, this is how to recover:<br><br>- stop database<br>- remov= e all ".log" transaction log files<br>- remove all ".lck&quo= t; lock files<br>- remove all ".dbx" files <b>EXCEPT dom.dbx, col= lections.dbx and symbols.dbx</b></div><div style=3D"margin-left:40px">- sta= rt database</div><div><br></div><div>I really don't like doing it witho= ut a backup. So first, make a tarball of=C2=A0/usr/local/src/exist3-data</d= iv><div>as a backup of the raw data then start the recovery process above.<= /div><div><br></div><div>And once the database is up and running again, run= :</div><div><br></div><div style=3D"margin-left:40px">cd /usr/local/src/exi= st3</div><div style=3D"margin-left:40px">bin/backup.sh --dir ~/tmp/2025-07-= 25-backup --backup /db</div><div><br></div><div>And this is my script modif= ied for your setup that I run daily via crontab:</div><div><br></div><div s= tyle=3D"margin-left:40px">#!/bin/sh<br>unset JAVA_OPTS<br>export JAVA_HOME= =3D/usr</div><div style=3D"margin-left:40px">export EXIST_HOME=3D/usr/local= /src/exist3</div><div style=3D"margin-left:40px">export TARGET=3D/tmp<br>ex= port DATE=3D`date +"%Y-%m-%d"`<br>export PATH=3D$JAVA_HOME/bin:/u= sr/local/bin:/usr/bin:/bin<br>cd $EXIST_HOME<br>bin/backup.sh --backup /db = --dir $TARGET/full-$DATE -u "admin" -p "password"</div>= <div><br></div><div>I thought it was supposed to get the username and passw= ord from the $EXIST_HOME/backup.properties</div><div>file, but when I teste= d it by hand just now without the -u/-p parameters it failed. Not sure why.= </div><div><br></div><div>Anyway, daily backups going forward would be a ve= ry good idea.</div><div><br></div><div>Take care.</div></div><br><div class= =3D"gmail_quote gmail_quote_container"><div dir=3D"ltr" class=3D"gmail_attr= ">2025=E5=B9=B47=E6=9C=8824=E6=97=A5(=E6=9C=A8) 23:21 Tony Graham <<a hr= ef=3D"mailto:[email protected]">[email protected]</a>>:<br></div= ><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border= -left:1px solid rgb(204,204,204);padding-left:1ex">Michael,<br> <br> Thanks for following through with this.<br> <br> On 23/07/2025 09:00, Michael Westbay wrote:<br> > Okay. Let's first find EXIST_HOME by finding where the conf.xml fi= le is.<br> > <br> > find /usr/local -name conf.xml<br> <br> It found multiple, but the ones of interest are:<br> <br> /usr/local/src/exist3/installer/conf.xml<br> /usr/local/src/exist3/conf.xml<br> <br> > If /usr/local is EXIST_HOME, then run.sh will be in /usr/local/bin alo= ng <br> > with all of the other eXist executables.<br> <br> The only things in '/usr/local/bin' are 'tracd' and 'tr= ac-admin' for use<br> with Trac. (Still the best issue tracking/wiki system.)<br> <br> > This conf.xml file will then specify where the data directory is:<br> <br> '/usr/local/src/exist3/conf.xml' has:<br> <br> =C2=A0 =C2=A0 =C2=A0<db-connection cacheSize=3D"256M" checkMax= CacheSize=3D"true" <br> collectionCache=3D"64M" database=3D"native"<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0files=3D"/usr/local/src/exist3-data&= quot; pageSize=3D"4096" <br> nodesBuffer=3D"1000" cacheShrinkThreshold=3D"10000"<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0doc-ids=3D"default" minDiskSpac= e=3D"128M"><br> <br> > You could also search for run.sh the same way:<br> > <br> > find /usr/local -name run.sh<br> <br> As before, I don't have a 'run.sh' (unless you count 'run.s= h' for<br> Antenna House Formatter).<br> <br> Running the Java installer as a non-root user has never given me a 'run= .sh'.<br> <br> I downloaded the 'run.sh' [1] for that eXist-db version from GitHub= ,<br> edited it to use the same Java alternative as used for the database, ran<br= > it, and got:<br> <br> ----<br> cd "/usr/local/src/exist3/" ; ./bin/run.sh org.exist.backup.Expor= tMain <br> -x -d /mnt/tmp<br> ERROR: Failed to open database: <br> org.exist.storage.BrokerPoolServiceException: org.exist.EXistException: <br= > java.io.IOException: Collection /db/system/security/exist/accounts <br> cannot be created.<br> ----<br> <br> If this can be solved, the problem then becomes that we don't know how<= br> many other collections will have similar problems, so I'm not hopeful.<= br> <br> Regards,<br> <br> <br> Tony Graham.<br> -- <br> Senior Architect<br> XML Division<br> Antenna House, Inc.<br> ----<br> Skerries, Ireland<br> <a href=3D"mailto:[email protected]" target=3D"_blank">tgraham@antenna.= co.jp</a><br> <br> [1] <br> <a href=3D"https://raw.githubusercontent.com/eXist-db/exist/refs/tags/eXist= -3.0.RC2/bin/run.sh" rel=3D"noreferrer" target=3D"_blank">https://raw.githu= busercontent.com/eXist-db/exist/refs/tags/eXist-3.0.RC2/bin/run.sh</a><br> <br> <br> _______________________________________________<br> Exist-open mailing list<br> <a href=3D"mailto:[email protected]" target=3D"_blank">Exist= [email protected]</a><br> <a href=3D"https://lists.sourceforge.net/lists/listinfo/exist-open" rel=3D"= noreferrer" target=3D"_blank">https://lists.sourceforge.net/lists/listinfo/= exist-open</a><br> </blockquote></div><div><br clear=3D"all"></div><br><span class=3D"gmail_si= gnature_prefix">-- </span><br><div dir=3D"ltr" class=3D"gmail_signature">Mi= chael Westbay<br>Writer/System Administrator<br><a href=3D"http://www.japan= esebaseball.com/" target=3D"_blank">http://www.japanesebaseball.com/</a></d= iv> --000000000000b89e30063ab62069-- --===============8530102589614200701== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline --===============8530102589614200701== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Exist-open mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/exist-open --===============8530102589614200701==--