RE: [bdbxml] memory leak and out of available locker entries

I'm getting the same problem, and have tried closing and reinitialising
objects all the bdbxml objects, but the JVM still grabs up to 800Mb
during queries...and sticks!


________________________________

	From: Matthew Ma [mailto:[email protected]] 
	Sent: 30 August 2005 06:46 pm
	To: [email protected]
	Subject: [bdbxml] memory leak and out of available locker
entries
	
	
	Hello,
	
	We use dbxml v 1.2 in a production environment and have been
experiencing a severe memory leak sometimes rendering the database
unusable with db_recover returning "out of available locker entries".
	
	We've tightened down everything that we could think of - 
	calling .delete() and .close() on every XmlResults,
XmlQueryContext and XmlDocument object.  In fact, we even went as far as
calling XmlDocument.getContentAsString() so that we could 
	XmlDocument.delete(); 
	XmlDocument.close(); 
	XmlDocument=null
	
	in case the XmlDocument objects were keeping handles open inside
Db.  
	
	We also close the XmlContainer objects by calling
	XmlContainer.close();
	XmlContainer = null;
	
	We also implemented periodic checking on the transactions inside
Dbxml by logging
	log.info("txnStat:" + env.txnStat(Db.DB_STAT_CLEAR).toString());
	
	which consistently produces something like:
	INFO KaRMI SocketThread-4
com.lightspoke.dbx.dao.DbxContainerManager  - txnStat:DbTxnStat:
	  st_last_ckp=com.sleepycat.db.DbLsn@2d802f
	  st_time_ckp=1125200718
	  st_last_txnid=-2147446531
	  st_maxtxns=20
	  st_naborts=0
	  st_nbegins=1
	  st_ncommits=1
	  st_nactive=0
	  st_nrestores=0
	  st_maxnactive=1
	  st_txnarray=
	  st_region_wait=0
	  st_region_nowait=4
	  st_regsize=16384
	
	But just to be sure, we even force all transactions to commit
periodically using the following code:
	
	DbPreplist[] txnlst = env.txnRecover(maxWorkingTxn,
Db.DB_FIRST);
	log.info("txn list size: " + txnlst.length);
	for (int t=0;t<txnlst.length;t++)       {
	  txnlst[t].txn.commit(Db.DB_TXN_NOSYNC);
	} //for txn
	
	But this piece of code never executes because we never have an
active transaction - which suggests that we are successfully committing
all transactions we start.
	
	The only object that we can see if being kept alive and out of
the garbage collector is then the DbEnv object which stays in memory
until we shut down our server wrapper around Dbxml.  
	
	We instantiate it like this:
	
	private int CFLAG =
Db.DB_CREATE|Db.DB_INIT_LOCK|Db.DB_INIT_LOG|Db.DB_INIT_MPOOL|Db.DB_INIT_
TXN;
	env = new DbEnv(0);
	env.setLockMaxLockers(dbMaxLockers);
	env.setLockMaxLocks(dbLockMaxLocks);
	env.setLockMaxObjects(dbLockMaxObjects);
	env.setCacheSize(  dbCacheSize, dbCacheNum );
	env.setLockDetect(Db.DB_LOCK_OLDEST);
	env.setErrorStream(System.err);
	env.open( dbxmlpath , CFLAG , 0);
	env.setErrorStream(System.err);
	
	How do we go about finding where the memory consumption is
happening?  We've run Jprobe on the server wrapper around dbxml but the
java portion of our code never goes beyond 50MB of memory usage
according to Jprobe - while the jvm running the code can easily run up
to 800MB after which it slows the entire server down to a crawl.  Once
we restart our wrapper (which closes the DbEnv object) we get all that
memory back.
	
	Any ideas on how to track this down or if we are missing
something totally obvious would be greatly appreciated.  
	
	Thank you in advance,
	
	-Matthew
	
	http://www.lightspoke.com
	Lightspoke Web-based Database