Re: "Dbc::get: Cannot allocate memory"

Yeah, I was having similar problems. There are a number of problems,  
but the latest code seems to be running more stable. In particular make  
sure you have the latest pathan installed. My site was going down a  
couple times a day, now it's been fine for about a week. The one thing  
that's still a problem is memory leaks, though that's much better now  
too. Sleepycat has fixed several things in that area.

As for thread safety there are several things I'm concerned about. In  
particular it seems libxslt is not thread safe either. However, from  
what I can tell the interpreter lock in Python mitigates that to some  
degree.

The reality is that this system doesn't really need transactions, it  
just needs concurrent access. When I get around to it I'll try to get  
the concurrent datastore to work with Python, I tried before but had  
problems.

On Sunday, October 5, 2003, at 12:53  PM, Rick Bradley wrote:

> I started seeing the message in the Subject: this afternoon after
> everything had been running fine all day.  Looking at the output from
> the AppServer is more telling:
>
>         Traceback (most recent call last):
> 	  File "WebKit/Application.py", line 415, in dispatchRequest
> 	  File "WebKit/Application.py", line 563, in handleGoodURL
> 	  File "WebKit/Application.py", line 1056, in  
> createServletInTransaction
> 	  File "WebKit/Application.py", line 990, in getServlet
> 	  File "WebKit/ServletFactory.py", line 206, in servletForTransaction
> 	  File "/root/syncato/install/syncato/dist/servlets/Main.py", line  
> 14, in __init__
> 	    self.weblog = Weblog("../config")
> 	  File "../dist/scripts/lib/Weblog.py", line 69, in Weblog
> 	    _Weblog.theInstance = _Weblog(configPath)
> 	  File "../dist/scripts/lib/Weblog.py", line 80, in __init__
> 	    self.db = Database(self.config)
> 	  File "../dist/scripts/lib/Database.py", line 72, in Database
> 	    _Database.theInstance = _Database(configPath)
> 	  File "../dist/scripts/lib/Database.py", line 95, in __init__
> 	    self.db.addIndex(txn, "", "seconds",  
> "node-attribute-equality-number")
> 	  File "/root/syncato/install//lib/python/dbxml.py", line 75, in  
> addIndex
> 	    def addIndex(*args): return  
> apply(_dbxml.XmlContainer_addIndex,args)
> 	RuntimeError: Error: Dbc::get: Cannot allocate memory
> 	Closing databases
> 	Exception bsddb._db.DBRunRecoveryError: (-30982, 'DB_RUNRECOVERY:  
> Fatal error, run database recovery -- fatal region error detected; run  
> recovery') in <bound method _Database.__del__ of <Database._Database  
> instance at 0x40522d0c>> ignored
> 	Exception exceptions.AttributeError: "_Weblog instance has no  
> attribute 'db'" in <bound method _Weblog.__del__ of <Weblog._Weblog  
> instance at 0x40522b6c>> ignored
>
>
> Having used subversion for some time, which is built on top of
> BerkeleyDB the "DB_RUNRECOVERY: Fatal error, run database recovery"
> message is familiar (FWIW, subversion is stable enough now that we  
> never
> see this anymore).
>
> So I ran the appropriate db_recover utility on the blog-data databases,
> like so:
>
>  ../../db/bin/db_recover -ve -h blog-data/
>
> and saw:
>
> db_recover: Finding last valid log LSN: file: 2 offset 5651150
> db_recover: Recovery starting from [2][5648282]
> db_recover: Recovery complete at Sun Oct  5 14:30:07 2003
> db_recover: Maximum transaction ID 80000001 Recovery checkpoint  
> [2][5651237]
> db_recover: Recovery complete at Sun Oct  5 14:30:07 2003
> db_recover: Maximum transaction id 80000000 Recovery checkpoint  
> [2][5651237]
>
> Which looks fine, until I loaded the webpage again and got the same
> error messages (btw, I also always make sure the database directory has
> open enough permissions after running db_recover).
>
> I shut down apache and webkit, re-ran the db_recover utility, restarted
> the apache and AppServer process, to no avail.
>
>
> So then I did a "catastrophic" db_recover, like so:
>
> ../../db/bin/db_recover -vce -h blog-data/
>
> and saw:
>
> db_recover: Finding last valid log LSN: file: 2 offset 5651150
> db_recover: Recovery starting from [1][28]
> db_recover: Improper file close at 2/5648195
> db_recover: Recovery function for LSN 2 5648195 failed on backward pass
> db_recover: PANIC: Invalid argument
> db_recover: fatal region error detected; run recovery
> db_recover: fatal region error detected; run recovery
> db_recover: fatal region error detected; run recovery
> db_recover: fatal region error detected; run recovery
> db_recover: fatal region error detected; run recovery
> db_recover: fatal region error detected; run recovery
> db_recover: fatal region error detected; run recovery
> db_recover: fatal region error detected; run recovery
> db_recover: fatal region error detected; run recovery
> db_recover: fatal region error detected; run recovery
> db_recover: DB_ENV->open: DB_RUNRECOVERY: Fatal error, run database  
> recovery
> db_recover: Database handles open during environment close
> db_recover: dbenv->close: Invalid argument
>
> Cripes.  Anyway, after some research I turned up this:
>
>      
> https://sourceforge.net/tracker/ 
> ?func=detail&atid=105470&aid=775414&group_id=5470
>
> Smells to me like bsddb3 isn't properly thread-safe.  This sort of  
> error
> turns up in a number of places in the python-bugs mailing lists.
>
> Any hints for digging out of this hole (now and/or in the future) are
> appreciated.
>
> Rick
> -- 
>  http://www.rickbradley.com    MUPRN: 993
>                        |  with Windows, much less
>    random email haiku  |  for and for Free...). Well, try doing
>                        |  Appletalk on NT.
>
>
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> Syncato-general mailing list
> Syncato-general-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org
> https://lists.sourceforge.net/lists/listinfo/syncato-general
>
>
Kimbro Staken
Software, Consulting and Writing http://www.xmldatabases.org/
Apache Xindice native XML database http://xml.apache.org/xindice
XML:DB Initiative http://www.xmldb.org



-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf