Re: "Dbc::get: Cannot allocate memory"

Something else I just thought of, I believe in the 0.5 code the  
database does not get properly shutdown when webware shuts down. That  
will definitely contribute to this problem and is one of the things  
fixed in the newer code.

If you want to track the updated code I can probably start using the  
sourceforge CVS server instead if my own.

On Sunday, October 5, 2003, at 07:57  PM, Kimbro Staken wrote:

> Yeah, I was having similar problems. There are a number of problems,  
> but the latest code seems to be running more stable. In particular  
> make sure you have the latest pathan installed. My site was going down  
> a couple times a day, now it's been fine for about a week. The one  
> thing that's still a problem is memory leaks, though that's much  
> better now too. Sleepycat has fixed several things in that area.
>
> As for thread safety there are several things I'm concerned about. In  
> particular it seems libxslt is not thread safe either. However, from  
> what I can tell the interpreter lock in Python mitigates that to some  
> degree.
>
> The reality is that this system doesn't really need transactions, it  
> just needs concurrent access. When I get around to it I'll try to get  
> the concurrent datastore to work with Python, I tried before but had  
> problems.
>
> On Sunday, October 5, 2003, at 12:53  PM, Rick Bradley wrote:
>
>> I started seeing the message in the Subject: this afternoon after
>> everything had been running fine all day.  Looking at the output from
>> the AppServer is more telling:
>>
>>         Traceback (most recent call last):
>> 	  File "WebKit/Application.py", line 415, in dispatchRequest
>> 	  File "WebKit/Application.py", line 563, in handleGoodURL
>> 	  File "WebKit/Application.py", line 1056, in  
>> createServletInTransaction
>> 	  File "WebKit/Application.py", line 990, in getServlet
>> 	  File "WebKit/ServletFactory.py", line 206, in servletForTransaction
>> 	  File "/root/syncato/install/syncato/dist/servlets/Main.py", line  
>> 14, in __init__
>> 	    self.weblog = Weblog("../config")
>> 	  File "../dist/scripts/lib/Weblog.py", line 69, in Weblog
>> 	    _Weblog.theInstance = _Weblog(configPath)
>> 	  File "../dist/scripts/lib/Weblog.py", line 80, in __init__
>> 	    self.db = Database(self.config)
>> 	  File "../dist/scripts/lib/Database.py", line 72, in Database
>> 	    _Database.theInstance = _Database(configPath)
>> 	  File "../dist/scripts/lib/Database.py", line 95, in __init__
>> 	    self.db.addIndex(txn, "", "seconds",  
>> "node-attribute-equality-number")
>> 	  File "/root/syncato/install//lib/python/dbxml.py", line 75, in  
>> addIndex
>> 	    def addIndex(*args): return  
>> apply(_dbxml.XmlContainer_addIndex,args)
>> 	RuntimeError: Error: Dbc::get: Cannot allocate memory
>> 	Closing databases
>> 	Exception bsddb._db.DBRunRecoveryError: (-30982, 'DB_RUNRECOVERY:  
>> Fatal error, run database recovery -- fatal region error detected;  
>> run recovery') in <bound method _Database.__del__ of  
>> <Database._Database instance at 0x40522d0c>> ignored
>> 	Exception exceptions.AttributeError: "_Weblog instance has no  
>> attribute 'db'" in <bound method _Weblog.__del__ of <Weblog._Weblog  
>> instance at 0x40522b6c>> ignored
>>
>>
>> Having used subversion for some time, which is built on top of
>> BerkeleyDB the "DB_RUNRECOVERY: Fatal error, run database recovery"
>> message is familiar (FWIW, subversion is stable enough now that we  
>> never
>> see this anymore).
>>
>> So I ran the appropriate db_recover utility on the blog-data  
>> databases,
>> like so:
>>
>>  ../../db/bin/db_recover -ve -h blog-data/
>>
>> and saw:
>>
>> db_recover: Finding last valid log LSN: file: 2 offset 5651150
>> db_recover: Recovery starting from [2][5648282]
>> db_recover: Recovery complete at Sun Oct  5 14:30:07 2003
>> db_recover: Maximum transaction ID 80000001 Recovery checkpoint  
>> [2][5651237]
>> db_recover: Recovery complete at Sun Oct  5 14:30:07 2003
>> db_recover: Maximum transaction id 80000000 Recovery checkpoint  
>> [2][5651237]
>>
>> Which looks fine, until I loaded the webpage again and got the same
>> error messages (btw, I also always make sure the database directory  
>> has
>> open enough permissions after running db_recover).
>>
>> I shut down apache and webkit, re-ran the db_recover utility,  
>> restarted
>> the apache and AppServer process, to no avail.
>>
>>
>> So then I did a "catastrophic" db_recover, like so:
>>
>> ../../db/bin/db_recover -vce -h blog-data/
>>
>> and saw:
>>
>> db_recover: Finding last valid log LSN: file: 2 offset 5651150
>> db_recover: Recovery starting from [1][28]
>> db_recover: Improper file close at 2/5648195
>> db_recover: Recovery function for LSN 2 5648195 failed on backward  
>> pass
>> db_recover: PANIC: Invalid argument
>> db_recover: fatal region error detected; run recovery
>> db_recover: fatal region error detected; run recovery
>> db_recover: fatal region error detected; run recovery
>> db_recover: fatal region error detected; run recovery
>> db_recover: fatal region error detected; run recovery
>> db_recover: fatal region error detected; run recovery
>> db_recover: fatal region error detected; run recovery
>> db_recover: fatal region error detected; run recovery
>> db_recover: fatal region error detected; run recovery
>> db_recover: fatal region error detected; run recovery
>> db_recover: DB_ENV->open: DB_RUNRECOVERY: Fatal error, run database  
>> recovery
>> db_recover: Database handles open during environment close
>> db_recover: dbenv->close: Invalid argument
>>
>> Cripes.  Anyway, after some research I turned up this:
>>
>>      
>> https://sourceforge.net/tracker/ 
>> ?func=detail&atid=105470&aid=775414&group_id=5470
>>
>> Smells to me like bsddb3 isn't properly thread-safe.  This sort of  
>> error
>> turns up in a number of places in the python-bugs mailing lists.
>>
>> Any hints for digging out of this hole (now and/or in the future) are
>> appreciated.
>>
>> Rick
>> -- 
>>  http://www.rickbradley.com    MUPRN: 993
>>                        |  with Windows, much less
>>    random email haiku  |  for and for Free...). Well, try doing
>>                        |  Appletalk on NT.
>>
>>
>> -------------------------------------------------------
>> This sf.net email is sponsored by:ThinkGeek
>> Welcome to geek heaven.
>> http://thinkgeek.com/sf
>> _______________________________________________
>> Syncato-general mailing list
>> Syncato-general-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org
>> https://lists.sourceforge.net/lists/listinfo/syncato-general
>>
>>
> Kimbro Staken
> Software, Consulting and Writing http://www.xmldatabases.org/
> Apache Xindice native XML database http://xml.apache.org/xindice
> XML:DB Initiative http://www.xmldb.org
>
>
>
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> Syncato-general mailing list
> Syncato-general-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org
> https://lists.sourceforge.net/lists/listinfo/syncato-general
>
>
Kimbro Staken
Software, Consulting and Writing http://www.xmldatabases.org/
Apache Xindice native XML database http://xml.apache.org/xindice
XML:DB Initiative http://www.xmldb.org



-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf