MiddleKit Threading Error?

I last reported this problem back on May 22, 2004.  I am not sure what I have been doing different lately, but I have encountered the problem on both my test system and my production system this past month.  I have never been able to reproduce this problem at will.  The problem last went away with some 0.8x or 0.9x release but has now occurred on 1.0RC1.

I think the problem is caused by:
  1. a freshly started AppServer
  2. a flurry of incoming transactions all trying to access a MySQL
table that has not been accessed before.

I think I am hitting the problem because my application generates web pages with multiple embedded images that are served by the Webware application.  So accessing a web page may create a flurry of transactions, the first has to wait for a disk read, and the second transactions overruns it resulting in duplicate entries in the cache.

The abend occurs in MiddleKit/Run/MiddleObject in readStoreData at line 92:
    assert len(cache) + 1 == len(row)

Per the fancy traceback, cache is equal to:
[<unbound method Folder.setFileName>, 
<unbound method Folder.setSurveyDate>, 
<unbound method Folder.setLockUserId>, 
<unbound method Folder.setPhotographer>, 
<unbound method Folder.setComments>, 
<unbound method Folder.setRootId>, 
<unbound method Folder.setSiteId>, 
<unbound method Folder.setNumberFiles>, 
<unbound method Folder.setUpdatedBy>, 
<unbound method Folder.setUpdateTime>, 
   -- the problem is here, the fields repeat
<unbound method Folder.setFileName>, 
<unbound method Folder.setSurveyDate>, 
<unbound method Folder.setLockUserId>, 
<unbound method Folder.setPhotographer>, 
<unbound method Folder.setComments>, 
<unbound method Folder.setRootId>, 
<unbound method Folder.setSiteId>, 
<unbound method Folder.setNumberFiles>, 
<unbound method Folder.setUpdatedBy>, 
<unbound method Folder.setUpdateTime>]

And row is equal to:
(96L, 
'MonumentNashRuin_AR-03-09-02-08', 
datetime.date(2007, 2, 6),
 2L, 
'RH, GH', 
'', 
8L, 
94L, 
42L, 
2L, 
datetime.datetime(2007, 9, 18, 12, 44, 3))

I think the solution is to insert a lock around the code that deals
with loading the cache the first time a table is accessed. This is my diff from MiddleKit/Run/MiddleObject.py (probably hard to read because of the line wraps).

@@ -5,6 +5,9 @@
 from MiddleKit.Core.ObjRefAttr import ObjRefAttr
 from MiddleKit.Core.ListAttr import ListAttr
 
+import thread 
+_cacheLock = thread.allocate_lock() 
+
 try: # for Python < 2.2
 	object
 except NameError:
@@ -78,6 +81,9 @@
 		if store.setting('UseBigIntObjRefColumns', False):
 			fullClassName = self.__class__.__module__ + '.' + self.__class__.__name__
 			cache = self._mk_setCache.setdefault(fullClassName, [])
+
+		if not cache: 
+			_cacheLock.acquire() 
 			if not cache:
 				allAttrs = self.klass().allDataAttrs()
 				# @@ 2000-10-29 ce: next line is major hack: hasSQLColumn()
@@ -88,6 +94,7 @@
 					setMethodName = 'set' + name[0].upper() + name[1:]
 					setMethod = getattr(self.__class__, setMethodName, '_'+name)
 					cache.append(setMethod)
+			_cacheLock.release() 
 
 			assert len(cache) + 1 == len(row)
 			self._mk_initing = 1


Roger Haase


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/