Re: Quixote-users Digest, Vol 89, Issue 5

Hi!
Well, if this is the expected behavior, should not this mean that this
retry wrapper be in the next version of Durus? Convenience will save a
lot of stress.

-Ernie

On Fri, Aug 27, 2010 at 3:56 AM,
<[email protected]> wrote:
> Send Quixote-users mailing list submissions to
> =A0 =A0 =A0 [email protected]
>
> To subscribe or unsubscribe via the World Wide Web, visit
> =A0 =A0 =A0 =A0http://mail.mems-exchange.org/mailman/listinfo/quixote-use=
rs
> or, via email, send a message with subject or body 'help' to
> =A0 =A0 =A0 [email protected]
>
> You can reach the person managing the list at
> =A0 =A0 =A0 [email protected]
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Quixote-users digest..."
>
>
> Today's Topics:
>
> =A0 1. Re: Persistent Session Management with Durus (Gabor Vitez)
> =A0 2. Re: Persistent Session Management with Durus (Salman Haq)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Thu, 26 Aug 2010 20:01:39 +0200
> From: Gabor Vitez <[email protected]>
> Subject: Re: [Quixote-users] Persistent Session Management with Durus
> To: Salman Haq <[email protected]>
> Cc: [email protected]
> Message-ID:
> =A0 =A0 =A0 =A0<[email protected]=
ail.com>
> Content-Type: text/plain; charset=3D"iso-8859-1"
>
> Hi!
>
> As far as I can tell, this is normal behaviour with Durus; I'd suggest
> wrapping every function that reads or writes durus with something like th=
is:
>
> def writetransaction(func):
> =A0"""wrap writers with this"""
> =A0def wrapper(*args, **kw):
> =A0 =A0conn=3Ddbconn.get_db()
> =A0 =A0#bail after 1000 failures
> =A0 =A0for dummy1 in xrange(1000):
> =A0 =A0 =A0try:
> =A0 =A0 =A0 =A0results=3Dfunc(*args,**kw)
> =A0 =A0 =A0 =A0conn.commit()
> =A0 =A0 =A0 =A0return results
> =A0 =A0 =A0except ConflictError:
> =A0 =A0 =A0 =A0pass
> =A0 =A0 =A0conn.abort()
> =A0 =A0raise ConflictError
> =A0return wrapper
>
> def readtransaction(func):
> =A0"""wrap readers with this"""
> =A0def wrapper(*args, **kw):
> =A0 =A0conn=3Ddbconn.get_db()
> =A0 =A0for dummy1 in xrange(1000):
> =A0 =A0 =A0try:
> =A0 =A0 =A0 =A0return func(*args,**kw)
> =A0 =A0 =A0except ConflictError:
> =A0 =A0 =A0 =A0pass
> =A0 =A0 =A0conn.abort()
> =A0 =A0raise ConflictError
> =A0return wrapper
>
>
> dbconn is the global database connection object
>
> Best,
> G?bor
>
> On Thu, Aug 26, 2010 at 5:23 PM, Salman Haq <[email protected]> wro=
te:
>
>> =A0Some new info included at the end:
>>
>>
>>
>> On 08/26/2010 10:49 AM, Salman Haq wrote:
>>
>> Hi,
>>
>> I'm developing an application using Quixote 2.6 and following the exampl=
e in altdemo.py pretty closely for persistent session management with Durus=
..
>> The application is serving static files very unreliably. One in five req=
uests for a static file receive a HTTP 500 response code. Eg: if an html pa=
ge embeds 50
>> script or img tags, requests for about 10 of them will fail. For each of=
 these failed requests, the following traceback is received:
>>
>> Traceback (most recent call last): =A0File "/usr/lib/python2.4/site-pack=
ages/quixote2/publish.py", line 275, in process_request =A0 =A0output =3D s=
elf.try_publish(request) =A0File "/usr/lib/python2.4/site-packages/quixote2=
/publish.py", line 255, in try_publish =A0 =A0self.finish_successful_reques=
t() =A0File "/usr/lib/python2.4/site-packages/quixote2/publish.py", line 13=
8, in finish_successful_request =A0 =A0self.session_manager.finish_successf=
ul_request() =A0File "/usr/lib/python2.4/site-packages/quixote2/session.py"=
, line 388, in finish_successful_request =A0 =A0self.commit_changes(session=
) =A0File "/usr/lib/python2..4/site-packages/ACE_rms/auth/session.py", line=
 44, in commit_changes =A0 =A0self.durus_connection.commit() =A0File "/usr/=
lib/python2.4/site-packages/durus/connection.py", line 254, in commit =A0 =
=A0assert not self.invalid_oids, "still conflicted: missing abort()"Asserti=
onError: still conflicted: missing abort()
>>
>> My application is running as a SCGI process behind Apache. Static files =
are served by Quixote via the StaticDirectory class and not by Apache.
>> The Durus server runs as a stand-alone process on localhost:2951 and a c=
onnection object to it is created as follows:
>>
>> def publisher_factory ():
>> =A0 =A0 """ returns a quixote2 Publisher instance that publishes
>> =A0 =A0 the RootDirectory with persistent session management support.
>> =A0 =A0 """
>> =A0 =A0 conn =3D Connection(ClientStorage("127.0.0.1", 2951))
>> =A0 =A0 session_manager =3D DurusSessionManager(conn)
>> =A0 =A0 return Publisher(RootDirectory(), session_manager=3Dsession_mana=
ger, display_exceptions=3D'plain')
>>
>>
>> The application sub-classes SessionManager and Session exactly as descri=
bed in the snippet below:
>>
>> ----
>>
>> from quixote2.session import Session, SessionManager
>> from durus.persistent import Persistent
>> from durus.persistent_dict import PersistentDict
>>
>> class PersistentSession (Session, Persistent):
>> =A0 =A0 """ Persistent version of default quixote Session class.
>> =A0 =A0 This class can be overridden if desired.
>> =A0 =A0 """
>> =A0 =A0 pass
>>
>> class DurusSessionManager (SessionManager):
>> =A0 =A0 """ Loads/stores quixote sessions from/to a Durus db.
>> =A0 =A0 Overrides forget_changes() and commit_changes() as described in =
quixote
>> =A0 =A0 docs and altdemo.py.
>> =A0 =A0 """
>>
>> =A0 =A0 def __init__ (self, durus_connection, session_class=3DPersistent=
Session):
>> =A0 =A0 =A0 =A0 """ Load sessions from the durus db and init the Session=
Manager.
>> =A0 =A0 =A0 =A0 (inst, class) : inst
>> =A0 =A0 =A0 =A0 durus_connection: a Connection object to a Durus db.
>> =A0 =A0 =A0 =A0 session_class: session class to use (must override Peris=
tentSession).
>> =A0 =A0 =A0 =A0 """
>> =A0 =A0 =A0 =A0 self.durus_connection =3D durus_connection
>> =A0 =A0 =A0 =A0 root =3D self.durus_connection.get_root()
>> =A0 =A0 =A0 =A0 sessions =3D root.get('auth.sessions', None)
>>
>> =A0 =A0 =A0 =A0 if not sessions:
>> =A0 =A0 =A0 =A0 =A0 =A0 sessions =3D PersistentDict()
>> =A0 =A0 =A0 =A0 =A0 =A0 root['auth.sessions'] =3D sessions
>> =A0 =A0 =A0 =A0 =A0 =A0 self.durus_connection.commit()
>>
>> =A0 =A0 =A0 =A0 SessionManager.__init__(self,
>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 session_=
class=3Dsession_class,
>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 session_=
mapping=3Dsessions)
>>
>> =A0 =A0 def forget_changes (self, session):
>> =A0 =A0 =A0 =A0 self.durus_connection.abort()
>>
>> =A0 =A0 def commit_changes (self, session):
>> =A0 =A0 =A0 =A0 self.durus_connection.commit()
>>
>> -----
>>
>> Can anyone tell me why my Durus database thinks it's in a conflicted sta=
te?
>>
>> Thank you,
>> Salman
>>
>>
>> Requests for static files are not the only ones that are likely to fail.
>> AJAX requests also tend to cause HTTP 500. Again, the failures are
>> unpredictable,
>> but usually one in a group of three or four simultaneous AJAX requests by
>> the browser can fail.
>>
>> And the AssertionError mentioned above is not the only error raised by t=
he
>> application. I have noticed a WriteConflictError too, albeit less
>> frequently:
>>
>> Traceback (most recent call last):
>>
>> =A0 =A0 =A0 File "/usr/lib/python2.4/site-packages/quixote2/publish.py",=
 line 275, in process_request
>> =A0 =A0 =A0 =A0 output =3D self.try_publish(request)
>> =A0 =A0 =A0 File "/usr/lib/python2.4/site-packages/quixote2/publish.py",=
 line 255, in try_publish
>> =A0 =A0 =A0 =A0 self.finish_successful_request()
>> =A0 =A0 =A0 File "/usr/lib/python2.4/site-packages/quixote2/publish.py",=
 line 138, in finish_successful_request
>> =A0 =A0 =A0 =A0 self.session_manager.finish_successful_request()
>> =A0 =A0 =A0 File "/usr/lib/python2.4/site-
>> packages/quixote2/session.py", line 388, in finish_successful_request
>> =A0 =A0 =A0 =A0 self.commit_changes(session)
>> =A0 =A0 =A0 File "/usr/lib/python2.4/site-packages/ACE_rms/auth/session.=
py", line 45, in commit_changes
>> =A0 =A0 =A0 =A0 self.durus_connection.commit()
>> =A0 =A0 =A0 File "/usr/lib/python2.4/site-packages/durus/connection.py",=
 line 273, in commit
>> =A0 =A0 =A0 =A0 self.storage.end(self._handle_invalidations)
>> =A0 =A0 =A0 File "/usr/lib/python2.4/site-packages/durus/client_storage.=
py", line 82, in end
>> =A0 =A0 =A0 =A0 handle_invalidations(oid_list)
>> =A0 =A0 =A0 File "/usr/lib/python2.4/site-packages/durus/connection.py",=
 line 303, in _handle_invalidations
>> =A0 =A0 =A0 =A0 raise WriteConflictError(conflicts)
>> =A0 =A0 WriteConflictError: oids=3D[2]
>>
>> I'm still learning Durus, but I suspect quite a few people on this list =
are
>> familiar with its internals.
>>
>> Thanks,
>> Salman
>>
>>
>> _______________________________________________
>> Quixote-users mailing list
>> [email protected]
>> http://mail.mems-exchange.org/mailman/listinfo/quixote-users
>>
>>
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <http://mail.mems-exchange.org/pipermail/quixote-users/attachments/2=
0100826/2df2b002/attachment.html>
>
> ------------------------------
>
> Message: 2
> Date: Thu, 26 Aug 2010 14:42:46 -0400
> From: Salman Haq <[email protected]>
> Subject: Re: [Quixote-users] Persistent Session Management with Durus
> To: [email protected]
> Message-ID: <[email protected]>
> Content-Type: text/plain; charset=3D"iso-8859-1"; Format=3D"flowed"
>
> Thanks Gabor! That works.
>
> Neither the Durus README file nor the online FAQs mention anything about
> re-try logic.
>
> Salman
>
> On 08/26/2010 02:01 PM, Gabor Vitez wrote:
>> Hi!
>>
>> As far as I can tell, this is normal behaviour with Durus; I'd suggest
>> wrapping every function that reads or writes durus with something like
>> this:
>>
>> def writetransaction(func):
>> =A0 """wrap writers with this"""
>> =A0 def wrapper(*args, **kw):
>> =A0 =A0 conn=3Ddbconn.get_db()
>> =A0 =A0 #bail after 1000 failures
>> =A0 =A0 for dummy1 in xrange(1000):
>> =A0 =A0 =A0 try:
>> =A0 =A0 =A0 =A0 results=3Dfunc(*args,**kw)
>> =A0 =A0 =A0 =A0 conn.commit()
>> =A0 =A0 =A0 =A0 return results
>> =A0 =A0 =A0 except ConflictError:
>> =A0 =A0 =A0 =A0 pass
>> =A0 =A0 =A0 conn.abort()
>> =A0 =A0 raise ConflictError
>> =A0 return wrapper
>>
>> def readtransaction(func):
>> =A0 """wrap readers with this"""
>> =A0 def wrapper(*args, **kw):
>> =A0 =A0 conn=3Ddbconn.get_db()
>> =A0 =A0 for dummy1 in xrange(1000):
>> =A0 =A0 =A0 try:
>> =A0 =A0 =A0 =A0 return func(*args,**kw)
>> =A0 =A0 =A0 except ConflictError:
>> =A0 =A0 =A0 =A0 pass
>> =A0 =A0 =A0 conn.abort()
>> =A0 =A0 raise ConflictError
>> =A0 return wrapper
>>
>>
>> dbconn is the global database connection object
>>
>> Best,
>> G?bor
>>
>> On Thu, Aug 26, 2010 at 5:23 PM, Salman Haq <[email protected]
>> <mailto:[email protected]>> wrote:
>>
>> =A0 =A0 Some new info included at the end:
>>
>>
>>
>> =A0 =A0 On 08/26/2010 10:49 AM, Salman Haq wrote:
>>> =A0 =A0 |Hi,
>>>
>>> =A0 =A0 I'm developing an application using Quixote 2.6 and following t=
he example in altdemo.py pretty closely for persistent session management w=
ith Durus.
>>> =A0 =A0 The application is serving static files very unreliably. One in=
 five requests for a static file receive a HTTP 500 response code. Eg: if a=
n html page embeds 50
>>> =A0 =A0 script or img tags, requests for about 10 of them will fail. Fo=
r each of these failed requests, the following traceback is received:
>>>
>>> =A0 =A0 Traceback (most recent call last):
>>> =A0 =A0 || =A0 File "/usr/lib/python2.4/site-packages/quixote2/publish.=
py", line 275, in process_request
>>> =A0 =A0 || =A0 =A0 output =3D self.try_publish(request)
>>> =A0 =A0 || =A0 File "/usr/lib/python2.4/site-packages/quixote2/publish.=
py", line 255, in try_publish
>>> =A0 =A0 || =A0 =A0 self.finish_successful_request()
>>> =A0 =A0 || =A0 File "/usr/lib/python2.4/site-packages/quixote2/publish.=
py", line 138, in finish_successful_request||
>>> =A0 =A0 || =A0 =A0 self.session_manager.finish_successful_request()
>>> =A0 =A0 || =A0 File "/usr/lib/python2.4/site-packages/quixote2/session.=
py", line 388, in finish_successful_request||
>>> =A0 =A0 || =A0 =A0 self.commit_changes(session)
>>> =A0 =A0 || =A0 File "/usr/lib/python2.4/site-packages/ACE_rms/auth/sess=
ion.py", line 44, in commit_changes
>>> =A0 =A0 || =A0 =A0 self.durus_connection.commit()
>>> =A0 =A0 || =A0 File "/usr/lib/python2.4/site-packages/durus/connection.=
py", line 254, in commit
>>> =A0 =A0 || =A0 =A0 assert not self.invalid_oids, "still conflicted: mis=
sing abort()"
>>> =A0 =A0 ||AssertionError: still conflicted: missing abort()
>>>
>>> =A0 =A0 My application is running as a SCGI process behind Apache. Stat=
ic files are served by Quixote via the StaticDirectory class and not by Apa=
che.
>>> =A0 =A0 The Durus server runs as a stand-alone process on localhost:295=
1 and a connection object to it is created as follows:
>>>
>>> =A0 =A0 def publisher_factory ():
>>> =A0 =A0 =A0 =A0 =A0""" returns a quixote2 Publisher instance that publi=
shes
>>> =A0 =A0 =A0 =A0 =A0the RootDirectory with persistent session management=
 support.
>>> =A0 =A0 =A0 =A0 =A0"""
>>> =A0 =A0 =A0 =A0 =A0conn =3D Connection(ClientStorage("127.0.0.1", 2951))
>>> =A0 =A0 =A0 =A0 =A0session_manager =3D DurusSessionManager(conn)
>>> =A0 =A0 =A0 =A0 =A0return Publisher(RootDirectory(), session_manager=3D=
session_manager, display_exceptions=3D'plain')
>>>
>>>
>>> =A0 =A0 The application sub-classes SessionManager and Session exactly =
as described in the snippet below:
>>>
>>> =A0 =A0 ----
>>>
>>> =A0 =A0 from quixote2.session import Session, SessionManager
>>> =A0 =A0 from durus.persistent import Persistent
>>> =A0 =A0 from durus.persistent_dict import PersistentDict
>>>
>>> =A0 =A0 class PersistentSession (Session, Persistent):
>>> =A0 =A0 =A0 =A0 =A0""" Persistent version of default quixote Session cl=
ass.
>>> =A0 =A0 =A0 =A0 =A0This class can be overridden if desired.
>>> =A0 =A0 =A0 =A0 =A0"""
>>> =A0 =A0 =A0 =A0 =A0pass
>>>
>>> =A0 =A0 class DurusSessionManager (SessionManager):
>>> =A0 =A0 =A0 =A0 =A0""" Loads/stores quixote sessions from/to a Durus db=
..
>>> =A0 =A0 =A0 =A0 =A0Overrides forget_changes() and commit_changes() as d=
escribed in quixote
>>> =A0 =A0 =A0 =A0 =A0docs and altdemo.py.
>>> =A0 =A0 =A0 =A0 =A0"""
>>>
>>> =A0 =A0 =A0 =A0 =A0def __init__ (self, durus_connection, session_class=
=3DPersistentSession):
>>> =A0 =A0 =A0 =A0 =A0 =A0 =A0""" Load sessions from the durus db and init=
 the SessionManager.
>>> =A0 =A0 =A0 =A0 =A0 =A0 =A0(inst, class) : inst
>>> =A0 =A0 =A0 =A0 =A0 =A0 =A0durus_connection: a Connection object to a D=
urus db.
>>> =A0 =A0 =A0 =A0 =A0 =A0 =A0session_class: session class to use (must ov=
erride PeristentSession).
>>> =A0 =A0 =A0 =A0 =A0 =A0 =A0"""
>>> =A0 =A0 =A0 =A0 =A0 =A0 =A0self.durus_connection =3D durus_connection
>>> =A0 =A0 =A0 =A0 =A0 =A0 =A0root =3D self.durus_connection.get_root()
>>> =A0 =A0 =A0 =A0 =A0 =A0 =A0sessions =3D root.get('auth.sessions', None)
>>>
>>> =A0 =A0 =A0 =A0 =A0 =A0 =A0if not sessions:
>>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0sessions =3D PersistentDict()
>>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0root['auth.sessions'] =3D sessions
>>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0self.durus_connection.commit()
>>>
>>> =A0 =A0 =A0 =A0 =A0 =A0 =A0SessionManager.__init__(self,
>>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=
 =A0session_class=3Dsession_class,
>>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=
 =A0session_mapping=3Dsessions)
>>>
>>> =A0 =A0 =A0 =A0 =A0def forget_changes (self, session):
>>> =A0 =A0 =A0 =A0 =A0 =A0 =A0self.durus_connection.abort()
>>>
>>> =A0 =A0 =A0 =A0 =A0def commit_changes (self, session):
>>> =A0 =A0 =A0 =A0 =A0 =A0 =A0self.durus_connection.commit()
>>>
>>> =A0 =A0 -----
>>>
>>> =A0 =A0 Can anyone tell me why my Durus database thinks it's in a confl=
icted state?
>>>
>>> =A0 =A0 Thank you,
>>> =A0 =A0 Salman
>>> =A0 =A0 |||
>>
>> =A0 =A0 Requests for static files are not the only ones that are likely =
to
>> =A0 =A0 fail. AJAX requests also tend to cause HTTP 500. Again, the
>> =A0 =A0 failures are unpredictable,
>> =A0 =A0 but usually one in a group of three or four simultaneous AJAX
>> =A0 =A0 requests by the browser can fail.
>>
>> =A0 =A0 And the AssertionError mentioned above is not the only error
>> =A0 =A0 raised by the application. I have noticed a WriteConflictError
>> =A0 =A0 too, albeit less frequently:
>>
>> =A0 =A0 Traceback (most recent call last):
>>
>> =A0 =A0 =A0 =A0 =A0 File "/usr/lib/python2.4/site-packages/quixote2/publ=
ish.py",
>> =A0 =A0 line 275, in process_request =A0 =A0 =A0 =A0 output =3D
>> =A0 =A0 self.try_publish(request) =A0 =A0 =A0 File
>> =A0 =A0 "/usr/lib/python2.4/site-packages/quixote2/publish.py", line 255,
>> =A0 =A0 in try_publish =A0 =A0 =A0 =A0 self.finish_successful_request() =
=A0 =A0 =A0 File
>> =A0 =A0 "/usr/lib/python2.4/site-packages/quixote2/publish.py", line 138,
>> =A0 =A0 in finish_successful_request
>> =A0 =A0 self.session_manager.finish_successful_request() =A0 =A0 =A0 File
>> =A0 =A0 "/usr/lib/python2.4/site-
>> =A0 =A0 packages/quixote2/session.py", line 388, in finish_successful_re=
quest
>> =A0 =A0 =A0 =A0 =A0 =A0 =A0self.commit_changes(session)
>> =A0 =A0 =A0 =A0 =A0 =A0File "/usr/lib/python2.4/site-packages/ACE_rms/au=
th/session.py", line 45, in commit_changes
>> =A0 =A0 =A0 =A0 =A0 =A0 =A0self.durus_connection.commit()
>> =A0 =A0 =A0 =A0 =A0 =A0File "/usr/lib/python2.4/site-packages/durus/conn=
ection.py", line 273, in commit
>> =A0 =A0 =A0 =A0 =A0 =A0 =A0self.storage.end(self._handle_invalidations)
>> =A0 =A0 =A0 =A0 =A0 =A0File "/usr/lib/python2.4/site-packages/durus/clie=
nt_storage.py", line 82, in end
>> =A0 =A0 =A0 =A0 =A0 =A0 =A0handle_invalidations(oid_list)
>> =A0 =A0 =A0 =A0 =A0 =A0File "/usr/lib/python2.4/site-packages/durus/conn=
ection.py", line 303, in _handle_invalidations
>> =A0 =A0 =A0 =A0 =A0 =A0 =A0raise WriteConflictError(conflicts)
>> =A0 =A0 =A0 =A0 =A0WriteConflictError: oids=3D[2]
>>
>> =A0 =A0 I'm still learning Durus, but I suspect quite a few people on th=
is
>> =A0 =A0 list are familiar with its internals.
>>
>> =A0 =A0 Thanks,
>> =A0 =A0 Salman
>>
>>
>> =A0 =A0 _______________________________________________
>> =A0 =A0 Quixote-users mailing list
>> =A0 =A0 [email protected]
>> =A0 =A0 <mailto:[email protected]>
>> =A0 =A0 http://mail.mems-exchange.org/mailman/listinfo/quixote-users
>>
>>
>
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <http://mail.mems-exchange.org/pipermail/quixote-users/attachments/2=
0100826/bc7c56d9/attachment.htm>
>
> ------------------------------
>
> _______________________________________________
> Quixote-users mailing list
> [email protected]
> http://mail.mems-exchange.org/mailman/listinfo/quixote-users
>
>
> End of Quixote-users Digest, Vol 89, Issue 5
> ********************************************
>