Re: Persistent Session Management with Durus
Salman Haq <[email protected]> Thu, 26 Aug 2010 14:42:46 -0400
| Newsgroups | gmane.comp.web.quixote.user |
|---|---|
| Message-ID | <[email protected]> |
This is a multi-part message in MIME format. --===============0914980726== Content-Type: multipart/alternative; boundary="------------080200000409050804050108" This is a multi-part message in MIME format. --------------080200000409050804050108 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit Thanks Gabor! That works. Neither the Durus README file nor the online FAQs mention anything about re-try logic. Salman On 08/26/2010 02:01 PM, Gabor Vitez wrote: > Hi! > > As far as I can tell, this is normal behaviour with Durus; I'd suggest > wrapping every function that reads or writes durus with something like > this: > > def writetransaction(func): > """wrap writers with this""" > def wrapper(*args, **kw): > conn=dbconn.get_db() > #bail after 1000 failures > for dummy1 in xrange(1000): > try: > results=func(*args,**kw) > conn.commit() > return results > except ConflictError: > pass > conn.abort() > raise ConflictError > return wrapper > > def readtransaction(func): > """wrap readers with this""" > def wrapper(*args, **kw): > conn=dbconn.get_db() > for dummy1 in xrange(1000): > try: > return func(*args,**kw) > except ConflictError: > pass > conn.abort() > raise ConflictError > return wrapper > > > dbconn is the global database connection object > > Best, > Gábor > > On Thu, Aug 26, 2010 at 5:23 PM, Salman Haq <[email protected] > <mailto:[email protected]>> wrote: > > Some new info included at the end: > > > > On 08/26/2010 10:49 AM, Salman Haq wrote: >> |Hi, >> >> I'm developing an application using Quixote 2.6 and following the example in altdemo.py pretty closely for persistent session management with Durus. >> The application is serving static files very unreliably. One in five requests for a static file receive a HTTP 500 response code. Eg: if an html page embeds 50 >> script or img tags, requests for about 10 of them will fail. For each of these failed requests, the following traceback is received: >> >> Traceback (most recent call last): >> || File "/usr/lib/python2.4/site-packages/quixote2/publish.py", line 275, in process_request >> || output = self.try_publish(request) >> || File "/usr/lib/python2.4/site-packages/quixote2/publish.py", line 255, in try_publish >> || self.finish_successful_request() >> || File "/usr/lib/python2.4/site-packages/quixote2/publish.py", line 138, in finish_successful_request|| >> || self.session_manager.finish_successful_request() >> || File "/usr/lib/python2.4/site-packages/quixote2/session.py", line 388, in finish_successful_request|| >> || self.commit_changes(session) >> || File "/usr/lib/python2.4/site-packages/ACE_rms/auth/session.py", line 44, in commit_changes >> || self.durus_connection.commit() >> || File "/usr/lib/python2.4/site-packages/durus/connection.py", line 254, in commit >> || assert not self.invalid_oids, "still conflicted: missing abort()" >> ||AssertionError: still conflicted: missing abort() >> >> My application is running as a SCGI process behind Apache. Static files are served by Quixote via the StaticDirectory class and not by Apache. >> The Durus server runs as a stand-alone process on localhost:2951 and a connection object to it is created as follows: >> >> def publisher_factory (): >> """ returns a quixote2 Publisher instance that publishes >> the RootDirectory with persistent session management support. >> """ >> conn = Connection(ClientStorage("127.0.0.1", 2951)) >> session_manager = DurusSessionManager(conn) >> return Publisher(RootDirectory(), session_manager=session_manager, display_exceptions='plain') >> >> >> The application sub-classes SessionManager and Session exactly as described in the snippet below: >> >> ---- >> >> from quixote2.session import Session, SessionManager >> from durus.persistent import Persistent >> from durus.persistent_dict import PersistentDict >> >> class PersistentSession (Session, Persistent): >> """ Persistent version of default quixote Session class. >> This class can be overridden if desired. >> """ >> pass >> >> class DurusSessionManager (SessionManager): >> """ Loads/stores quixote sessions from/to a Durus db. >> Overrides forget_changes() and commit_changes() as described in quixote >> docs and altdemo.py. >> """ >> >> def __init__ (self, durus_connection, session_class=PersistentSession): >> """ Load sessions from the durus db and init the SessionManager. >> (inst, class) : inst >> durus_connection: a Connection object to a Durus db. >> session_class: session class to use (must override PeristentSession). >> """ >> self.durus_connection = durus_connection >> root = self.durus_connection.get_root() >> sessions = root.get('auth.sessions', None) >> >> if not sessions: >> sessions = PersistentDict() >> root['auth.sessions'] = sessions >> self.durus_connection.commit() >> >> SessionManager.__init__(self, >> session_class=session_class, >> session_mapping=sessions) >> >> def forget_changes (self, session): >> self.durus_connection.abort() >> >> def commit_changes (self, session): >> self.durus_connection.commit() >> >> ----- >> >> Can anyone tell me why my Durus database thinks it's in a conflicted state? >> >> Thank you, >> Salman >> ||| > > Requests for static files are not the only ones that are likely to > fail. AJAX requests also tend to cause HTTP 500. Again, the > failures are unpredictable, > but usually one in a group of three or four simultaneous AJAX > requests by the browser can fail. > > And the AssertionError mentioned above is not the only error > raised by the application. I have noticed a WriteConflictError > too, albeit less frequently: > > Traceback (most recent call last): > > File "/usr/lib/python2.4/site-packages/quixote2/publish.py", > line 275, in process_request output = > self.try_publish(request) File > "/usr/lib/python2.4/site-packages/quixote2/publish.py", line 255, > in try_publish self.finish_successful_request() File > "/usr/lib/python2.4/site-packages/quixote2/publish.py", line 138, > in finish_successful_request > self.session_manager.finish_successful_request() File > "/usr/lib/python2.4/site- > packages/quixote2/session.py", line 388, in finish_successful_request > self.commit_changes(session) > File "/usr/lib/python2.4/site-packages/ACE_rms/auth/session.py", line 45, in commit_changes > self.durus_connection.commit() > File "/usr/lib/python2.4/site-packages/durus/connection.py", line 273, in commit > self.storage.end(self._handle_invalidations) > File "/usr/lib/python2.4/site-packages/durus/client_storage.py", line 82, in end > handle_invalidations(oid_list) > File "/usr/lib/python2.4/site-packages/durus/connection.py", line 303, in _handle_invalidations > raise WriteConflictError(conflicts) > WriteConflictError: oids=[2] > > I'm still learning Durus, but I suspect quite a few people on this > list are familiar with its internals. > > Thanks, > Salman > > > _______________________________________________ > Quixote-users mailing list > [email protected] > <mailto:[email protected]> > http://mail.mems-exchange.org/mailman/listinfo/quixote-users > > --------------080200000409050804050108 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> <html> <head> <meta content="text/html; charset=ISO-8859-1" http-equiv="Content-Type"> </head> <body text="#000000" bgcolor="#ffffff"> Thanks Gabor! That works. <br> <br> Neither the Durus README file nor the online FAQs mention anything about re-try logic.<br> <br> Salman<br> <br> On 08/26/2010 02:01 PM, Gabor Vitez wrote: <blockquote cite="mid:[email protected]" type="cite">Hi!<br> <br> As far as I can tell, this is normal behaviour with Durus; I'd suggest wrapping every function that reads or writes durus with something like this:<br> <br> <span style="font-family: courier new,monospace;">def writetransaction(func):</span><br style="font-family: courier new,monospace;"> <span style="font-family: courier new,monospace;"></span><span style="font-family: courier new,monospace;"> """wrap writers with this"""<br> def wrapper(*args, **kw):</span><br style="font-family: courier new,monospace;"> <span style="font-family: courier new,monospace;"> conn=dbconn.get_db()<br> #bail after 1000 failures<br style="font-family: courier new,monospace;"> </span><span style="font-family: courier new,monospace;"> for dummy1 in xrange(1000):</span><br style="font-family: courier new,monospace;"> <span style="font-family: courier new,monospace;"> try:</span><br style="font-family: courier new,monospace;"> <span style="font-family: courier new,monospace;"> results=func(*args,**kw)</span><br style="font-family: courier new,monospace;"> <span style="font-family: courier new,monospace;"> conn.commit()</span><br style="font-family: courier new,monospace;"> <span style="font-family: courier new,monospace;"> return results</span><br style="font-family: courier new,monospace;"> <span style="font-family: courier new,monospace;"> except ConflictError:</span><br style="font-family: courier new,monospace;"> <span style="font-family: courier new,monospace;"> pass</span><br style="font-family: courier new,monospace;"> <span style="font-family: courier new,monospace;"> conn.abort()</span><br style="font-family: courier new,monospace;"> <span style="font-family: courier new,monospace;"> raise ConflictError</span><br style="font-family: courier new,monospace;"> <span style="font-family: courier new,monospace;"> return wrapper</span><br style="font-family: courier new,monospace;"> <br style="font-family: courier new,monospace;"> <span style="font-family: courier new,monospace;">def readtransaction(func):<br> """wrap readers with this"""<br style="font-family: courier new,monospace;"> </span><span style="font-family: courier new,monospace;"></span><span style="font-family: courier new,monospace;"> def wrapper(*args, **kw):</span><br style="font-family: courier new,monospace;"> <span style="font-family: courier new,monospace;"> conn=dbconn.get_db()</span><br style="font-family: courier new,monospace;"> <span style="font-family: courier new,monospace;"> for dummy1 in xrange(1000):</span><br style="font-family: courier new,monospace;"> <span style="font-family: courier new,monospace;"> try:</span><br style="font-family: courier new,monospace;"> <span style="font-family: courier new,monospace;"> return func(*args,**kw)</span><br style="font-family: courier new,monospace;"> <span style="font-family: courier new,monospace;"> except ConflictError:</span><br style="font-family: courier new,monospace;"> <span style="font-family: courier new,monospace;"> pass</span><br style="font-family: courier new,monospace;"> <span style="font-family: courier new,monospace;"> conn.abort()</span><br style="font-family: courier new,monospace;"> <span style="font-family: courier new,monospace;"> raise ConflictError</span><br style="font-family: courier new,monospace;"> <span style="font-family: courier new,monospace;"> return wrapper</span><br style="font-family: courier new,monospace;"> <br> <br> dbconn is the global database connection object<br> <br> Best,<br> Gábor<br> <br> <div class="gmail_quote">On Thu, Aug 26, 2010 at 5:23 PM, Salman Haq <span dir="ltr"><<a moz-do-not-send="true" href="mailto:[email protected]" target="_blank">[email protected]</a>></span> wrote:<br> <blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"> <div text="#000000" bgcolor="#ffffff">Some new info included at the end: <div> <div><br> <br> <br> On 08/26/2010 10:49 AM, Salman Haq wrote: <blockquote type="cite"> <pre><code>Hi, I'm developing an application using Quixote 2.6 and following the example in altdemo.py pretty closely for persistent session management with Durus. The application is serving static files very unreliably. One in five requests for a static file receive a HTTP 500 response code. Eg: if an html page embeds 50 script or img tags, requests for about 10 of them will fail. For each of these failed requests, the following traceback is received: Traceback (most recent call last): </code><code> File "/usr/lib/python2.4/site-packages/quixote2/publish.py", line 275, in process_request </code><code> output = self.try_publish(request) </code><code> File "/usr/lib/python2.4/site-packages/quixote2/publish.py", line 255, in try_publish </code><code> self.finish_successful_request() </code><code> File "/usr/lib/python2.4/site-packages/quixote2/publish.py", line 138, in finish_successful_request</code><code> </code><code> self.session_manager.finish_successful_request() </code><code> File "/usr/lib/python2.4/site-packages/quixote2/session.py", line 388, in finish_successful_request</code><code> </code><code> self.commit_changes(session) </code><code> File "/usr/lib/python2.4/site-packages/ACE_rms/auth/session.py", line 44, in commit_changes </code><code> self.durus_connection.commit() </code><code> File "/usr/lib/python2.4/site-packages/durus/connection.py", line 254, in commit </code><code> assert not self.invalid_oids, "still conflicted: missing abort()" </code><code>AssertionError: still conflicted: missing abort() My application is running as a SCGI process behind Apache. Static files are served by Quixote via the StaticDirectory class and not by Apache. The Durus server runs as a stand-alone process on localhost:2951 and a connection object to it is created as follows: def publisher_factory (): """ returns a quixote2 Publisher instance that publishes the RootDirectory with persistent session management support. """ conn = Connection(ClientStorage("127.0.0.1", 2951)) session_manager = DurusSessionManager(conn) return Publisher(RootDirectory(), session_manager=session_manager, display_exceptions='plain') The application sub-classes SessionManager and Session exactly as described in the snippet below: ---- from quixote2.session import Session, SessionManager from durus.persistent import Persistent from durus.persistent_dict import PersistentDict class PersistentSession (Session, Persistent): """ Persistent version of default quixote Session class. This class can be overridden if desired. """ pass class DurusSessionManager (SessionManager): """ Loads/stores quixote sessions from/to a Durus db. Overrides forget_changes() and commit_changes() as described in quixote docs and altdemo.py. """ def __init__ (self, durus_connection, session_class=PersistentSession): """ Load sessions from the durus db and init the SessionManager. (inst, class) : inst durus_connection: a Connection object to a Durus db. session_class: session class to use (must override PeristentSession). """ self.durus_connection = durus_connection root = self.durus_connection.get_root() sessions = root.get('auth.sessions', None) if not sessions: sessions = PersistentDict() root['auth.sessions'] = sessions self.durus_connection.commit() SessionManager.__init__(self, session_class=session_class, session_mapping=sessions) def forget_changes (self, session): self.durus_connection.abort() def commit_changes (self, session): self.durus_connection.commit() ----- Can anyone tell me why my Durus database thinks it's in a conflicted state? Thank you, Salman </code><code></code></pre> </blockquote> <br> </div> </div> Requests for static files are not the only ones that are likely to fail. AJAX requests also tend to cause HTTP 500. Again, the failures are unpredictable,<br> but usually one in a group of three or four simultaneous AJAX requests by the browser can fail. <br> <br> And the AssertionError mentioned above is not the only error raised by the application. I have noticed a WriteConflictError too, albeit less frequently: <div><br> <pre>Traceback (most recent call last):</pre> </div> <pre><div> File "/usr/lib/python2.4/site-packages/quixote2/publish.py", line 275, in process_request output = self.try_publish(request) File "/usr/lib/python2.4/site-packages/quixote2/publish.py", line 255, in try_publish self.finish_successful_request() File "/usr/lib/python2.4/site-packages/quixote2/publish.py", line 138, in finish_successful_request self.session_manager.finish_successful_request() File "/usr/lib/python2.4/site-</div>packages/quixote2/session.py", line 388, in finish_successful_request self.commit_changes(session) File "/usr/lib/python2.4/site-packages/ACE_rms/auth/session.py", line 45, in commit_changes self.durus_connection.commit() File "/usr/lib/python2.4/site-packages/durus/connection.py", line 273, in commit self.storage.end(self._handle_invalidations) File "/usr/lib/python2.4/site-packages/durus/client_storage.py", line 82, in end handle_invalidations(oid_list) File "/usr/lib/python2.4/site-packages/durus/connection.py", line 303, in _handle_invalidations raise WriteConflictError(conflicts) WriteConflictError: oids=[2]</pre> I'm still learning Durus, but I suspect quite a few people on this list are familiar with its internals.<br> <br> Thanks,<br> Salman<br> <br> </div> <br> _______________________________________________<br> Quixote-users mailing list<br> <a moz-do-not-send="true" href="mailto:[email protected]" target="_blank">[email protected]</a><br> <a moz-do-not-send="true" href="http://mail.mems-exchange.org/mailman/listinfo/quixote-users" target="_blank">http://mail.mems-exchange.org/mailman/listinfo/quixote-users</a><br> <br> </blockquote> </div> <br> </blockquote> <br> </body> </html> --------------080200000409050804050108-- --===============0914980726== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Quixote-users mailing list [email protected] http://mail.mems-exchange.org/mailman/listinfo/quixote-users --===============0914980726==--