Re: Persistent Session Management with Durus
Gabor Vitez <[email protected]> Thu, 26 Aug 2010 20:01:39 +0200
| Newsgroups | gmane.comp.web.quixote.user |
|---|---|
| Message-ID | <[email protected]> |
--===============1286729468==
Content-Type: multipart/alternative; boundary=002215046dcf9971e7048ebdcced
--002215046dcf9971e7048ebdcced
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Hi!
As far as I can tell, this is normal behaviour with Durus; I'd suggest
wrapping every function that reads or writes durus with something like this=
:
def writetransaction(func):
"""wrap writers with this"""
def wrapper(*args, **kw):
conn=3Ddbconn.get_db()
#bail after 1000 failures
for dummy1 in xrange(1000):
try:
results=3Dfunc(*args,**kw)
conn.commit()
return results
except ConflictError:
pass
conn.abort()
raise ConflictError
return wrapper
def readtransaction(func):
"""wrap readers with this"""
def wrapper(*args, **kw):
conn=3Ddbconn.get_db()
for dummy1 in xrange(1000):
try:
return func(*args,**kw)
except ConflictError:
pass
conn.abort()
raise ConflictError
return wrapper
dbconn is the global database connection object
Best,
G=E1bor
On Thu, Aug 26, 2010 at 5:23 PM, Salman Haq <[email protected]> wrote=
:
> Some new info included at the end:
>
>
>
> On 08/26/2010 10:49 AM, Salman Haq wrote:
>
> Hi,
>
> I'm developing an application using Quixote 2.6 and following the example=
in altdemo.py pretty closely for persistent session management with Durus.
> The application is serving static files very unreliably. One in five requ=
ests for a static file receive a HTTP 500 response code. Eg: if an html pag=
e embeds 50
> script or img tags, requests for about 10 of them will fail. For each of =
these failed requests, the following traceback is received:
>
> Traceback (most recent call last): File "/usr/lib/python2.4/site-package=
s/quixote2/publish.py", line 275, in process_request output =3D self.try=
_publish(request) File "/usr/lib/python2.4/site-packages/quixote2/publish.=
py", line 255, in try_publish self.finish_successful_request() File "/u=
sr/lib/python2.4/site-packages/quixote2/publish.py", line 138, in finish_su=
ccessful_request self.session_manager.finish_successful_request() File =
"/usr/lib/python2.4/site-packages/quixote2/session.py", line 388, in finish=
_successful_request self.commit_changes(session) File "/usr/lib/python2=
..4/site-packages/ACE_rms/auth/session.py", line 44, in commit_changes se=
lf.durus_connection.commit() File "/usr/lib/python2.4/site-packages/durus/=
connection.py", line 254, in commit assert not self.invalid_oids, "still=
conflicted: missing abort()"AssertionError: still conflicted: missing abor=
t()
>
> My application is running as a SCGI process behind Apache. Static files a=
re served by Quixote via the StaticDirectory class and not by Apache.
> The Durus server runs as a stand-alone process on localhost:2951 and a co=
nnection object to it is created as follows:
>
> def publisher_factory ():
> """ returns a quixote2 Publisher instance that publishes
> the RootDirectory with persistent session management support.
> """
> conn =3D Connection(ClientStorage("127.0.0.1", 2951))
> session_manager =3D DurusSessionManager(conn)
> return Publisher(RootDirectory(), session_manager=3Dsession_manager, =
display_exceptions=3D'plain')
>
>
> The application sub-classes SessionManager and Session exactly as describ=
ed in the snippet below:
>
> ----
>
> from quixote2.session import Session, SessionManager
> from durus.persistent import Persistent
> from durus.persistent_dict import PersistentDict
>
> class PersistentSession (Session, Persistent):
> """ Persistent version of default quixote Session class.
> This class can be overridden if desired.
> """
> pass
>
> class DurusSessionManager (SessionManager):
> """ Loads/stores quixote sessions from/to a Durus db.
> Overrides forget_changes() and commit_changes() as described in quixo=
te
> docs and altdemo.py.
> """
>
> def __init__ (self, durus_connection, session_class=3DPersistentSessi=
on):
> """ Load sessions from the durus db and init the SessionManager.
> (inst, class) : inst
> durus_connection: a Connection object to a Durus db.
> session_class: session class to use (must override PeristentSessi=
on).
> """
> self.durus_connection =3D durus_connection
> root =3D self.durus_connection.get_root()
> sessions =3D root.get('auth.sessions', None)
>
> if not sessions:
> sessions =3D PersistentDict()
> root['auth.sessions'] =3D sessions
> self.durus_connection.commit()
>
> SessionManager.__init__(self,
> session_class=3Dsession_class,
> session_mapping=3Dsessions)
>
> def forget_changes (self, session):
> self.durus_connection.abort()
>
> def commit_changes (self, session):
> self.durus_connection.commit()
>
> -----
>
> Can anyone tell me why my Durus database thinks it's in a conflicted stat=
e?
>
> Thank you,
> Salman
>
>
> Requests for static files are not the only ones that are likely to fail.
> AJAX requests also tend to cause HTTP 500. Again, the failures are
> unpredictable,
> but usually one in a group of three or four simultaneous AJAX requests by
> the browser can fail.
>
> And the AssertionError mentioned above is not the only error raised by th=
e
> application. I have noticed a WriteConflictError too, albeit less
> frequently:
>
> Traceback (most recent call last):
>
> File "/usr/lib/python2.4/site-packages/quixote2/publish.py", line 2=
75, in process_request
> output =3D self.try_publish(request)
> File "/usr/lib/python2.4/site-packages/quixote2/publish.py", line 2=
55, in try_publish
> self.finish_successful_request()
> File "/usr/lib/python2.4/site-packages/quixote2/publish.py", line 1=
38, in finish_successful_request
> self.session_manager.finish_successful_request()
> File "/usr/lib/python2.4/site-
> packages/quixote2/session.py", line 388, in finish_successful_request
> self.commit_changes(session)
> File "/usr/lib/python2.4/site-packages/ACE_rms/auth/session.py", li=
ne 45, in commit_changes
> self.durus_connection.commit()
> File "/usr/lib/python2.4/site-packages/durus/connection.py", line 2=
73, in commit
> self.storage.end(self._handle_invalidations)
> File "/usr/lib/python2.4/site-packages/durus/client_storage.py", li=
ne 82, in end
> handle_invalidations(oid_list)
> File "/usr/lib/python2.4/site-packages/durus/connection.py", line 3=
03, in _handle_invalidations
> raise WriteConflictError(conflicts)
> WriteConflictError: oids=3D[2]
>
> I'm still learning Durus, but I suspect quite a few people on this list a=
re
> familiar with its internals.
>
> Thanks,
> Salman
>
>
> _______________________________________________
> Quixote-users mailing list
> [email protected]
> http://mail.mems-exchange.org/mailman/listinfo/quixote-users
>
>
--002215046dcf9971e7048ebdcced
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Hi!<br><br>As far as I can tell, this is normal behaviour with Durus; I'=
;d suggest wrapping every function that reads or writes durus with somethin=
g like this:<br><br><span style=3D"font-family: courier new,monospace;">def=
writetransaction(func):</span><br style=3D"font-family: courier new,monosp=
ace;">
<span style=3D"font-family: courier new,monospace;"></span><span style=3D"f=
ont-family: courier new,monospace;">=A0 """wrap writers with=
this"""<br>=A0 def wrapper(*args, **kw):</span><br style=3D=
"font-family: courier new,monospace;">
<span style=3D"font-family: courier new,monospace;">=A0=A0=A0 conn=3Ddbconn=
..get_db()<br>=A0=A0=A0 #bail after 1000 failures<br style=3D"font-family: c=
ourier new,monospace;"></span><span style=3D"font-family: courier new,monos=
pace;">=A0=A0=A0 for dummy1 in xrange(1000):</span><br style=3D"font-family=
: courier new,monospace;">
<span style=3D"font-family: courier new,monospace;">=A0=A0=A0=A0=A0 try:</s=
pan><br style=3D"font-family: courier new,monospace;"><span style=3D"font-f=
amily: courier new,monospace;">=A0=A0=A0=A0=A0=A0=A0 results=3Dfunc(*args,*=
*kw)</span><br style=3D"font-family: courier new,monospace;">
<span style=3D"font-family: courier new,monospace;">=A0=A0=A0=A0=A0=A0=A0 c=
onn.commit()</span><br style=3D"font-family: courier new,monospace;"><span =
style=3D"font-family: courier new,monospace;">=A0=A0=A0=A0=A0=A0=A0 return =
results</span><br style=3D"font-family: courier new,monospace;">
<span style=3D"font-family: courier new,monospace;">=A0=A0=A0=A0=A0 except =
ConflictError:</span><br style=3D"font-family: courier new,monospace;"><spa=
n style=3D"font-family: courier new,monospace;">=A0=A0=A0=A0=A0=A0=A0 pass<=
/span><br style=3D"font-family: courier new,monospace;">
<span style=3D"font-family: courier new,monospace;">=A0=A0=A0=A0=A0 conn.ab=
ort()</span><br style=3D"font-family: courier new,monospace;"><span style=
=3D"font-family: courier new,monospace;">=A0=A0=A0 raise ConflictError</spa=
n><br style=3D"font-family: courier new,monospace;">
<span style=3D"font-family: courier new,monospace;">=A0 return wrapper</spa=
n><br style=3D"font-family: courier new,monospace;"><br style=3D"font-famil=
y: courier new,monospace;"><span style=3D"font-family: courier new,monospac=
e;">def readtransaction(func):<br>
=A0 """wrap readers with this"""<br style=3D"=
font-family: courier new,monospace;"></span><span style=3D"font-family: cou=
rier new,monospace;"></span><span style=3D"font-family: courier new,monospa=
ce;">=A0 def wrapper(*args, **kw):</span><br style=3D"font-family: courier =
new,monospace;">
<span style=3D"font-family: courier new,monospace;">=A0=A0=A0 conn=3Ddbconn=
..get_db()</span><br style=3D"font-family: courier new,monospace;"><span sty=
le=3D"font-family: courier new,monospace;">=A0=A0=A0 for dummy1 in xrange(1=
000):</span><br style=3D"font-family: courier new,monospace;">
<span style=3D"font-family: courier new,monospace;">=A0=A0=A0=A0=A0 try:</s=
pan><br style=3D"font-family: courier new,monospace;"><span style=3D"font-f=
amily: courier new,monospace;">=A0=A0=A0=A0=A0=A0=A0 return func(*args,**kw=
)</span><br style=3D"font-family: courier new,monospace;">
<span style=3D"font-family: courier new,monospace;">=A0=A0=A0=A0=A0 except =
ConflictError:</span><br style=3D"font-family: courier new,monospace;"><spa=
n style=3D"font-family: courier new,monospace;">=A0=A0=A0=A0=A0=A0=A0 pass<=
/span><br style=3D"font-family: courier new,monospace;">
<span style=3D"font-family: courier new,monospace;">=A0=A0=A0=A0=A0 conn.ab=
ort()</span><br style=3D"font-family: courier new,monospace;"><span style=
=3D"font-family: courier new,monospace;">=A0=A0=A0 raise ConflictError</spa=
n><br style=3D"font-family: courier new,monospace;">
<span style=3D"font-family: courier new,monospace;">=A0 return wrapper</spa=
n><br style=3D"font-family: courier new,monospace;"><br><br>dbconn is the g=
lobal database connection object<br>
<br>Best,<br>G=E1bor<br><br><div class=3D"gmail_quote">
On Thu, Aug 26, 2010 at 5:23 PM, Salman Haq <span dir=3D"ltr"><<a href=
=3D"mailto:[email protected]" target=3D"_blank">[email protected]=
om</a>></span> wrote:<br><blockquote class=3D"gmail_quote" style=3D"bord=
er-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-l=
eft: 1ex;">
=20
<div text=3D"#000000" bgcolor=3D"#ffffff">
Some new info included at the end:<div><div></div><div><br>
<br>
<br>
On 08/26/2010 10:49 AM, Salman Haq wrote:
<blockquote type=3D"cite">
=20
<pre><code>Hi,
I'm developing an application using Quixote 2.6 and following the examp=
le in altdemo.py pretty closely for persistent session management with Duru=
s.
The application is serving static files very unreliably. One in five reques=
ts for a static file receive a HTTP 500 response code. Eg: if an html page =
embeds 50
script or img tags, requests for about 10 of them will fail. For each of th=
ese failed requests, the following traceback is received:
Traceback (most recent call last):
</code><code> File "/usr/lib/python2.4/site-packages/quixote2/publish=
..py", line 275, in process_request
</code><code> output =3D self.try_publish(request)
</code><code> File "/usr/lib/python2.4/site-packages/quixote2/publish=
..py", line 255, in try_publish
</code><code> self.finish_successful_request()
</code><code> File "/usr/lib/python2.4/site-packages/quixote2/publish=
..py", line 138, in finish_successful_request</code><code>
</code><code> self.session_manager.finish_successful_request()
</code><code> File "/usr/lib/python2.4/site-packages/quixote2/session=
..py", line 388, in finish_successful_request</code><code>
</code><code> self.commit_changes(session)
</code><code> File "/usr/lib/python2.4/site-packages/ACE_rms/auth/ses=
sion.py", line 44, in commit_changes
</code><code> self.durus_connection.commit()
</code><code> File "/usr/lib/python2.4/site-packages/durus/connection=
..py", line 254, in commit
</code><code> assert not self.invalid_oids, "still conflicted: miss=
ing abort()"
</code><code>AssertionError: still conflicted: missing abort()
My application is running as a SCGI process behind Apache. Static files are=
served by Quixote via the StaticDirectory class and not by Apache.
The Durus server runs as a stand-alone process on localhost:2951 and a conn=
ection object to it is created as follows:
def publisher_factory ():
""" returns a quixote2 Publisher instance that publishes
the RootDirectory with persistent session management support.
"""
conn =3D Connection(ClientStorage("127.0.0.1", 2951))
session_manager =3D DurusSessionManager(conn)
return Publisher(RootDirectory(), session_manager=3Dsession_manager, di=
splay_exceptions=3D'plain')
The application sub-classes SessionManager and Session exactly as described=
in the snippet below:
----
from quixote2.session import Session, SessionManager
from durus.persistent import Persistent
from durus.persistent_dict import PersistentDict
class PersistentSession (Session, Persistent):
""" Persistent version of default quixote Session class.
This class can be overridden if desired.
"""
pass
class DurusSessionManager (SessionManager):
""" Loads/stores quixote sessions from/to a Durus db.
Overrides forget_changes() and commit_changes() as described in quixote
docs and altdemo.py.
"""
def __init__ (self, durus_connection, session_class=3DPersistentSession=
):
""" Load sessions from the durus db and init the Ses=
sionManager.
(inst, class) : inst
durus_connection: a Connection object to a Durus db.
session_class: session class to use (must override PeristentSession=
).
"""
self.durus_connection =3D durus_connection
root =3D self.durus_connection.get_root()
sessions =3D root.get('auth.sessions', None)
if not sessions:
sessions =3D PersistentDict()
root['auth.sessions'] =3D sessions
self.durus_connection.commit()
SessionManager.__init__(self,
session_class=3Dsession_class,
session_mapping=3Dsessions)
def forget_changes (self, session):
self.durus_connection.abort()
def commit_changes (self, session):
self.durus_connection.commit()
-----
Can anyone tell me why my Durus database thinks it's in a conflicted st=
ate?=20
Thank you,
Salman
</code><code></code></pre>
</blockquote>
<br></div></div>
Requests for static files are not the only ones that are likely to
fail. AJAX requests also tend to cause HTTP 500. Again, the failures
are unpredictable,<br>
but usually one in a group of three or four simultaneous AJAX requests
by the browser can fail. <br>
<br>
And the AssertionError mentioned above is not the only error raised by
the application. I have noticed a WriteConflictError too, albeit less
frequently:<div><br>
<pre>Traceback (most recent call last):</pre>
</div><pre><div>=A0=A0=A0 =A0 File "/usr/lib/python2.4/site-packages/q=
uixote2/publish.py", line 275, in process_request
=A0=A0=A0 =A0=A0=A0 output =3D self.try_publish(request)
=A0=A0=A0 =A0 File "/usr/lib/python2.4/site-packages/quixote2/publish.=
py", line 255, in try_publish
=A0=A0=A0 =A0=A0=A0 self.finish_successful_request()
=A0=A0=A0 =A0 File "/usr/lib/python2.4/site-packages/quixote2/publish.=
py", line 138, in finish_successful_request
=A0=A0=A0 =A0=A0=A0 self.session_manager.finish_successful_request()
=A0=A0=A0 =A0 File "/usr/lib/python2.4/site-</div>packages/quixote2/se=
ssion.py", line 388, in finish_successful_request
=A0=A0=A0 =A0=A0=A0 self.commit_changes(session)
=A0=A0=A0 =A0 File "/usr/lib/python2.4/site-packages/ACE_rms/auth/sess=
ion.py", line 45, in commit_changes
=A0=A0=A0 =A0=A0=A0 self.durus_connection.commit()
=A0=A0=A0 =A0 File "/usr/lib/python2.4/site-packages/durus/connection.=
py", line 273, in commit
=A0=A0=A0 =A0=A0=A0 self.storage.end(self._handle_invalidations)
=A0=A0=A0 =A0 File "/usr/lib/python2.4/site-packages/durus/client_stor=
age.py", line 82, in end
=A0=A0=A0 =A0=A0=A0 handle_invalidations(oid_list)
=A0=A0=A0 =A0 File "/usr/lib/python2.4/site-packages/durus/connection.=
py", line 303, in _handle_invalidations
=A0=A0=A0 =A0=A0=A0 raise WriteConflictError(conflicts)
=A0=A0=A0 WriteConflictError: oids=3D[2]</pre>
I'm still learning Durus, but I suspect quite a few people on this list
are familiar with its internals.<br>
<br>
Thanks,<br>
Salman<br>
<br>
</div>
<br>_______________________________________________<br>
Quixote-users mailing list<br>
<a href=3D"mailto:[email protected]" target=3D"_blank">Quixot=
[email protected]</a><br>
<a href=3D"http://mail.mems-exchange.org/mailman/listinfo/quixote-users" ta=
rget=3D"_blank">http://mail.mems-exchange.org/mailman/listinfo/quixote-user=
s</a><br>
<br></blockquote></div><br>
--002215046dcf9971e7048ebdcced--
--===============1286729468==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
_______________________________________________
Quixote-users mailing list
[email protected]
http://mail.mems-exchange.org/mailman/listinfo/quixote-users
--===============1286729468==--