AW: AW: DorothyLocker and performance

Hi!

Ok, that one works now. Commited to CVS. Thanks.

bye, Georg

-----Urspr=FCngliche Nachricht-----
Von: Thomas Klaeger [mailto:[email protected]]
Gesendet: Mittwoch, 29. September 2004 13:33
An: Bauer, Georg
Cc: 'Garth T Kidd'; 'PyDS Developer List (E-Mail)'
Betreff: Re: AW: [Pyds-dev] DorothyLocker and performance


Hello!

My favority patch:

thread.lock.acquire() returns nothing if called without argument,=20
according to the docs for python 2.2/2.3, even if on win32 it seems to=20
return something.

The attached patch calls thread.lock.acquire(1), which returns true if=20
the acquiration succeded, so the following assert won't fail.

Regards, Thomas

Bauer, Georg schrieb:
> Hi!
>=20
> I now get the following. Looks like the _acquire fails in =
DorothyLocker -
> but I don't know why. Without DorothyLocker it works fine. Another =
2.2
> problem?
>=20
> bye, Georg
>=20
> _acquire:=20
> Exception exceptions.AssertionError:=20
> ('/opt/pyds/lib/python2.2/site-packages/PyDS/Tool.py', 1393, =
'_acquire',
> 'return self.lock.acquire(blocking)')
> ('/opt/pyds/lib/python2.2/site-packages/PyDS/DorothyLocker.py', 342,
> 'acquire', 'assert result # ... but you can never be too careful.')
> -----------------------
> ReleaseOfUnAcquiredLock
> -----------------------
> An attempt to release an un-acquired lock. Probably caused by failure =
to
> acquire.
> Detection context:
>   File "/opt/pyds/lib/python2.2/site-packages/PyDS/EventsTool.py", =
line
93,
> in _log
>   File "/opt/pyds/lib/python2.2/site-packages/PyDS/EventsTool.py", =
line
99,
> in _logVerbose
>   File "/opt/pyds/lib/python2.2/site-packages/PyDS/Tool.py", line =
1631, in
> logVerbose
>   File "/opt/pyds/lib/python2.2/site-packages/PyDS/Tool.py", line =
1322, in
> thunk
>   File "/opt/pyds/lib/python2.2/threading.py", line 396, in run
>   File "/opt/pyds/lib/python2.2/threading.py", line 408, in =
__bootstrap
> Exception in thread aggregator:
> Traceback (most recent call last):
>   File "/opt/pyds/lib/python2.2/threading.py", line 408, in =
__bootstrap
>     self.run()
>   File "/opt/pyds/lib/python2.2/threading.py", line 396, in run
>     apply(self.__target, self.__args, self.__kwargs)
>   File "/opt/pyds/lib/python2.2/site-packages/PyDS/Tool.py", line =
1322, in
> thunk
>     tool.logVerbose(_('Background process started'))
>   File "/opt/pyds/lib/python2.2/site-packages/PyDS/Tool.py", line =
1631, in
> logVerbose
>     getTool('events')._logVerbose(self.name, msg)
>   File "/opt/pyds/lib/python2.2/site-packages/PyDS/EventsTool.py", =
line
99,
> in _logVerbose
>     self._log(tool, message)
>   File "/opt/pyds/lib/python2.2/site-packages/PyDS/EventsTool.py", =
line
93,
> in _log
>     finally: self._release()
>   File "/opt/pyds/lib/python2.2/site-packages/PyDS/Tool.py", line =
1407, in
> _release
>     return self.lock.release()
>   File "/opt/pyds/lib/python2.2/site-packages/PyDS/DorothyLocker.py", =
line
> 363, in release
>     self.__sanity_check('release', my_context)
>   File "/opt/pyds/lib/python2.2/site-packages/PyDS/DorothyLocker.py", =
line
> 290, in __sanity_check
>     raise ReleaseOfUnAcquiredLock, (None, my_context)
> ReleaseOfUnAcquiredLock: (None, <PyDS.DorothyLocker.CallingContext =
object
at
> 0x8927614>, 'An attempt to release an un-acquired lock. Probably =
caused by
> failure to acquire.')
>=20
> bye, Georg
>=20
> -----Urspr=FCngliche Nachricht-----
> Von: Garth T Kidd [mailto:garth-OnzZ1s1DREKDegMON/[email protected]]
> Gesendet: Dienstag, 28. September 2004 22:29
> An: 'Bauer, Georg'; 'Thomas Klaeger'
> Cc: 'PyDS Developer List (E-Mail)'
> Betreff: RE: [Pyds-dev] DorothyLocker and performance
>=20
>=20
> cvs diff -c PyDS\DorothyLocker.py > 20040928-2222-dorothyagain.diff
>=20
> I've updated, backed out the conflict damage (CVS deciding to replace
whole
> sections of identical code when only __revision__ had changed), and
> re-generated. Attached.=20
>=20
> I'm beginning to like unit tests. It was quite reassuring to run the
module
> under 2.2 and see a line of dots followed by OK. I'd otherwise have =
spent
a
> lot of time stressing that I had some *other* 2.2 compatibility =
issues
> somewhere. I can't run PyDS up wholesale under 2.2 because all my =
third
> party libraries are loaded for 2.3...=20
>=20
> -----Original Message-----
> From: pyds-dev-admin-iYtK5bfT9M//Ad8WF/[email protected]
> [mailto:pyds-dev-admin-iYtK5bfT9M//Ad8WF/[email protected]] On Behalf Of Bauer, =
Georg
> Sent: Wednesday, 29 September 2004 1:02 AM
> To: 'Garth T Kidd'; 'Thomas Klaeger'
> Cc: 'PyDS Developer List (E-Mail)'
> Subject: AW: [Pyds-dev] DorothyLocker and performance
>=20
> Hi!
>=20
> Sorry, but that's not a patch against the current CVS. That patch =
can't be
> merged because it contains stuff that was already applied or =
something
like
> that:
>=20
> bash-2.05a$ patch --dry-run -p0 <20040928-2222-dorothyagain.diff =
patching
> file PyDS/DorothyLocker.py Reversed (or previously applied) patch
detected!
> Assume -R? [n] n Apply anyway? [n] y Hunk #1 FAILED at 1.
> Hunk #2 FAILED at 37.
> 2 out of 2 hunks FAILED -- saving rejects to file
PyDS/DorothyLocker.py.rej
> bash-2.05a$ patch --dry-run -p0 <20040928-2222-dorothyagain.diff =
patching
> file PyDS/DorothyLocker.py Reversed (or previously applied) patch
detected!
> Assume -R? [n] y Hunk #2 FAILED at 21.
> 1 out of 2 hunks FAILED -- saving rejects to file
PyDS/DorothyLocker.py.rej
>=20
> bye, Georg
>=20
> -----Urspr=FCngliche Nachricht-----
> Von: Garth T Kidd [mailto:garth-OnzZ1s1DREKDegMON/[email protected]]
> Gesendet: Dienstag, 28. September 2004 14:23
> An: 'Bauer, Georg'; 'Thomas Klaeger'
> Cc: 'PyDS Developer List (E-Mail)'
> Betreff: RE: [Pyds-dev] DorothyLocker and performance
>=20
>=20
> That's a weird one.=20
>=20
> Oh, hang on.
>=20
> E.E..E.........E.E....E
>=20
> Yep. Python 2.2 works differently to Python 2.3. Here's a fix.=20
>=20
> -----Original Message-----
> From: pyds-dev-admin-iYtK5bfT9M//Ad8WF/[email protected]
> [mailto:pyds-dev-admin-iYtK5bfT9M//Ad8WF/[email protected]] On Behalf Of Bauer, =
Georg
> Sent: Tuesday, 28 September 2004 5:24 PM
> To: 'Garth T Kidd'; Bauer, Georg; 'Thomas Klaeger'
> Cc: PyDS Developer List (E-Mail)
> Subject: AW: [Pyds-dev] DorothyLocker and performance
>=20
> Hi!
>=20
>=20
>>Oh, did I forget to send that one in? Sorry. :| I had that, =
alternative=20
>>logging for those post-shutdown logs, and code to make sure =
EventsTool=20
>>was last.
>=20
>=20
> Your patch wasn't able to be applied - I wrote you this directly and
> off-list :-)
>=20
> My solution is a bit more flexible, though: you now have a flag to =
tell
> where to sort the tool on shutdown. I use that to move the =
PreferencesTool
> directly before EventsTool so that the preferences can be still =
fetched
> until the very last moment in shutdown.
>=20
>=20
>>You're calling PyDS.Tool.getToolsForShutdown twice, just like you =
used=20
>>to getTools. Is there anything wrong with saving the value and using =
it
>=20
> twice?
>=20
>=20
> Changed that, although it really doesn't matter much - the shutdown =
is
only
> called once in a running instance ;-)
>=20
>=20
>>Could you drop in this latest version of DorothyLocker? You're now a=20
>>fair way behind, and I've lost track of when I fixed certain bugs. =
That=20
>>might
>=20
>=20
> Did that. But it would really be a bit better if you stick to CVS
mechanisms
> - just do an update and merge in the changes I commited and then =
reproduce
> your patch with regard to current CVS. It's much more likely that I =
put in
a
> working patch than full module sources, as the latter would require =
me to
> make the diff myself to see what changed. And I am known to be lazy =
;-)
>=20
> For example this time: after adding the full module I could'nt start =
PyDS,
> because of some changes to the interface of DorothyLocker that =
required
> additional patches. But your patch didn't work, as your full module
already
> was installed. Mixing different patch/dist methods allways makes =
things
more
> complicated.
>=20
> Oh, and now the CVS is broken. One thing was a small typo in Tool.py
(wrong
> indention) that was easily fixed. But now it barfs about "sequence =
must be
> integer" on start. What's that? It happens in your __getitem__ method =
in
> your FramingContext where it accesses the framestack (line 103). =
Looks
like
> ltrim_common doesn't produce integers as index somehow (maybe pos not
> initialized?). This happens directly at startup of PyDS:
>=20
> Exception exceptions.TypeError: sequence index must be integer
> ('/opt/pyds/lib/python2.2/site-packages/PyDS/Tool.py', 1407, =
'_release',
> 'return self.lock.release()')
> ('/opt/pyds/lib/python2.2/site-packages/PyDS/DorothyLocker.py', 359,
> 'release', "self.__sanity_check('release', my_context)")
> ('/opt/pyds/lib/python2.2/site-packages/PyDS/DorothyLocker.py', 260,
> '__sanity_check', 'lock_uniques, my_uniques =3D =
ltrim_common(lock_context,
> my_context)')
> ('/opt/pyds/lib/python2.2/site-packages/PyDS/DorothyLocker.py', 54,
> 'ltrim_common', 'return oblist1[pos:], oblist2[pos:]')
> ('/opt/pyds/lib/python2.2/site-packages/PyDS/DorothyLocker.py', 103,
> '__getitem__', 'return self.framestack[index]') Traceback (most =
recent
call
> last):
>    File "/opt/pyds/bin/pyds-start", line 51, in ?
>      PyDS.Server.start_server(daemon=3D_PyDS.daemon)
>    File "/opt/pyds/lib/python2.2/site-packages/PyDS/Server.py", line =
298,
in
> start_server
>      PyDS.Tool.initTools(PyDS.Tool.mainstatus)
>    File "/opt/pyds/lib/python2.2/site-packages/PyDS/Tool.py", line =
317, in
> initTools
>      tool._initdb()
>    File "/opt/pyds/lib/python2.2/site-packages/PyDS/WeblogTool.py", =
line
77,
> in _initdb
>      finally: self._release()
>    File "/opt/pyds/lib/python2.2/site-packages/PyDS/Tool.py", line =
1407,
in
> _release
>      return self.lock.release()
>    File =
"/opt/pyds/lib/python2.2/site-packages/PyDS/DorothyLocker.py",
> line 359, in release
>      self.__sanity_check('release', my_context)
>    File =
"/opt/pyds/lib/python2.2/site-packages/PyDS/DorothyLocker.py",
> line 260, in __sanity_check
>      lock_uniques, my_uniques =3D ltrim_common(lock_context, =
my_context)
>    File =
"/opt/pyds/lib/python2.2/site-packages/PyDS/DorothyLocker.py",
> line 54, in ltrim_common
>      return oblist1[pos:], oblist2[pos:]
>    File =
"/opt/pyds/lib/python2.2/site-packages/PyDS/DorothyLocker.py",
> line 103, in __getitem__
>      return self.framestack[index]
> TypeError: sequence index must be integer
>=20
>=20
>=20
>>also be behind some of the performance hits you're seeing (I used the =

>>latest, assuming you'd incorporated it after that submission I made=20
>>that
>=20
> got
>=20
>>blocked for size).
>=20
>=20
> I usually write about what I commited (and at least it's in the CVS =
RSS
> feed). So if you don't see me mentioning it, it supposedly isn't in =
CVS. I
> either wasn't able to apply it (then I usually write something) or I =
just
> plain missed it.
>=20
> bye, Georg
>=20
>=20
> _______________________________________________
> Pyds-dev mailing list
> Pyds-dev-iYtK5bfT9M//Ad8WF/[email protected]
> http://www.westfalen.de/cgi-bin/mailman/listinfo/pyds-dev
>=20
>=20
> _______________________________________________
> Pyds-dev mailing list
> Pyds-dev-iYtK5bfT9M//Ad8WF/[email protected]
> http://www.westfalen.de/cgi-bin/mailman/listinfo/pyds-dev
>=20
>=20