RE: AW: DorothyLocker and performance

Did the unit tests show anything before you applied your fix? If I need
another regression test, I'd like to add it. If not, it looks like the =
code
is in pretty good shape. All I need to do now is figure out why ignoring
self._acquire.func_code wasn't working in situ...=20

-----Original Message-----
From: pyds-dev-admin-iYtK5bfT9M//Ad8WF/[email protected]
[mailto:pyds-dev-admin-iYtK5bfT9M//Ad8WF/[email protected]] On Behalf Of Thomas =
Klaeger
Sent: Wednesday, 29 September 2004 9:33 PM
To: Bauer, Georg
Cc: 'Garth T Kidd'; 'PyDS Developer List (E-Mail)'
Subject: Re: AW: [Pyds-dev] DorothyLocker and performance

Hello!

My favority patch:

thread.lock.acquire() returns nothing if called without argument, =
according
to the docs for python 2.2/2.3, even if on win32 it seems to return
something.

The attached patch calls thread.lock.acquire(1), which returns true if =
the
acquiration succeded, so the following assert won't fail.

Regards, Thomas

Bauer, Georg schrieb:
> Hi!
>=20
> I now get the following. Looks like the _acquire fails in=20
> DorothyLocker - but I don't know why. Without DorothyLocker it works=20
> fine. Another 2.2 problem?
>=20
> bye, Georg
>=20
> _acquire:=20
> Exception exceptions.AssertionError:=20
> ('/opt/pyds/lib/python2.2/site-packages/PyDS/Tool.py', 1393,=20
> '_acquire', 'return self.lock.acquire(blocking)')=20
> ('/opt/pyds/lib/python2.2/site-packages/PyDS/DorothyLocker.py', 342,=20
> 'acquire', 'assert result # ... but you can never be too careful.')
> -----------------------
> ReleaseOfUnAcquiredLock
> -----------------------
> An attempt to release an un-acquired lock. Probably caused by failure=20
> to acquire.
> Detection context:
>   File "/opt/pyds/lib/python2.2/site-packages/PyDS/EventsTool.py",=20
> line 93, in _log
>   File "/opt/pyds/lib/python2.2/site-packages/PyDS/EventsTool.py",=20
> line 99, in _logVerbose
>   File "/opt/pyds/lib/python2.2/site-packages/PyDS/Tool.py", line=20
> 1631, in logVerbose
>   File "/opt/pyds/lib/python2.2/site-packages/PyDS/Tool.py", line=20
> 1322, in thunk
>   File "/opt/pyds/lib/python2.2/threading.py", line 396, in run
>   File "/opt/pyds/lib/python2.2/threading.py", line 408, in=20
> __bootstrap Exception in thread aggregator:
> Traceback (most recent call last):
>   File "/opt/pyds/lib/python2.2/threading.py", line 408, in =
__bootstrap
>     self.run()
>   File "/opt/pyds/lib/python2.2/threading.py", line 396, in run
>     apply(self.__target, self.__args, self.__kwargs)
>   File "/opt/pyds/lib/python2.2/site-packages/PyDS/Tool.py", line=20
> 1322, in thunk
>     tool.logVerbose(_('Background process started'))
>   File "/opt/pyds/lib/python2.2/site-packages/PyDS/Tool.py", line=20
> 1631, in logVerbose
>     getTool('events')._logVerbose(self.name, msg)
>   File "/opt/pyds/lib/python2.2/site-packages/PyDS/EventsTool.py",=20
> line 99, in _logVerbose
>     self._log(tool, message)
>   File "/opt/pyds/lib/python2.2/site-packages/PyDS/EventsTool.py",=20
> line 93, in _log
>     finally: self._release()
>   File "/opt/pyds/lib/python2.2/site-packages/PyDS/Tool.py", line=20
> 1407, in _release
>     return self.lock.release()
>   File "/opt/pyds/lib/python2.2/site-packages/PyDS/DorothyLocker.py",=20
> line 363, in release
>     self.__sanity_check('release', my_context)
>   File "/opt/pyds/lib/python2.2/site-packages/PyDS/DorothyLocker.py",=20
> line 290, in __sanity_check
>     raise ReleaseOfUnAcquiredLock, (None, my_context)
> ReleaseOfUnAcquiredLock: (None, <PyDS.DorothyLocker.CallingContext=20
> object at
> 0x8927614>, 'An attempt to release an un-acquired lock. Probably=20
> 0x8927614>caused by
> failure to acquire.')
>=20
> bye, Georg
>=20
> -----Urspr=FCngliche Nachricht-----
> Von: Garth T Kidd [mailto:garth-OnzZ1s1DREKDegMON/[email protected]]
> Gesendet: Dienstag, 28. September 2004 22:29
> An: 'Bauer, Georg'; 'Thomas Klaeger'
> Cc: 'PyDS Developer List (E-Mail)'
> Betreff: RE: [Pyds-dev] DorothyLocker and performance
>=20
>=20
> cvs diff -c PyDS\DorothyLocker.py > 20040928-2222-dorothyagain.diff
>=20
> I've updated, backed out the conflict damage (CVS deciding to replace=20
> whole sections of identical code when only __revision__ had changed),=20
> and re-generated. Attached.
>=20
> I'm beginning to like unit tests. It was quite reassuring to run the=20
> module under 2.2 and see a line of dots followed by OK. I'd otherwise=20
> have spent a lot of time stressing that I had some *other* 2.2=20
> compatibility issues somewhere. I can't run PyDS up wholesale under=20
> 2.2 because all my third party libraries are loaded for 2.3...
>=20
> -----Original Message-----
> From: pyds-dev-admin-iYtK5bfT9M//Ad8WF/[email protected]
> [mailto:pyds-dev-admin-iYtK5bfT9M//Ad8WF/[email protected]] On Behalf Of Bauer,=20
> Georg
> Sent: Wednesday, 29 September 2004 1:02 AM
> To: 'Garth T Kidd'; 'Thomas Klaeger'
> Cc: 'PyDS Developer List (E-Mail)'
> Subject: AW: [Pyds-dev] DorothyLocker and performance
>=20
> Hi!
>=20
> Sorry, but that's not a patch against the current CVS. That patch=20
> can't be merged because it contains stuff that was already applied or=20
> something like
> that:
>=20
> bash-2.05a$ patch --dry-run -p0 <20040928-2222-dorothyagain.diff=20
> patching file PyDS/DorothyLocker.py Reversed (or previously applied) =
patch
detected!
> Assume -R? [n] n Apply anyway? [n] y Hunk #1 FAILED at 1.
> Hunk #2 FAILED at 37.
> 2 out of 2 hunks FAILED -- saving rejects to file=20
> PyDS/DorothyLocker.py.rej bash-2.05a$ patch --dry-run -p0=20
> <20040928-2222-dorothyagain.diff patching file PyDS/DorothyLocker.py
Reversed (or previously applied) patch detected!
> Assume -R? [n] y Hunk #2 FAILED at 21.
> 1 out of 2 hunks FAILED -- saving rejects to file=20
> PyDS/DorothyLocker.py.rej
>=20
> bye, Georg
>=20
> -----Urspr=FCngliche Nachricht-----
> Von: Garth T Kidd [mailto:garth-OnzZ1s1DREKDegMON/[email protected]]
> Gesendet: Dienstag, 28. September 2004 14:23
> An: 'Bauer, Georg'; 'Thomas Klaeger'
> Cc: 'PyDS Developer List (E-Mail)'
> Betreff: RE: [Pyds-dev] DorothyLocker and performance
>=20
>=20
> That's a weird one.=20
>=20
> Oh, hang on.
>=20
> E.E..E.........E.E....E
>=20
> Yep. Python 2.2 works differently to Python 2.3. Here's a fix.=20
>=20
> -----Original Message-----
> From: pyds-dev-admin-iYtK5bfT9M//Ad8WF/[email protected]
> [mailto:pyds-dev-admin-iYtK5bfT9M//Ad8WF/[email protected]] On Behalf Of Bauer,=20
> Georg
> Sent: Tuesday, 28 September 2004 5:24 PM
> To: 'Garth T Kidd'; Bauer, Georg; 'Thomas Klaeger'
> Cc: PyDS Developer List (E-Mail)
> Subject: AW: [Pyds-dev] DorothyLocker and performance
>=20
> Hi!
>=20
>=20
>>Oh, did I forget to send that one in? Sorry. :| I had that,=20
>>alternative logging for those post-shutdown logs, and code to make=20
>>sure EventsTool was last.
>=20
>=20
> Your patch wasn't able to be applied - I wrote you this directly and=20
> off-list :-)
>=20
> My solution is a bit more flexible, though: you now have a flag to=20
> tell where to sort the tool on shutdown. I use that to move the=20
> PreferencesTool directly before EventsTool so that the preferences can =

> be still fetched until the very last moment in shutdown.
>=20
>=20
>>You're calling PyDS.Tool.getToolsForShutdown twice, just like you used =

>>to getTools. Is there anything wrong with saving the value and using=20
>>it
>=20
> twice?
>=20
>=20
> Changed that, although it really doesn't matter much - the shutdown is =

> only called once in a running instance ;-)
>=20
>=20
>>Could you drop in this latest version of DorothyLocker? You're now a=20
>>fair way behind, and I've lost track of when I fixed certain bugs.=20
>>That might
>=20
>=20
> Did that. But it would really be a bit better if you stick to CVS=20
> mechanisms
> - just do an update and merge in the changes I commited and then=20
> reproduce your patch with regard to current CVS. It's much more likely =

> that I put in a working patch than full module sources, as the latter=20
> would require me to make the diff myself to see what changed. And I am =

> known to be lazy ;-)
>=20
> For example this time: after adding the full module I could'nt start=20
> PyDS, because of some changes to the interface of DorothyLocker that=20
> required additional patches. But your patch didn't work, as your full=20
> module already was installed. Mixing different patch/dist methods=20
> allways makes things more complicated.
>=20
> Oh, and now the CVS is broken. One thing was a small typo in Tool.py=20
> (wrong
> indention) that was easily fixed. But now it barfs about "sequence=20
> must be integer" on start. What's that? It happens in your __getitem__ =

> method in your FramingContext where it accesses the framestack (line=20
> 103). Looks like ltrim_common doesn't produce integers as index=20
> somehow (maybe pos not initialized?). This happens directly at startup =
of
PyDS:
>=20
> Exception exceptions.TypeError: sequence index must be integer=20
> ('/opt/pyds/lib/python2.2/site-packages/PyDS/Tool.py', 1407,=20
> '_release', 'return self.lock.release()')=20
> ('/opt/pyds/lib/python2.2/site-packages/PyDS/DorothyLocker.py', 359,=20
> 'release', "self.__sanity_check('release', my_context)")=20
> ('/opt/pyds/lib/python2.2/site-packages/PyDS/DorothyLocker.py', 260,=20
> '__sanity_check', 'lock_uniques, my_uniques =3D=20
> ltrim_common(lock_context,
> my_context)')
> ('/opt/pyds/lib/python2.2/site-packages/PyDS/DorothyLocker.py', 54,=20
> 'ltrim_common', 'return oblist1[pos:], oblist2[pos:]')=20
> ('/opt/pyds/lib/python2.2/site-packages/PyDS/DorothyLocker.py', 103,=20
> '__getitem__', 'return self.framestack[index]') Traceback (most recent =

> call
> last):
>    File "/opt/pyds/bin/pyds-start", line 51, in ?
>      PyDS.Server.start_server(daemon=3D_PyDS.daemon)
>    File "/opt/pyds/lib/python2.2/site-packages/PyDS/Server.py", line=20
> 298, in start_server
>      PyDS.Tool.initTools(PyDS.Tool.mainstatus)
>    File "/opt/pyds/lib/python2.2/site-packages/PyDS/Tool.py", line=20
> 317, in initTools
>      tool._initdb()
>    File "/opt/pyds/lib/python2.2/site-packages/PyDS/WeblogTool.py",=20
> line 77, in _initdb
>      finally: self._release()
>    File "/opt/pyds/lib/python2.2/site-packages/PyDS/Tool.py", line=20
> 1407, in _release
>      return self.lock.release()
>    File "/opt/pyds/lib/python2.2/site-packages/PyDS/DorothyLocker.py",
> line 359, in release
>      self.__sanity_check('release', my_context)
>    File "/opt/pyds/lib/python2.2/site-packages/PyDS/DorothyLocker.py",
> line 260, in __sanity_check
>      lock_uniques, my_uniques =3D ltrim_common(lock_context, =
my_context)
>    File "/opt/pyds/lib/python2.2/site-packages/PyDS/DorothyLocker.py",
> line 54, in ltrim_common
>      return oblist1[pos:], oblist2[pos:]
>    File "/opt/pyds/lib/python2.2/site-packages/PyDS/DorothyLocker.py",
> line 103, in __getitem__
>      return self.framestack[index]
> TypeError: sequence index must be integer
>=20
>=20
>=20
>>also be behind some of the performance hits you're seeing (I used the=20
>>latest, assuming you'd incorporated it after that submission I made=20
>>that
>=20
> got
>=20
>>blocked for size).
>=20
>=20
> I usually write about what I commited (and at least it's in the CVS=20
> RSS feed). So if you don't see me mentioning it, it supposedly isn't=20
> in CVS. I either wasn't able to apply it (then I usually write=20
> something) or I just plain missed it.
>=20
> bye, Georg
>=20
>=20
> _______________________________________________
> Pyds-dev mailing list
> Pyds-dev-iYtK5bfT9M//Ad8WF/[email protected]
> http://www.westfalen.de/cgi-bin/mailman/listinfo/pyds-dev
>=20
>=20
> _______________________________________________
> Pyds-dev mailing list
> Pyds-dev-iYtK5bfT9M//Ad8WF/[email protected]
> http://www.westfalen.de/cgi-bin/mailman/listinfo/pyds-dev
>=20
>=20