AW: DorothyLocker and performance

Hi!

I now get the following. Looks like the _acquire fails in DorothyLocker =
-
but I don't know why. Without DorothyLocker it works fine. Another 2.2
problem?

bye, Georg

_acquire:=20
Exception exceptions.AssertionError:=20
('/opt/pyds/lib/python2.2/site-packages/PyDS/Tool.py', 1393, =
'_acquire',
'return self.lock.acquire(blocking)')
('/opt/pyds/lib/python2.2/site-packages/PyDS/DorothyLocker.py', 342,
'acquire', 'assert result # ... but you can never be too careful.')
-----------------------
ReleaseOfUnAcquiredLock
-----------------------
An attempt to release an un-acquired lock. Probably caused by failure =
to
acquire.
Detection context:
  File "/opt/pyds/lib/python2.2/site-packages/PyDS/EventsTool.py", line =
93,
in _log
  File "/opt/pyds/lib/python2.2/site-packages/PyDS/EventsTool.py", line =
99,
in _logVerbose
  File "/opt/pyds/lib/python2.2/site-packages/PyDS/Tool.py", line 1631, =
in
logVerbose
  File "/opt/pyds/lib/python2.2/site-packages/PyDS/Tool.py", line 1322, =
in
thunk
  File "/opt/pyds/lib/python2.2/threading.py", line 396, in run
  File "/opt/pyds/lib/python2.2/threading.py", line 408, in __bootstrap
Exception in thread aggregator:
Traceback (most recent call last):
  File "/opt/pyds/lib/python2.2/threading.py", line 408, in __bootstrap
    self.run()
  File "/opt/pyds/lib/python2.2/threading.py", line 396, in run
    apply(self.__target, self.__args, self.__kwargs)
  File "/opt/pyds/lib/python2.2/site-packages/PyDS/Tool.py", line 1322, =
in
thunk
    tool.logVerbose(_('Background process started'))
  File "/opt/pyds/lib/python2.2/site-packages/PyDS/Tool.py", line 1631, =
in
logVerbose
    getTool('events')._logVerbose(self.name, msg)
  File "/opt/pyds/lib/python2.2/site-packages/PyDS/EventsTool.py", line =
99,
in _logVerbose
    self._log(tool, message)
  File "/opt/pyds/lib/python2.2/site-packages/PyDS/EventsTool.py", line =
93,
in _log
    finally: self._release()
  File "/opt/pyds/lib/python2.2/site-packages/PyDS/Tool.py", line 1407, =
in
_release
    return self.lock.release()
  File "/opt/pyds/lib/python2.2/site-packages/PyDS/DorothyLocker.py", =
line
363, in release
    self.__sanity_check('release', my_context)
  File "/opt/pyds/lib/python2.2/site-packages/PyDS/DorothyLocker.py", =
line
290, in __sanity_check
    raise ReleaseOfUnAcquiredLock, (None, my_context)
ReleaseOfUnAcquiredLock: (None, <PyDS.DorothyLocker.CallingContext =
object at
0x8927614>, 'An attempt to release an un-acquired lock. Probably caused =
by
failure to acquire.')

bye, Georg

-----Urspr=FCngliche Nachricht-----
Von: Garth T Kidd [mailto:garth-OnzZ1s1DREKDegMON/[email protected]]
Gesendet: Dienstag, 28. September 2004 22:29
An: 'Bauer, Georg'; 'Thomas Klaeger'
Cc: 'PyDS Developer List (E-Mail)'
Betreff: RE: [Pyds-dev] DorothyLocker and performance


cvs diff -c PyDS\DorothyLocker.py > 20040928-2222-dorothyagain.diff

I've updated, backed out the conflict damage (CVS deciding to replace =
whole
sections of identical code when only __revision__ had changed), and
re-generated. Attached.=20

I'm beginning to like unit tests. It was quite reassuring to run the =
module
under 2.2 and see a line of dots followed by OK. I'd otherwise have =
spent a
lot of time stressing that I had some *other* 2.2 compatibility issues
somewhere. I can't run PyDS up wholesale under 2.2 because all my third
party libraries are loaded for 2.3...=20

-----Original Message-----
From: pyds-dev-admin-iYtK5bfT9M//Ad8WF/[email protected]
[mailto:pyds-dev-admin-iYtK5bfT9M//Ad8WF/[email protected]] On Behalf Of Bauer, Georg
Sent: Wednesday, 29 September 2004 1:02 AM
To: 'Garth T Kidd'; 'Thomas Klaeger'
Cc: 'PyDS Developer List (E-Mail)'
Subject: AW: [Pyds-dev] DorothyLocker and performance

Hi!

Sorry, but that's not a patch against the current CVS. That patch can't =
be
merged because it contains stuff that was already applied or something =
like
that:

bash-2.05a$ patch --dry-run -p0 <20040928-2222-dorothyagain.diff =
patching
file PyDS/DorothyLocker.py Reversed (or previously applied) patch =
detected!
Assume -R? [n] n Apply anyway? [n] y Hunk #1 FAILED at 1.
Hunk #2 FAILED at 37.
2 out of 2 hunks FAILED -- saving rejects to file =
PyDS/DorothyLocker.py.rej
bash-2.05a$ patch --dry-run -p0 <20040928-2222-dorothyagain.diff =
patching
file PyDS/DorothyLocker.py Reversed (or previously applied) patch =
detected!
Assume -R? [n] y Hunk #2 FAILED at 21.
1 out of 2 hunks FAILED -- saving rejects to file =
PyDS/DorothyLocker.py.rej

bye, Georg

-----Urspr=FCngliche Nachricht-----
Von: Garth T Kidd [mailto:garth-OnzZ1s1DREKDegMON/[email protected]]
Gesendet: Dienstag, 28. September 2004 14:23
An: 'Bauer, Georg'; 'Thomas Klaeger'
Cc: 'PyDS Developer List (E-Mail)'
Betreff: RE: [Pyds-dev] DorothyLocker and performance


That's a weird one.=20

Oh, hang on.

E.E..E.........E.E....E

Yep. Python 2.2 works differently to Python 2.3. Here's a fix.=20

-----Original Message-----
From: pyds-dev-admin-iYtK5bfT9M//Ad8WF/[email protected]
[mailto:pyds-dev-admin-iYtK5bfT9M//Ad8WF/[email protected]] On Behalf Of Bauer, Georg
Sent: Tuesday, 28 September 2004 5:24 PM
To: 'Garth T Kidd'; Bauer, Georg; 'Thomas Klaeger'
Cc: PyDS Developer List (E-Mail)
Subject: AW: [Pyds-dev] DorothyLocker and performance

Hi!

>Oh, did I forget to send that one in? Sorry. :| I had that, =
alternative=20
>logging for those post-shutdown logs, and code to make sure EventsTool =

>was last.

Your patch wasn't able to be applied - I wrote you this directly and
off-list :-)

My solution is a bit more flexible, though: you now have a flag to tell
where to sort the tool on shutdown. I use that to move the =
PreferencesTool
directly before EventsTool so that the preferences can be still fetched
until the very last moment in shutdown.

>You're calling PyDS.Tool.getToolsForShutdown twice, just like you used =

>to getTools. Is there anything wrong with saving the value and using =
it
twice?


Changed that, although it really doesn't matter much - the shutdown is =
only
called once in a running instance ;-)

>Could you drop in this latest version of DorothyLocker? You're now a=20
>fair way behind, and I've lost track of when I fixed certain bugs. =
That=20
>might

Did that. But it would really be a bit better if you stick to CVS =
mechanisms
- just do an update and merge in the changes I commited and then =
reproduce
your patch with regard to current CVS. It's much more likely that I put =
in a
working patch than full module sources, as the latter would require me =
to
make the diff myself to see what changed. And I am known to be lazy ;-)

For example this time: after adding the full module I could'nt start =
PyDS,
because of some changes to the interface of DorothyLocker that required
additional patches. But your patch didn't work, as your full module =
already
was installed. Mixing different patch/dist methods allways makes things =
more
complicated.

Oh, and now the CVS is broken. One thing was a small typo in Tool.py =
(wrong
indention) that was easily fixed. But now it barfs about "sequence must =
be
integer" on start. What's that? It happens in your __getitem__ method =
in
your FramingContext where it accesses the framestack (line 103). Looks =
like
ltrim_common doesn't produce integers as index somehow (maybe pos not
initialized?). This happens directly at startup of PyDS:

Exception exceptions.TypeError: sequence index must be integer
('/opt/pyds/lib/python2.2/site-packages/PyDS/Tool.py', 1407, =
'_release',
'return self.lock.release()')
('/opt/pyds/lib/python2.2/site-packages/PyDS/DorothyLocker.py', 359,
'release', "self.__sanity_check('release', my_context)")
('/opt/pyds/lib/python2.2/site-packages/PyDS/DorothyLocker.py', 260,
'__sanity_check', 'lock_uniques, my_uniques =3D =
ltrim_common(lock_context,
my_context)')
('/opt/pyds/lib/python2.2/site-packages/PyDS/DorothyLocker.py', 54,
'ltrim_common', 'return oblist1[pos:], oblist2[pos:]')
('/opt/pyds/lib/python2.2/site-packages/PyDS/DorothyLocker.py', 103,
'__getitem__', 'return self.framestack[index]') Traceback (most recent =
call
last):
   File "/opt/pyds/bin/pyds-start", line 51, in ?
     PyDS.Server.start_server(daemon=3D_PyDS.daemon)
   File "/opt/pyds/lib/python2.2/site-packages/PyDS/Server.py", line =
298, in
start_server
     PyDS.Tool.initTools(PyDS.Tool.mainstatus)
   File "/opt/pyds/lib/python2.2/site-packages/PyDS/Tool.py", line 317, =
in
initTools
     tool._initdb()
   File "/opt/pyds/lib/python2.2/site-packages/PyDS/WeblogTool.py", =
line 77,
in _initdb
     finally: self._release()
   File "/opt/pyds/lib/python2.2/site-packages/PyDS/Tool.py", line =
1407, in
_release
     return self.lock.release()
   File "/opt/pyds/lib/python2.2/site-packages/PyDS/DorothyLocker.py",
line 359, in release
     self.__sanity_check('release', my_context)
   File "/opt/pyds/lib/python2.2/site-packages/PyDS/DorothyLocker.py",
line 260, in __sanity_check
     lock_uniques, my_uniques =3D ltrim_common(lock_context, =
my_context)
   File "/opt/pyds/lib/python2.2/site-packages/PyDS/DorothyLocker.py",
line 54, in ltrim_common
     return oblist1[pos:], oblist2[pos:]
   File "/opt/pyds/lib/python2.2/site-packages/PyDS/DorothyLocker.py",
line 103, in __getitem__
     return self.framestack[index]
TypeError: sequence index must be integer


>also be behind some of the performance hits you're seeing (I used the=20
>latest, assuming you'd incorporated it after that submission I made=20
>that
got
>blocked for size).

I usually write about what I commited (and at least it's in the CVS RSS
feed). So if you don't see me mentioning it, it supposedly isn't in =
CVS. I
either wasn't able to apply it (then I usually write something) or I =
just
plain missed it.

bye, Georg


_______________________________________________
Pyds-dev mailing list
Pyds-dev-iYtK5bfT9M//Ad8WF/[email protected]
http://www.westfalen.de/cgi-bin/mailman/listinfo/pyds-dev


_______________________________________________
Pyds-dev mailing list
Pyds-dev-iYtK5bfT9M//Ad8WF/[email protected]
http://www.westfalen.de/cgi-bin/mailman/listinfo/pyds-dev