Re: Just a short status on CVS

Bauer, Georg wrote:
> The main problem _now_ is that PyDS segfaults after some time, or just
> behaves very weird (and segfaults later on). I added libwadpy (a handler for
> segfaults and other system signals that can give you a traceback much in the
> same way as Python can do for Python code) support to PyDS to find out what
> produces the segfault and it's allways the same place in metakit.

I'm trying to follow this and as you know I've only been poking around 
the source for a little while ;-), but the structure of the code is 
strong so I think I understand the architecture.

Are you saying that MetaKit doesn't like opening and closing files in 
quick succession, or that you still have multiple threads hitting 
MetaKit at the same time, causing errors? (I think it's the first but 
it's hard to tell.)

Would it solve the problem if:

* A User object was created to encapsulate all User-specific data, such 
that changing the User object would be exactly the same as using PyDS 
for another user.

* A thread was created for this User that mediated all access to Metakit 
(and indeed, to the user), and did nothing else, so the MetaKit file for 
that User was guarenteed to never be accessed from any other thread. 
(I'm assuming/hoping MetaKit only wants one thread to be working on a 
file, and not that MetaKit only wants to run in one thread period. (Tk 
works this way.))

If so, I've got a trick up my sleeve that I developed for a now-defunct 
aspect of Iron Lute that I never actually used, but may work here. It's 
a meta-class (aieee!) that turns an object into it's own thread; you 
write the class natually and use this metaclass on it and the metaclass 
automatically moves the method calls to the object across thread 
boundaries, executes the code, and moves the return (either returned 
value or raised exception) across the thread boundary again. The thread 
transition is transparent to *both* the class and callers.

(I was using this for Tk thread protection, but it didn't quite work 
since I wanted all Tk classes in the same thread, which IIRC never quite 
worked right. Then, later Tkinter implemented it for me anyhow. It 
worked peachy for one object per thread though.)

You can tie whether the database is opened to reference counting on this 
object, with either manual or automatic reference counting. (Both have 
advantages and disadvantages which can be discussed later; it would bulk 
this message up too far.)

I'd have to reconstruct the meta-class, I don't think I have a copy of 
it around anymore. But it's actually fairly easy to write.

The biggest potential downside is that I have no idea what this does to 
performance under heavy use; I never got that far. I do know there's no 
copying of values across the thread boundary, which would *really* hurt, 
but I don't know what the thread switching does to performance.

Come to think of it, I could build a metaclass that implements 
token-based access to the meta-kit too, giving a token to the thread 
currently allowed to access the database and making all other threads 
wait until its done, again without making either the accessing class or 
the User class unneccessarily complicated. This would probably be loads 
more efficient, since it would not switch threads for a function call... 
cool, wish I'd thought of this last time!

Normally I'd edit this message down to remove the thread suggestion, but 
I'm not sure how MetaKit would feel about being accessed from multiple 
threads, even if I guarentee the requests never overlap. So I'll leave 
them both in in case they give somebody else ideas.

If any of this sounds useful, let me know and I can give you a prototype 
of the relevant metaclass (thread-based or token-based access to a 
class) if you'd like. (I *think* the token solution will be superior, as 
long as MetaKit can handle it.) A little experimentation will be 
necessary either way.

The meta-lesson is that as long as the user-specific data is 
encapsulated behind some new-style class, you've got some really 
powerful options on how to manage access to objects of this class. ;-)

If this is way off base, sorry, I'm trying. ;-)