RE: Just a short status on CVS

Hi!

> Are you saying that MetaKit doesn't like opening and closing files in 
> quick succession, or that you still have multiple threads hitting 
> MetaKit at the same time, causing errors? (I think it's the first but 
> it's hard to tell.)

Yep, it's the first. Ok, there is the slight possibility that multiple opens
take place, but I am quite sure that is not the case. I suspect there are
some problems with construction/destruction of Metakit objects - sometimes
objects seem not to be constructed right, or destructed before they actually
fall out of scope. At least that's what it looks like from the crash dump -
it's allways some calls that try to access an element in the result that's
not there any more and so segfaults. If the Python wrapper kicks in, access
beyond boundaries is mapped to Python Exceptions, so I suspect the internal
problem.

> * A User object was created to encapsulate all User-specific 
> data, such 
> that changing the User object would be exactly the same as using PyDS 
> for another user.

This is already done, it's the PyDSProcessor class. One instance for each
context (in PyDS that's one for main.server and one for user.default). The
PyDSProcessor instance is where the thread for handling requests is hooked
up to.

> * A thread was created for this User that mediated all access 
> to Metakit 

That's something I was thinking about already, but I dislike the performance
hit this would produce. Ok, passing stuff in is not the problem, as you can
use Condition variables to trigger database functionality and push/pull
parameters via a sync queue. But to fetch the results from a result queue is
quite heavy: Metakit uses Metakit views for results, not a special result
object. Those views are Metakit library objects, not pure pythonic stuff. So
to "cut" the connection to Metakit, one would have to repackage all data -
otherwise you still would use the Metakit view objects (and so the metakit
library) from all threads that get data from the database.

What it would boil down to would be building a usefull abstraction for data
storage in Metakit with a Pythonic wrapper around that so that you don't
shuffle Metakit library stuff around. This _might_ solve the problem - at
least PyDS was very stable when it still stayed initialized all the time.

But I think throwing out just the dynamicism would be much more sensible to
do. It's main effect for PyDS - reducing resource usage - can be reached by
other means, too (for example the dynamic thread model from TooFPy). And
it's problems would go away when the problematic code is thrown out.

> (I'm assuming/hoping MetaKit only wants one thread to be working on a 
> file, and not that MetaKit only wants to run in one thread 
> period. (Tk 
> works this way.))

Oh, Metakit is quite fine with multiple threads - at least the Python
wrapper is fine with that. Native metakit actually doesn't like threads
much, but the Python wrapper provides enough atomicity to prevent damage.
PyDS just uses locks to block concurrent write access, but that's mostly to
ensure data integrity. It would actually work to do even writes in different
threads - technically.

> Come to think of it, I could build a metaclass that implements 
> token-based access to the meta-kit too, giving a token to the thread 
> currently allowed to access the database and making all other threads 
> wait until its done, again without making either the 

Actually that's what I am doing with the database locks in PyDS. Although
there are locks on level of tools, as each tool has it's own database file.
So write access to different files (=tools) can happen freely.

> The meta-lesson is that as long as the user-specific data is 
> encapsulated behind some new-style class, you've got some really 
> powerful options on how to manage access to objects of this class. ;-)

That's something that's a bit weird in PyDS since it started. I never
implemented a layer between tools and their databases, as the tool itself is
the abstraction you work with. There just wasn't the need to abstract the
database away, as it's only used locally in each tool - all other tools
allways go through exported interfaces of a tool. So there is just the
StandardTool (a classic class, but that could be changed to a new-style
class) derived class as a wrapper around the metakit database, since the
metakit database itself uses types, not classes. Ok, that's with Python 2.2,
I can't say much about Python 2.3.

bye, Georg