RE: Just a short status on CVS

Hi!

> There is a new release of metakit as of Jan, 26/2004 claiming to fix
> numerous bugs, etc. Have you had a chance to review it?

Not yet. I will look into that, of course.

> As for multiuser.. isn't PyDS was designed to build a queue 
> of database
> write requests which are released individually, so at any 
> point in time,
> there is only one db writer. Are you changing this? If so, to 
> what purpose?

PyDS itself won't change much - only internally. The idea was to restructure
PyDS in a way that you have different exection contexts to execute tools in.
A normal PyDS would just have main.server for global tools and user.default
for all the users tools. The user.* contexts could be dynamically
instantiated or destroyed. So as soon as you access a user tool, the
user.default context will start up and be available. As soon as you don't do
any stuff with it any more, it will close down (after a time of inactivity,
not immediately). The main reason for this was to be lower on resources, as
PyDS already has lot's of threads running and most of them aren't needed all
the time - so why not just stop them when they are not needed?

> I would have thought that the community server, pycs, would 
> have choked on
> multiple db writes because it would face many more requests than pyds.

PyCS is only single threaded. So if you access something, that request needs
to be fullfilled before any other request can be handled. PyDS itself was
structured in very much the same way (all web access was handled by one
thread), only that tools had background threads so they could push stuff
into the background. The CVS version now has one request handling thread per
user context - this would enable in the future to run several PyDS instances
in one server framework and starting up and stopping user contexts
dynamically. The TooFPy project has yet another different threading model:
it's much like Apache 1.3 - starting and stopping worker threads as needed.
The worker threads are not specialized like the threads in PyDS (after
hacking PyDS for over a year I think specialized threads wasn't a really
good idea anyway ;-) ) but can handle all kinds of requests (SOAP, XMLRPC,
REST-style and internal asynchronous calls). It looks like a very usefull
threading model, so maybe I will move PyDS over to that infrastructure some
day. It's very low on resources, and that was what I wanted in the first
place, so maybe it might be the solution.

The main problem _now_ is that PyDS segfaults after some time, or just
behaves very weird (and segfaults later on). I added libwadpy (a handler for
segfaults and other system signals that can give you a traceback much in the
same way as Python can do for Python code) support to PyDS to find out what
produces the segfault and it's allways the same place in metakit.

It's not a big deal and not a big showstopper, as most of the restructuring
was needed anyway (to better encapsulate system state), so throwing out the
dynamic loading/unloading won't be much work. It's just that CVS at the
moment is broken.

The idea in the long run was to build a project based on PyDS that would
allow over-the-web registration and would handle multiple users. You could
envision something like this as a Manila alike solution, only that every
user would get his very own PyDS context and would still use the same
rendering stages, only that rendering would copy stuff to the static
webspace (much like the LFSDriver does already).

The main reason for a project like this would be to provide the
functionality of PyDS (although a bit limited - some areas need to be
restricted out of security reasons) in a hosted service. This would give
much of the power of PyDS to people who are not able or not willing to run
their own PyDS installation. But that would have been a new project, PyDS
itself wouldn't change, just share the same internal structure.

bye, Georg