Re: Patch for simple walk caching

On Mon, 6 Dec 2004, Ted Leung wrote:
>
> Are those environments actually configurations that we want to support?

It's interesting you use "we".  I've been trying to minimize the number of 
executive decisions I've been making because I'm not really sure who "we" 
is.

If the decision were up to me, I'd support Python by CGI and mod_python 
and tell everyone else they should find another blog system.  To be 
honest, I don't think we test with any other systems anyhow.

Last year sometime, I did a minor overhaul of the system which made it 
more complex but which removed global caching.  So it's definitely been a 
requirement to allow for multiple threads.

Also, a while back you mentioned intertwining PyBlosxom with Chandler--I'm 
not sure what that would entail.


> Ok, I'll keep this local for now.  But I think that we need to look a a 
> decent way of solving this problem.  If we have to have a thread safe 
> cache for mod_python or whatever, that's fine by me.
>
> My blog is up to almost 1200 entries and I've been getting complaints 
> about the slowness.  I turned on entryshelve caching which helped some, 
> and this cache patch helped a lot.  In my blog the bottleneck is now 
> tools.walk_internal, and I'm thinking about implementing a 2.3 only 
> version to see if os.walk is any better.

As near as I can see it, PyBlosxom's big issues are:

1) total inability to scale due to poor architecture in the realm of
    storage (and the huge scope of the project)

2) half-assed implementation of static rendering (for which I take full
    responsibility, but I don't use it, so I just wrote what seemed to
    make sense at the time)

3) configuration/testing/setup is difficult for newbies

4) mediocre to poor documentation


I really do want to address issue 1 in pyblosxom 1.2 and it'll involve a 
refactoring of a large chunk of how pyblosxom deals with entries.  Bill 
Mill and Robert Wall have expressed interest in this area as well.  The 
plan is to build something that allows for index caching and metadata 
caching as well as the ability to store mtimes in the file itself.  All of 
that is a total pain in the ass with the current architecture.  More 
thoughts on that at some point in the future.

I'd like to address issue 2, but I'm not qualified to do it so I'm going 
to defer until someone steps up to the plate.  At that point, I'll let 
them know everything I know about it (assuming I remember) and they can go 
off and fix the issues they see with the current implementation.  It's 
possible issue 2 is related to issue 1--I have no idea.

Issues 3 and 4 will get fixed as time goes on assuming we keep up some 
development momentum.

Anyhow, so that's the plan as far as I know it.  I'm thinking that I'm 
going to "take-over" in the full sense of the term.  If that does happen, 
I'm going to seriously reduce the scope of the project from meeting 
everyone's needs to meeting a specific set of requirements which are 
explicitly laid out on the web-site.

I'm still waffling about whether I want to put the hours in and take over, 
though.  So that's where I lie on that.

Getting there...  albeit extremely slowly right now.

/will


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now. 
http://productguide.itmanagersjournal.com/