Re: Patch for simple walk caching

Will,

sure there is; I don't have to worry about it. If I render my website
statically, I have to upload it to my webserver every time I change
something. Now, I could write a cron script to do that every so often
and get a similar effect, but I don't know if my laptop's on, or
online, or in windows, or ...

Furthermore, we have the possibility of making it truly dynamic with
an invalidate_cache call (or some way I haven't thought of yet) as I
started discussing with Ted. If we can get that working, we can skip
the parsing of the entries when nothing's changed, and really limit
the parsing to when a file is changed.

Peace
Bill Mill
bill.mill at gmail.com


On Tue, 7 Dec 2004 09:59:06 -0600 (CST), will guaraldi
<[email protected]> wrote:
> 
> You know, this sounds an awful lot like static rendering.  Especially
> considering I was trying to solve the exact same problems.
> 
> The difference being that static rendering caches the finally rendered
> page while your caching thing is caching the entry list.  Is there an
> advantage to caching the entry list as opposed to the finally rendered
> page?
> 
> /will
> 
> 
> 
> 
> On Tue, 7 Dec 2004, Bill Mill wrote:
> >
> > That's excellent; I'm glad it's speeding things up. I knew that I
> > still had issues with how to invalidate the cache, but I couldn't
> > really resolve them until a) I knew the cache actually speeded things
> > up in real life and b) I could talk with some other people about it.
> >
> > More comments below.
> >
> > On Tue, 7 Dec 2004 00:51:21 -0800, Ted Leung <[email protected]> wrote:
> >> Hi Bill,
> >>
> >> I tried out your cache, and it appears to make a sizable difference.
> >> I've attached 3 hotshot dumps that show the various combinations.
> >>
> >> pbls-nocache-prof.txt shows the results before I did any improvements
> >> to the blog.
> >> pbls-walk-cache-prof.txt shows the results after entryshelve caching
> >> and the walk patch that I posted.
> >> pbls-filecache.prof.txt shows the results after using your cache.
> >>
> >> You can't make much hay about the actual times because I ran these
> >> profiles on my production blog and there were other processes running
> >> at the same time, but the drop in function calls is appreciable, and
> >> the impact on response time is noticeable.
> >>
> >> The big outstanding issue is how to invalidate the cache.  This is
> >> symptomatic of one of the problems with pyblosxom's plugin
> >> architecture: it is hard for plugins that know about each other to
> >> co-operate with each other.  I would like see the filecache "pass on"
> >> the cached data that it using to the rest of the plugin pipeline, so
> >> that plugins later in the chain could invalidate the cache (presumably
> >> by deleting the appropriate pickle -- this would be a function in
> >> filelist_cache.py.   That way both the comment plugin and the
> >> xmlrpc_metaweblog plugins could invalidate the cache and the next
> >> request would cause the cache to get filled.
> >>
> >
> > What if there was a function invalidate_cache(path) - and when a
> > plugin received an update to a page, it would invalidate the cache for
> > that path. From the client (plugin) side, ideally it would look
> > something like:
> >
> > if config['cache_dir']:                        #if using cache
> >    invalidate(path_to_changed_file)   #invalidate the path
> >
> > the problem is from the cache server side - how does it figure out
> > which caches include that file? Does it open and search through all of
> > them? Can you launch a background thread from a CGI script? If you
> > can, then the user shouldn't experience too much delay, and his
> > changes should be implemented soon but not instantly. If not, he's
> > going to have to wait a while before his change is implemented.
> >
> > The other thought I have would be to maintain a listing of what files
> > are in what caches. However, it's obviously really crucial to keep
> > overhead on the cache as low as possible, or the fix becomes the
> > problem.
> >
> > Also, as Wari said, permalinks don't even benefit from caching much.
> > The question is how to determine that something is a permalink - I
> > think I'll just set the cache up to ignore filelists with only one
> > entry.
> >
> > Thanks for trying the patch out and for the ideas on how to improve
> > it; I think optimization is all kinds of fun, so I'll play with it
> > some today.
> >
> > Peace
> > Bill Mill
> > bill.mill at gmail.com
> >
> 
> --
> 
> 
> whatever it is, you can find it at http://www.bluesock.org/~willg/
> except Will--you can only see him in real life.
>


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now. 
http://productguide.itmanagersjournal.com/