Re: filestat - a different perspective

Will and everyone,

Over the last nine months I have been toying with various blogging 
systems and I keep coming back to pyblosxom. I like the information 
architecture of blosxom sites, and python is my preferred language. I 
also like the portability of blosxom-style sites, both in moving 
between servers, and between blosxom implementations. The basic file 
format and is plain text, and we can easily move between various 
blosxom implementations.

To me, blosxom-style sites have two weakness. First is the lack of 
storing of meta data, especially permanent timestamps, but also 
information like authorship, draft/publish status, keywords, etc. 
Secondly, in the lack of good web based editing tools. However, 
directly adding meta data and web based editing will cause pyblosxom to 
diverge too much from the original blosxom, and there are complex 
issues that affect performance and make caching more difficult.

I know that we are also facing the fact that the core group of 
pyblosxom programmers do not have the time to invest in significant 
changes to pyblosxom. Yet these are the people who know the current 
system well enough to deal with the complex performance issues that 
meta data will add.


So what can we do?


Here are my thoughts: We should _NOT_ solve these issues in pyblosxom. 
I say that we leave pyblosxom alone, and create a NewProject. (Replace 
with clever name)

NewProject would do the following:

1) Maintain a separate directory structure of all blog entries. These 
blog files will be stored in email format with meta data as headers.  
(I believe this is called RFC 2822 format?).

2) NewProject would be responsible for syncing its directory structure 
with the one used by pyblosxom. At this point the timestamp of the file 
used by pyblosxom would be set according to the meta data.

3) The syncing would convert the  RFC 2822 format to standard blosxom. 
We could also preform other processor intensive rendering. I currently 
do this for rendering of MarkDown formating and my glossary 
substitutions.

4) NewProject would handle blog entry editing and creation via variety 
of mechanisms such as: XML-RPC, webforms, POP3, IMAP, command line, 
etc.

5) Deal with user authentication, draft/publish status, etc.

I have actually implemented parts of each of the above points, but I 
feel that my code is far from acceptable for public consumption and is 
highly incomplete. I hope to work more on this in the middle of 
September.

Other ways of looking at the relationship between Pyblosxom and 
NewProject:

Pyblosxom focuses on Content Serving
NewProject focuses on Content Management.

Pyblosxom is compatible with blosxom-style blog systems.
NewProject _ADDS_ features to _ANY_ plain text file blosxom-style system

Pyblosxom remains a "simple" blog system
NewProject adds features and complexity for people who want it.


-- 
Lee Joramo
(970) 261-8426            www.joramo.com
959 Ouray Ave.            [email protected]
Grand Junction, Colorado, 81501



On Aug 18, 2004, at 4:49 PM, will guaraldi wrote:

>
> I was looking into whether we could adjust the filestat callback so 
> that
> it took as arguments the request, the filename, the mtime of the 
> previous
> filestat call, and also the entry object.  That way you could 
> implement a
> plugin that skimmed the entry object data and populated the mtime from
> data contained therein.  For example, if you had a line in the entry 
> file:
>
> TIMESTAMP::200408161614
>
> the plugin could take that line and extract the date from it and send 
> that
> as the mtime.
>
> I see a few complications with this.  First off, we have to change all
> uses of the filestat callback so that they retrieve a list of entries
> rather than just the list of files and pass those entries into the
> filestat callback.  I'm pretty sure I can solve this without further
> complications or changing the existing API.
>
> The second issue is more complicated: with the current architecture we 
> may
> not open the file at all.  If you're using entry caching, the thing 
> that's
> being cached is the parsed file data after entry parsing, not the raw 
> file
> itself.
>
> If you're using entry caching, you don't get the raw file, you get the
> cooked file after it's gone through tne entry parser.
>
> That just seems messy.  I certainly don't want to build a matrix of 
> what
> happens for what configuration.  It's too difficult to 
> explain/document.
>
>
> I have a couple of solutions for those of you who want to store mtimes 
> in
> the blog entry files:
>
> 1. We move cache handling from the entry objects to the entry parsers.
> The text parser, for example, has no need of caching.  The moinmoin 
> parser
> does--because it takes a bunch of time to do its thing.  The entry 
> parser
> gets the request object and the filename--that's pretty much all it 
> needs
> to figure out whether something is in the cache or not.  This has the
> added nicety of the default blosxom behavior being easier to deal with.
>
> 2. Those of you building plugins that pull the mtime from the contents 
> of
> the blog entry take the filename from teh filestat call, open the file,
> read it in, pull out the line you need, and return that.  You could 
> make
> this a little easier by caching the mtime in a file.  Or additionally
> build an index of files and their mtimes via cron.
>
> 3. Some third option I haven't thought of.  (I'm all ears if other 
> people
> have other ideas.)
>
>
> I'm opting for the first one since it cleans up some stuff I dislike 
> (and
> wrote in the first place) and it's cleaner for folks implementing
> interesting mtime retrieval plugins.  Additionally, it won't affect the
> behavior or API except for people who are using caching and 
> entryparsers
> that haven't been modified to do the cache thing.
>
> Any thoughts?
>
> /will



-------------------------------------------------------
SF.Net email is sponsored by Shop4tech.com-Lowest price on Blank Media
100pk Sonic DVD-R 4x for only $29 -100pk Sonic DVD+R for only $33
Save 50% off Retail on Ink & Toner - Free Shipping and Free Gift.
http://www.shop4tech.com/z/Inkjet_Cartridges/9_108_r285