Re: importing entries

On Apr 7, 2005 1:19 PM, will guaraldi <[email protected]> wrote:
> On Thu, 7 Apr 2005, Bill Mill wrote:
> >
> >> Then, I'd split on _, and take out "stop words" like "and", "but",
> >> "then", "the", ...  Then join on _ again.
> >
> > perhaps. Do other people like this? I'm unsure on it.
> 
> I was thinking this would make it easier for people to recognize blog
> entries.  It just popped in my head when I was writing the email.
> 
> 
> >> Also, in preparation of us fixing the problem we have with mtime and
> >> metadata, I'd toss an mtime into the entry in the metadata.  Call it
> >> something like "creationdate" or something like that.  Maybe toss other
> >> things in the metadata as well.  Can't hurt.  Maybe ask the user what
> >> they want in the metadata?
> >
> > In what metadata format? Mine (<!-- key: value1, value2 -->) or somebody
> > else's? I call my datetime key "date" (clever, I know). I could make
> > command-line options to embed values in the metadata, if that's desired.
> 
> I'm not sure I'm following this.  The entry format we support allows for
> metadata items on lines between the title and the body.
> 
>     http://pyblosxom.sourceforge.net/1.2/manual/x566.html
> 
> I didn't realize you had your own metadata format.  The one I'm describing
> above has been supported by PyBlosxom for over a year now though it has
> only very recently been officially documented as far as I know.

Holy cow, why didn't anybody tell me about this? The only code in
pyblosxom I haven't read is in the entryparser, and now I really wish
I had. I wrote my own plugin for this because I didn't know it
existed. wow.

> 
> Also, I'd stay away from metadata names that might conflict with existing
> entry variables.
> 

Well, I wrote my own metadata parser which puts them into a dictionary
of their own, so they can't conflict - they also can't be referenced
directly from the entry (it's *meta* data).

> 
> > Are we planning to stop using mtime as a sorting mechanism?  How are we
> > "preparing to fix the problem we have with mtime and metadata"?
> 
> The problem we have with mtime and metadata is that metadata doesn't get
> absorbed until long after we've:
> 
>    1. sorted the entries
>    2. sliced the list of entries down to num_entries size
> 
> Since we sort on mtime and the only things that mtime can be based on are
> the file name or the file mtime, we're kind of hosed.

Yup. I override cb_filelist to get custom time sorting.

> 
> This would get fixed if we were looking at fully parsed entries (i.e. the
> file has been read and parsed out) before we sorted and sliced the list.
> Then we could allow metadata to override the file mtime.

Or we could just parse the metadata (not the whole story, it'd be very
fast, esp since the metadata is right at the top of the file) look for
a specific key, and use it to sort with if it exists.

> 
> The problem with that is that it forces us to read and parse _all_ the
> entries when we go to get all the entries.  We have all this code in the
> entry class and elsewhere that tries to reduce the parsing to the entries
> that we're sure are going to be displayed.
> 

Or, you do like I do, and cache the metadata for each entry so you
don't have to reread it every time.

This is why I added the setTimeLazy method to the FileEntry object -
so I can put in the cached time without parsing the whole entry. This
way, sorting works properly and fast.

> It's messy.
> 
> So the answers to your questions are these:
> 
> 1. Are we going to stop using mtime to sort the entries?  Yes and no.  The
> default sorting will still use mtime, but the filestat callback needs to
> have more information available to let us determine the mtime of the file
> from metadata.  Ultimately, I'd like to create a sort callback as well,
> but haven't been able to do it because all we have at that point in the
> code are filenames.
> 

I think you see now why we were on different pages.

> 2. How are we going to do it?  It's something I want to fix in the next
> version and it's wrapped up in the storage api changes I want to make but
> haven't prototyped or specified yet.  So for now, it's all under the same
> mystery umbrella until I get some time to wrap my arms around it.
> 

I say cache the metadata so the entry can still be lazily parsed. But
this is turning into a different topic...

Peace
Bill Mill
bill.mill at gmail.com


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click