Re: Lucene Excerpt Highlighting

Hi Michael,

>> When I use the lucene-tag-name and lucene-class-value META tags,
>> the indexing works correctly (only the text within the tag is
>> indexed). But if the search term appears in the excerpt before
>> the indexed text, it is highlighted at this position.
>
> Well, I guess a  simple solution would be to dump (during indexing) the
> text which is actually being indexed
> into another directory, e.g.
>
> Crawling (dump ...)
>
> Original document:    ..../htdocs_dump/employees/andreas.html
>
> Indexing (dump once more ...)
>
> Indexed text for creating the excerpt: ..../excerpts/employees/andreas.html
>
> and then  you have to specify this "excerpts" directory within your
> sitemap rather than the "htdocs_dump" directory

This sounds good and should not be too hard to implement.

>
> One remark:
> The "problem" exists because on the Lucene mailing list they are
> recommending that
> you should NOT add your indexed text to the index as a field, because
> the index is becoming to large
> which is going to slow down the searching. Maybe it would help to do
> some numbers: how slow does it become
> with regard to how many number of characters and documents being
> indexed, etc.
>
> I guess it would be nice if the "Lenya indexer" could be configured such
> that it can do either or.
> (And the searcher as well of course).
>
> What do you think?

We have a high rating for industrial strength on our roadmap,
so we should definitely take care of search speed. It should be
quite easy to add this feature.

Regards,
Andreas