Issue #SCB2413 In progress

Activity report on

  *Defect SCB2413 - lucene RC in scarab works wrong. Need update*
 
  Scarab Link: http://www.solitone.org/scarab/issues/id/SCB2413
  Module: Scarab


  Activity generated by  Ronny Voelker ([email protected])  at 12/02/2007 22:46

  *Reasons for the changes*
  - In pogress


  *Personnel changes*
  - User "ronvoe122" assigned as "Assigned to"

-------------------------------------------------------------------------------
  *Attribute changes*
  - "Status" changed from "Unconfirmed" to "In progress"

-------------------------------------------------------------------------------
  *Comments*
  - By Ronny Voelker - 12/02/2007 22:46 ---
  "I too encountered some complaints lately about the full text search - in mainly German text.

I decided to upgrade to the current Lucene2.2 instead of 1.4 because:
* Lucene2.2 is more mature and has better support than Lucene1.2 or 1.4
* the API hasn't changed very much, so upgrading is easy
* there were some suggestions for enhancements (like regexp-search) which rely on features,
  which are only available in younger versions of Lucene

The most effective way to improve the search result is probably to use an Analyzer optimized for the language of the searched text.
The default analyzer is heavily optimized for English.
So I added a new configuration-property (searchIndex.analyzerClass) to allow to change the used Analyzer without customizing the source code.

Besides that I did some additional improvements:
* Updating the search index can now be canceled and has a progress indicator.
* the indexing  and searching is faster:
    * LuceneSearchIndex keeps one writer open all the time, 
      instead of closing and reopening it for each indexing operation.
    * lesser Documents - a Lucene-Document contains now a whole issue 
      instead of a single attribute value or comment
    * shifted more work from Scarab to Lucene in case of complex Queries
      a Scarab-query over multiple attributes is now executed as one Lucene-query, 
      instead of executing one Lucene-query per attribute 
      and later merging the results manually
    * the query-api of LuceneSearchIndex is now stateless,
      so multiple queries can be executed in parallel
* removed code which was probably there to to work around 
  the limitations of a very early version of Lucene

I have tested the new LuceneSearchIndex with 5000 issues containing mostly German text and should be able to commit it in the next few days.

Any suggestions or objections?"