Re: Some questions about file handling + some other things

Dear John,
 
> >4. I do not really understand how indexing works yet, and
> >the documentation is a bit terse on that topic. I just vaguely looked
> >into the code, but it seems that
> >The content of files (file, msg, etc.) are indexed (?), but only if
> >their type is text/plain (?). This indexing can (?) be disabled by
> >explicitly setting
> >
> >file = FileClass(db, "file",
> >                name=String(),
> >                content=String(indexme='no'))
> >
> >in schema.py. Btw shall I install Xapian, or not? :)  
> 
> IIRC any String is indexed. So title fields, your memo field above
> etc. should all be indexed. It also strips words < 1 character or
> words > 25 characters I guess it might be a problem for German 8-).
> 
> You can always add an indexer later and run reindex. (See
> upgrading.doc for an example). By default it uses native indexing
> based on inserting tokenized words into tables in the backend
> database. I don't think we use the text search capabilities of either
> sqlite (FTS4/5) or postgres (GIN index).
> 
> If you need a porter stemmer, xapian is set up to that IIRC. Woosh is
> supposed to do "Fuzzy" matching. I don't remember exactly what the
> rules are there. The test suite just does the most basic tests of the
> indexers. So Porter stemming, fuzzy matching etc aren't tested.

I don't know whether I need external text indexer or not. (I expect
more than 5000 issues, but text search will be used only for a few
String() properties, like "customer_name", "customer_address" etc. But
who knows...) I think I will just go with the default indexer first,
and may install it later, if needed.

Btw, if I want to set up Xapian for Hungarian language, then it is
enough to change the 

stemmer = xapian.Stem("english")

occurrences to

stemmer = xapian.Stem("hungarian")

in roundup/backends/indexer_xapian.py, or do I have to do anything
else? (To be honest, I do not know how these indexing algorithms work.)

Regards,
Gábor


_______________________________________________
Roundup-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/roundup-users