Re: Some questions about file handling + some other things

Hi Nagy:

In message <20210325154202.5e83e67c@Dell>,
Nagy Gabor writes:
>> >4. I do not really understand how indexing works yet, and
>> >the documentation is a bit terse on that topic. I just vaguely looked
>> >into the code, but it seems that
>> >The content of files (file, msg, etc.) are indexed (?), but only if
>> >their type is text/plain (?). This indexing can (?) be disabled by
>> >explicitly setting
>> >
>> >file = FileClass(db, "file",
>> >                name=String(),
>> >                content=String(indexme='no'))
>> >
>> >in schema.py. Btw shall I install Xapian, or not? :)  
>> [...]
>> You can always add an indexer later and run reindex. (See
>> upgrading.doc for an example). By default it uses native indexing
>> based on inserting tokenized words into tables in the backend
>> database. I don't think we use the text search capabilities of either
>> sqlite (FTS4/5) or postgres (GIN index).
>> 
>> If you need a porter stemmer, xapian is set up to that IIRC. Woosh is
>> supposed to do "Fuzzy" matching. I don't remember exactly what the
>> rules are there. The test suite just does the most basic tests of the
>> indexers. So Porter stemming, fuzzy matching etc aren't tested.
>
>I don't know whether I need external text indexer or not. (I expect
>more than 5000 issues, but text search will be used only for a few
>String() properties, like "customer_name", "customer_address" etc. But
>who knows...) I think I will just go with the default indexer first,
>and may install it later, if needed.
>
>Btw, if I want to set up Xapian for Hungarian language, then it is
>enough to change the 
>
>stemmer = xapian.Stem("english")
>
>occurrences to
>
>stemmer = xapian.Stem("hungarian")
>
>in roundup/backends/indexer_xapian.py, or do I have to do anything
>else? (To be honest, I do not know how these indexing algorithms work.)

Yes, there are two places where you have to change that. One for
adding terms and one for finding terms.

You will not have to do this with the next release of roundup (or if
you run the current development trunk). In revision 9d209d2b34ae I
pulled that value into the config.ini file. Default is english.

The change needs better error detection. A bad language value results
in an error in the web interface and I guess in the email interface as
well. But at least the language is tunable on a per tracker basis.

Have a great week.
--
				-- rouilj
John Rouillard
===========================================================================
My employers don't acknowledge my existence much less my opinions.