Re: Some questions about file handling + some other things

Hi John,

I hope you feel better.

> Hi Nagy:
> 
> In message <20210325154202.5e83e67c@Dell>,
> Nagy Gabor writes:
> >> >4. I do not really understand how indexing works yet, and
> >> >the documentation is a bit terse on that topic. I just vaguely
> >> >looked into the code, but it seems that
> >> >The content of files (file, msg, etc.) are indexed (?), but only
> >> >if their type is text/plain (?). This indexing can (?) be
> >> >disabled by explicitly setting
> >> >
> >> >file = FileClass(db, "file",
> >> >                name=String(),
> >> >                content=String(indexme='no'))
> >> >
> >> >in schema.py. Btw shall I install Xapian, or not? :)    
> >> [...]
> >> You can always add an indexer later and run reindex. (See
> >> upgrading.doc for an example). By default it uses native indexing
> >> based on inserting tokenized words into tables in the backend
> >> database. I don't think we use the text search capabilities of
> >> either sqlite (FTS4/5) or postgres (GIN index).
> >> 
> >> If you need a porter stemmer, xapian is set up to that IIRC. Woosh
> >> is supposed to do "Fuzzy" matching. I don't remember exactly what
> >> the rules are there. The test suite just does the most basic tests
> >> of the indexers. So Porter stemming, fuzzy matching etc aren't
> >> tested.  
> >
> >I don't know whether I need external text indexer or not. (I expect
> >more than 5000 issues, but text search will be used only for a few
> >String() properties, like "customer_name", "customer_address" etc.
> >But who knows...) I think I will just go with the default indexer
> >first, and may install it later, if needed.
> >
> >Btw, if I want to set up Xapian for Hungarian language, then it is
> >enough to change the 
> >
> >stemmer = xapian.Stem("english")
> >
> >occurrences to
> >
> >stemmer = xapian.Stem("hungarian")
> >
> >in roundup/backends/indexer_xapian.py, or do I have to do anything
> >else? (To be honest, I do not know how these indexing algorithms
> >work.)  
> 
> Yes, there are two places where you have to change that. One for
> adding terms and one for finding terms.
> 
> You will not have to do this with the next release of roundup (or if
> you run the current development trunk). In revision 9d209d2b34ae I
> pulled that value into the config.ini file. Default is english.
> 
> The change needs better error detection. A bad language value results
> in an error in the web interface and I guess in the email interface as
> well. But at least the language is tunable on a per tracker basis.
> 
> Have a great week.

OK, thank you. I think I will use the native indexer until other
language support is officially implemented. I guess the fact that I can
reindex the db later, this also means that I can change the 'indexme'
arguments in schema.py, if I want (and perform a reindexing).

Finally, I note that indexing of strings is turned off by default.
See hyperdb.py:

class String(_Type):
    """An object designating a String property."""
    def __init__(self, indexme='no', required=False, default_value="",
                 quiet=False):

However, it is indexme='yes' is set by default for the Title, and the
content property of FileClasses.

Regards,
Gábor

> --
> 				-- rouilj
> John Rouillard
> ===========================================================================
> My employers don't acknowledge my existence much less my opinions.
> 
> 
> _______________________________________________
> Roundup-users mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/roundup-users


_______________________________________________
Roundup-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/roundup-users