Re: mRFC 0024: Full text indexing in Midgard

Torben Nehmer <[email protected]> wrote:

Hi,

My knowledge about indexing services is NULL , so forgive me my ignorance ( if happens ).

> > The proposed full text index shall be a Lucene index directory residing on 
> > the host that runs the Midgard applications. If there are multiple web 
> > frontend hosts accessing a shared backend database, then each frontend shall
> > have its own full text index.
> 
> What is a "web frontend" in this context? A Host?
> 
> You are saying that "Each Midgard database shall have its own full text index".
> This is a bit of a change compared to the current situation where each logical
> site (which could be driven by more then one host) has its own index. You can
> basically configure this using the indexname configuration directive (which
> defaults to hostname:port).

Something between I think. One sitegroup in one database may use 10 different hosts
where search functionality should be reusable IMO.

 
> Right now, this'll change indexed field names. The current index is not tied to
> the MgdSchemas but to the Datamanger one's, at least in all cases where indexing
> is fully automated. The main reason for this is, that DM* will allow you to
> define your own custom additional fields and have simple names for it.
> Regardless *where* the data is actually stored. The uniqueness constraints on
> the DM field name listing makes indexing them safe in this context.

The main idea of MgdSchema objects is : the same reusability on any level.
We are not going to write DM for midgard-java or midgard-python or midgard-ruby.
 
> > MultiLang content shall be handled by prefixing all the MultiLang property names
> > with the language code. A German version of a "title" property would be indexed
> > as "de:title" while the default "title" property (MultiLang zero) would be
> > indexed simply as "title". Also the general node content field will be versioned
> > by language.
> 
> Shouldn't multilang be queried implicitly?
> 
> My current integration idea would contradict this API: I planned to have a
> distinct document for each language available in the index. The framework would
> add that language field constraint automatically then.
> 
> The only (performance) problem I currently see here is the known cavet of the
> Lang 0 fallback semantics, but since I have to post-process the entire resultset
> anyway (ACLs and the like), that should not be too hard in that respect.
> 
> In all cases the queries would always be independant of the actual object
> language, you always query "title", not "lang:title". If you want to query only
> german results, you'd query for "title" and "__LANG".

I think I agree with Torben. Multilang is enough transparent to always get property
without taking much care about what language is currently set.
 
Piotras