Metadata indexing

The level of sophistication required for matching learning objects based
on their metadata has increased somewhat for version 1.1. I've been
looking at how we currently handle indexing and how we may extend the
indexing to encompass more complex needs.

First (for those not familiar with the code a little overview of the
current system.

All of the data objects in belts are expressed as java interfaces. For
most interfaces an immutable and mutable version exists (one with the
property getters the other with the property setters).

To index a object the indexer is passed the objects immutable interface
and the object instance to index. The indexer inspects the object and
finds its simple properties (for example the users given and family names)
and adds them to a lucene document that is placed in the index.

When a search is performed the index creates a proxy object that obeys the
immutable interface definition but is backed buy the information in the
lucene index.

This is working very well and is highly suited to the majority of simple
flat object searches we do. An issue arises though when we want to do more
complex searches based on the metadatas XML representation. What we want
there is a facility to match based on xpath expressions.

Enter exist (http://exist.sourceforge.net/). Exist is a native XML
datastore that allows you to locate stored documents that match a given
xpath expression (actually the xpath string matching is extended to allow
much stronger string matching, see
http://exist.sourceforge.net/xpath.html). Some rough initial tests show
that the matching speed is well within our tolerance levels (sub second
response times with a 10,000 document repository, three concurrent
clients).

As a bonus a exist jboss mbean service already exists :) so integration
into the server environment is eased.

Thoughts?

-k.

-- 
If you don't test then your code is only a collection of bugs which 
apparently behave like a working program. 

Website: http://www.rocketred.com.au/blogs/kevin/




-------------------------------------------------------
This SF.net email is sponsored by:
The Definitive IT and Networking Event. Be There!
NetWorld+Interop Las Vegas 2003 -- Register today!
http://ads.sourceforge.net/cgi-bin/redirect.pl?keyn0001en