Merging solr indexes with duplicate keys - merging duplicate documents

Gagandeep singh <[email protected]>
Newsgroups gmane.comp.jakarta.lucene.solr.user,gmane.comp.apache.commons.general
Message-ID <CABq=fs_uOKi1U7_1UpgUvZeO967itFx1mStuPoX-RnTCRKpkcw@mail.gmail.com>
Hi folks

We have a use case where i have 2 solr indexes with the same schema but
different field populated, for example:

Common schema:
<field name="url" type="text" />      // Unique key
<field name="product_name" type="text" />
<field name="image" type="text" />
<field name="brand" type="text" />
<field name="description" type="text" />

<field name="out_of_stock" type="boolean" />
<field name="num_likes" type="int" />
<field name="num_add_2_cart" type="int" />

Now i have one index which stores the information about products (first 5
fields). This index is built every 2 days.
I have a 2nd index which stores social signals (url + out_of_stock  +
num_likes + num_add_2_cart). This index is built every 2 hours and is used
for a near realtime boosting products.
The processes for building these indexes are independent, and for
operational management and for sake of reuse i would like to build these
indexes separately.

My question is, is there a convenient way of merging these 2 indexes (other
than applying document updates in a loop)? The IndexMergeTool from lucene
is not capable of applying document updates and would end up keeping either
first 5 field or last 3.

Thanks
Gagan
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.