Re: Xapian searches of the body of an email

Hi Egon,

Yes, the slave should index in conversations.db automatically AFAIK. 

You should run squatter in rolling mode on the master, too.  

BTW: in 2014, Bron wrote a blog post about the search setup at FastMail: https://fastmail.blog/2014/12/01/email-search-system/It’s quite technical, but should give you a good idea at how it’s set up
for fast indexing and search
Cheers, Robert 


On Mon, Jan 7, 2019, at 5:54 PM, Egoitz Aurrekoetxea wrote:
> Hi Robert!


> 


> Thank you so much for helping us (mainly which is the one boring the
> list with questions :) although I promise I've checked the doc before
> asking :) :)  ).> 


> When you have a master/slave config... in the slave one, when running
> Squatter in rolling mode... does it update the conversations db too?.
> By the way, Squatter in rolling mode only makes sense in slave
> machines isn't it?.> 


> Many thanks!


> 


> ---
> 
> sarenet
> *Egoitz Aurrekoetxea*
> Departamento de sistemas
> 944 209 470
> Parque Tecnológico. Edificio 103
> 48170 Zamudio (Bizkaia)
> [email protected]
> www.sarenet.es
> 
> Antes de imprimir este correo electrónico piense si es necesario
> hacerlo.> 


> El 07-01-2019 16:42, Robert Stepanek escribió:


>> Hi,
>>  
>> Sebastian is right:
>>  
>> On Mon, Jan 7, 2019, at 3:57 PM, Sebastian Hagedorn wrote:
>>>  
>>> squatter is nowadays a bit of a misnomer, because it uses
>>> whatever index>>> you have configured. In cyrus 2.4, squatter would always create
>>> a SQUAT>>> index. When you run squatter with Xapian, it will build the index,
>>> but for>>> the index to actually work, you also need the conversationsdb.
>>  
>> conversations.db is indeed a misnomer now. The database was only used
>> to keep track of mail threads (hence the name), but its role
>> expanded. One of the indexes it stores is the SHA1 hashes of every
>> message, and separate hashes for each of that message MIME parts.
>> Such a hash is named the GUID, and for each GUID we store a list of
>> all mailbox:UID[bodypart] pairs where this content occurs in.>>  
>> For search, we keep track of the indexed messages by GUID, so we can
>> avoid reindexing duplicate mails. To return a search result, we can
>> now map that GUID back to its mailbox:message pairs. That's why we
>> need conversations.db for search.>>  
>> I can't help with upgrading from 2.4, unfortunately, but if you re-
>> index your mailboxes once in conversations.db, you should be all set.>>  
>> Cheers,
>> Robert
>> 
>> ----
>>  Cyrus Home Page: http://www.cyrusimap.org/
>>  List Archives/Info:
>>  http://lists.andrew.cmu.edu/pipermail/info-cyrus/>>  To Unsubscribe:
>> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

----
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus