Re: How to «bootstrap» xapian/squatter

On 2024-09-26 3:47 AM, Albert Shih wrote:
> Le 25/09/2024 à 17:39:36-0600, Nels Lindquist a écrit
> 
>> On 2024-09-25 1:23 PM, Albert Shih wrote:
>>
>> You could try rebuilding the conversations databases with
>> "ctl_conversationsdb -v -b -r".
>>
>> Did you try increasing search_batchsize? That was the "one thing" that fixed
>> it for me:
> 
> Indeed !!!.
> 
> I'v  put 524288 for the search_batchsize and I think I will be
> over it in few hours.
> 
> Big thanks.

You're welcome! Glad that's working for you.

> How do you launch squatter ? Because I intend to launch squatter using
> sync_log so as I understand (but maybe I'm wrong that's why I ask),
> squatter are just going to use what inside the log file and index only the
> new mail.

I believe the sync_log is for rolling mode. I have it configured as a 
daemon in /etc/cyrus.conf.

squatter        cmd="squatter -R"

For doing an initial xapian index build (or for manually updating for 
whatever reason) I use "squatter -v -i" (as the cyrus user) which 
iterates over each mailbox and builds/updates the index. If there's no 
existing index for a mailbox, the "incremental" update builds the index 
from scratch using the existing mail. The "-v" can be skipped if you 
don't care to see the increased logging verbosity.

>> Run "squatter -v -i" in a screen/tmux window so you can see what it's doing;
>> if you observe repeated processing of the same folder/mailbox this could be
>> your answer.
> 
> And if it's the case what should I do ?

Sorry, that was unclear. I meant that to be a diagnostic to see whether 
increasing your search_batchsize might help. Since you've already done 
that (and it worked) you're all set.

-- 
Nels Lindquist
[email protected]


------------------------------------------
Cyrus: Info
Permalink: https://cyrus.topicbox.com/groups/info/Tb146f45ff23d5ffa-Md2748c180d1e5f79ab254376
Delivery options: https://cyrus.topicbox.com/groups/info/subscription