Re: separate indexing/crawling and web recherche

Alexander Barkov <[email protected]> Wed, 08 Jan 2014 18:21:42 +0400
Newsgroups gmane.comp.web.mnogosearch.general
Message-ID <[email protected]>
Hi,

On 01/06/2014 02:56 PM, Developpement Team Hodei wrote:
> hi,
>
> i have installed mnogosearch 3.2 on an unbutu server with wamp and mysql
> database.
>
> In ly indexer.conf i'm using *DBMode=blob
>
> *In my crontab i have this :
>
> 00 23 1 * * /usr/local/mnogosearch/sbin/indexer -d
> /usr/local/mnogosearch/etc/indexer.conf
>  >/usr/local/mnogosearch/sbin/logmng 2>/usr/local/mnogosearch/sbin/logmng2
> 30 23 1 * * /usr/local/mnogosearch/sbin/indexer -Eblob
> /usr/local/mnogosearch/etc/indexer.conf
>  >/usr/local/mnogosearch/sbin/logmng 2>/usr/local/mnogosearch/sbin/logmng2
>
> My question :
>
> to improve performanceis it necessary to separate the crawling/indexing
> of mnogosearch engine from which users make webs research ?
> in other words, is it necessary to create two servers
> * 1 ubuntu server wamp / mnogosearch with the mysql database and crontab
> commands
> * 1 ubuntu serverwamp / mnogosearch / wordpress (my website is on
> wordpress) with the same mysql database would be replicated (frequency
> like crontab) and without crontab commands (i'm using php frontend to
> access mnogosearch engine)

How big is your search database?
What does "indexer -S" return?

- crawling is not CPU consuming,
   it can co-exist with indexing and search without any problems.

- indexing and search are CPU consuming


For small databases (e.g. less than 100k documents) having is single
box for all three tasks is fine.

For bigger databases with a few hundred thousand documents a box with
multiple CPU cores can also handle all three tasks.

For huge databases with millions documents separation makes some sense.
However, instead of separating crawling/indexing/search tasks, I'd
propose to consider using cluster solution described here:
http://www.mnogosearch.org/doc33/msearch-cluster.html
It significantly improves search performance.

>
> thanks for your help
>
> _______________________________________________
> General mailing list
> [email protected]
> http://lists.mnogosearch.org/listinfo/general
>
_______________________________________________
General mailing list
[email protected]
http://lists.mnogosearch.org/listinfo/general