Webboard: index only new pages

Author: Jeff Dwork
Email: [email protected]
Message:
Unfortunately it did not work, but I found a working method.

I added 'Period 30y' before my 'Server' command in config file and 
did
 indexer --drop
 indexer --create
 indexer -a
It ran forever. I killed it (ctrl-C) and it reported crawling over 
500000 pages - there are about 16000 pages on the site.

I removed the 'Period' command and reindexed the site. I then 
added a new directory with the newest pages and did:

 indexer -ai -u 'https://domain.com/msgs/v117n014/%.html'
 indexer --index

This processed only the new pages and correctly added them to the 
index.

Thanks,
Jeff

Reply: <http://www.mnogosearch.org/board/message.php?id=21819>

_______________________________________________
General mailing list
[email protected]
http://lists.mnogosearch.org/listinfo/general