Re: reindexing with rundig

On Wed, 13 Jul 2005, Clift, Robert T CTR K55-Branch wrote:

> All, I have read the threads about reindexing a site by removing the "-i" from rundig in order to just do an update. I have removed the -i option and it seems to still reindex every document. Any ideas??

The determination of whether a document needs to be reindexed is based 
on the Last-Modified header returned by the server the last time the 
document was indexed and how the server responds to the 
If-Modified-Since header. If the server does not provide last modified
headers, then htdig defaults to using the current time. Typically this
is the right thing to do as far as updates go.

If the server does not honor the If-Modified-Since header, then update
indexing will not work. The same is often true for sites that build
pages dynamically since it is not unusual for such sites to always
return the current time as the last-modified time.

If you run an initial dig with at least three v's (e.g. ./rundig -vvv)
the output should include any Last-Modified headers returned by the
server(s) in question.

Jim


-------------------------------------------------------
SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
from IBM. Find simple to follow Roadmaps, straightforward articles,
informative Webcasts and more! Get everything you need to get up to
speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
_______________________________________________
ht://Dig general mailing list: <[email protected]>
ht://Dig FAQ: http://htdig.sourceforge.net/FAQ.html
List information (subscribe/unsubscribe, etc.)
https://lists.sourceforge.net/lists/listinfo/htdig-general