Re: Indexing protected area

Bill Moseley wrote:
> It can get confusing since there's so many ways to do things and since
> different programs are doing different parts of the indexing.

Yes, I noticed that. :-)

I'm starting to get the idea of how it works.

>     spider.pl default http://yoursite.to.index/ > out.txt

Thanks, I hadn't read far enough to know about that "default" option. I was
busy setting up a config file based on the minimal example - if I'd seen
that line in the docs first I would have done that straight away.

So, anyway, I did that and got the main index page, so I know it works.

My mission is to allow searching in some password-protected sub-sites that
aren't linked from the main page so I think I'll have to do them each
individually.

Would it make sense to maintain a separate index for each one rather than
put it all in together with the main index, even though they're all pretty
small?

I think I like the idea of leaving the main site index as it is and treating
the new bits separately.

LesleyW.
(Thinking out loud...)