Re: Sessions and search spiders

On 21 Oct 2006, at 08:54, Will Scheidegger wrote:

> Some of our webapps create a session for every visitor, which is  
> rather a common thing I would say. This works more or less fine  
> when the page content does not have to be indexed by search  
> spiders: Simply mark the pages as noindex, nofollow and block the  
> IP# of the spiders that do not respect these instructions.
>
> But how do you deal with the situation where the pages _should_ be  
> indexed? Here, the search spiders are creating new sessions until  
> the app goes down when they revisit old links (containing old  
> session ids). What is the correct way to handle this?

Session based apps can't be meaningfully indexed without a lot of  
work.  You need either to switch to direct actions without sessions,  
so that the paths can be followed safely from outside of the session,  
or to devise a way to recreate the session context when revisited -  
which, to my mind, is the same as creating the pages as direct actions.

Paul