Sessions and search spiders

Will Scheidegger <[email protected]>
Newsgroups gmane.comp.web.webobjects.devel
Message-ID <[email protected]>
Some of our webapps create a session for every visitor, which is  
rather a common thing I would say. This works more or less fine when  
the page content does not have to be indexed by search spiders:  
Simply mark the pages as noindex, nofollow and block the IP# of the  
spiders that do not respect these instructions.

But how do you deal with the situation where the pages _should_ be  
indexed? Here, the search spiders are creating new sessions until the  
app goes down when they revisit old links (containing old session  
ids). What is the correct way to handle this?

Thanks!

Will
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.