Forum Moderators: open

Message Too Old, No Replies

Does Googlebot (or any spider) accept Cookies?

Wondering if every request will cause a new session to start

         

the_nerd

8:54 am on Jul 30, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Does anybody know if spiders behave like a browser - accepting cookies for example - or do they just read text that comes from a URL?

The reason I'm curious about this: all my pages are served dynamically and spiders combined take in a 5-digits number of pages daily.

If no cookie representing the current session-ID is available then a new session will be started on the server - meaning they are opened much faster than they can time-out.

Any experience?

sabai

1:23 pm on Jul 30, 2004 (gmt 0)

10+ Year Member



Search engines spiders don't accept session cookies, or any other kind of cookie. If your site works without sessions then turn them off for search engine spiders. Depending on how you are handling your sessions, you may be passing the session ID in get requests too (php does this by default, for example) - in this case you definately want sessions turned off for spiders or you'll find them spirdering the same pages over and over again with different session IDs.