Forum Moderators: open

Message Too Old, No Replies

Why are search engine bots identifiable

Perhaps a dumb question

         

hannamyluv

7:53 pm on Oct 27, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



I have always wondered this, but why are search engine bots identifible?

I know why and how you can identify them but why do they allow it. Why don't they change the way they appear in your log so that you wouldn't know it was them, so that they looked like any other mozilla or have super secret ip addresses that they spider from? What is stopping them from already doing this to help them identify some of the shadier SEO tecniques?

It's something I always wondered, but never came across the answer.

Sanenet

2:12 pm on Oct 28, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Courtesy - so that the webmaster can seperate users and bots, and see what the bot is doing on their sites.

Remember, we webmasters allow the bots onto our sites, despite SEs attempts to pollute our point of view, it's an honour for Google, Yahoo and the rest of them to be allowed onto our sites. Misbehave - and you're barred for life :)