Forum Moderators: open
I know why and how you can identify them but why do they allow it. Why don't they change the way they appear in your log so that you wouldn't know it was them, so that they looked like any other mozilla or have super secret ip addresses that they spider from? What is stopping them from already doing this to help them identify some of the shadier SEO tecniques?
It's something I always wondered, but never came across the answer.
Remember, we webmasters allow the bots onto our sites, despite SEs attempts to pollute our point of view, it's an honour for Google, Yahoo and the rest of them to be allowed onto our sites. Misbehave - and you're barred for life :)