Welcome to WebmasterWorld Guest from 188.8.131.52
Forum Moderators: lawman
How about changing:
"Search Engine Spider Identification"
"Search Engine & Crawler Identification"
Might not be a bad idea to lump the good guys with the bad guys in one set of discussions since IDing the good guys is only an issue because there are bad guys and vice versa.
JD is on the right track there, but I wonder how we avoid the privacy issues?
We don't like to get into senarios where we are id'ing private individuals
And I think that's why Brett you guys made that forum pre-moderated, also hiding the last IP octet rule except for recognized search engines takes care of those scenarios.
If anything, we need to spot, analyze and discuss everything new crawling our sites search engines or not, specially those originating from known hosting data centers, this interest ranges from business survival down to a hobby levels like plane & train spotting, I sure like to know more about them before getting run over by one.
Deep inside I wish it becomes as simple as "Crawler Identification", search engines thrown in as a bonus :-)