| 7:14 pm on Oct 29, 2008 (gmt 0)|
|6. Kind of redundant but, stopping nefarious googlebot spoofers from hammering away at your server |
A better way to phrase #6 IMO would be to "Stop SE hijackings of pages on your site" as more often that not I've found it's really the SE being directed to crawl through a proxy server.
| 8:03 pm on Oct 29, 2008 (gmt 0)|
Lately, most of the fake googlebot IPs I've seen are from broadband. Possibly someone is using a proxy hosted on a compromised computer but they never seem to go beyond two or three hits.
Item 5 should be worded more strongly or another item added. Script kiddies ain't too much hassle and fairly easily blocked - one false move and they're toast. The major problem is botnets trying to plant SQL Injection and suchlike exploits on your sites. Although at the moment they seem to be off attacking something else - Georgia, again?
| 9:18 pm on Oct 29, 2008 (gmt 0)|
For the past 2-3 months all my GoogleBot spoofers have been leaving a referrer that tracks back to a link in a message someone posted on a university-based forum. Weird.
My only reason for blocking bots: Stealing is wrong.
| 11:07 pm on Oct 29, 2008 (gmt 0)|
Out of the last 29 fake googlebot occurrences only one came with a referer - from google australia (I'm in uk but the site was relevant). Which, as noted elsewhere, makes me wonder if this is some idiots following some stupid recommendation on a dummies forum somewhere.
Not sure if this reason is a big one: stop sites pretending to have information about your site in order to draw traffic to themselves. For example, aboutus.org has absolutely no information about our domains except the name but claims it has. One of them has never been (legitimately) indexed by any search engine - it's disallowed in robots.txt - and when you try to go to the page it's completely blank. Personally I blame google for letting them get away with this: I see several other similar scams when googling for stuff. It's another of those scams that ebay and kelkoo (used to?) perpetrate: "Buy DSTILES Now" - I KNOW I'm not for sale!
Not sure about some of the "security" services, either, some of which give a single-point data source to hackers. I'm not really keen on the world knowing what OS my server is running and how fast my pages load and punters can discover that for themselves by legitimate browsing.
| 11:16 pm on Oct 29, 2008 (gmt 0)|
Here's an old thread [webmasterworld.com] of a similar topic