GoogleDocs

[Pfui]

In 10 minutes, this UA --

Mozilla/5.0 (compatible; GoogleDocs; +http://docs.google.com)

-- hailing from bare (no rDNS) Google IPs and postnews2 hit root 218 and 8 times respectively.

No robots.txt.
No referrer.
No graphics.
No way to figure out who may have done what with my stuff.
No info other than a ridiculous, relentless number of bot-baited hits:

[1n:49:38] - 74.125.64.95
[1n:49:41] - 74.125.64.95
[1n:49:43] - 74.125.64.83
[1n:49:46] - 74.125.156.85
[1n:49:48] - 74.125.64.94
[1n:49:50] - 74.125.64.88
[1n:49:53] - 74.125.64.83
[1n:49:55] - 74.125.64.93
[1n:49:57] - 74.125.156.94
[1n:50:00] - 74.125.64.93
[1n:50:02] - 74.125.156.85
[1n:50:04] - 74.125.156.83
[1n:50:07] - 74.125.64.95
[1n:50:09] - postnews2.google.com [rDNS: 74.125.64.80]
[1n:50:12] - 74.125.64.87
[1n:50:14] - 74.125.64.92
(very partial listing)

220 hits in 10 minutes. That's approx. one hit every two to three seconds.

Thoughts?

[dstiles]

If anything from google IP, block access :(

If only! Trouble is: in UK about 90% of SE traffic is from google. :(

As it is, I have (eg) 74.125.64.80 - 74.125.64.95 blocked as carrying unwanted google utils. Even if they are not unwanted, they get blocked. It's time G (and others) assigned specific traffic to specific IPs - and told us about them!

[Staffa]

Ditto, I have all (the known to me) sidelines of google blocked, some are unwanted and some are irrelevant to my sites and it's a way for them to crawl while bypassing robots.txt

[keyplyr]

I have a different approach. Since SE's do not always assign specific IP ranges to their respective utilities, I allow all verified SE IPs and block those UAs I don't want.