lucy24 - 10:13 pm on May 14, 2011 (gmt 0)

It's their web preview ... I guess they think it's cool to break protocol and standards when it comes to making their visitors happy and possibly keeping them on their site rather than just saying 'this site does not allow previews' and sending them to the site in the results. They're actually making the request too ... The system checks for an X-Forwarded-For so if they were 'proxy requesting' for visitors that click on the preview it should show the visitor's IP Address, not theirs.

But this makes it easy to lock them out.

Far as I can tell, 64.233.x.x, 72.14.x.x, 74.125.x.x are strictly Google Web Preview. Yesterday they threw me a curve ball with 66.249.85.x from the range they normally use for the Googlebot. I hope they don't make a habit of this. Anyway they're perfectly open about it-- or brazen, if you prefer-- so you can always do it by UA.

I wouldn't mind GWP except that the raw stats don't say what search term the user entered, so there's no way of knowing what it is that the user was almost interested in enough to visit your page. And I'm glad I don't pay for bandwidth, since they insist on pulling up every image, whether or not it will actually be displayed in the preview. Can't they get the ### thing from their own cache?

The first thing to establish is whether it is a genuine GoogleBot.

Yup, just found a spoofer at 110.44.29.76 (in Australia, apparently). Their sole request was for robots.txt, which was a dead giveaway. :: snicker ::