Fake Googlebot

Had me puzzled for a minute...

[xlcus]

I was looking through the logs of one of my sites for Googlebot visits tonight. I was happily scrolling through, noticing a few Freshbot visits here and there, and then suddenly saw a Googlebot visit from a 2xx.xxx.xxx.xxx address! :o

A few thoughts raced through my head for a while... Was this a surprise mid month crawl? Was it someone from Google checking out my site?

Then looked at the ip address a little closer... It looked a little unusual for Google. Checked it out a little more thoroughly, and it turned out to be a dial-up ip of some isp.

Someone's obviously having a laugh going round scaring paranoid webmasters with a fake user agent set in their browser! ;)

[fiestagirl]

It was probably someone surfing around looking for UA cloaking...

[Giacomo]

It's quite easy to spoof the user agent with Mozilla.

[Dreamquick]

Just as an aside 216.239.* is supposed to be google's "fresh" bot, but obviously if you have traced it to a dialup then this one could be ruled out - just felt I should point that not all 2XX.* traffic claiming to be googlebot was going to be fake.

- Tony

[jdMorgan]

xlcus,

I'd appreciate it if you would post the IP address - at least the first three digit groupings. As Dreamquick points out, you might save the casual reader from banning the real googlebot.

Thanks,
Jim

[Giacomo]

jdMorgan, if you're afraid of the risk of banning the real G'bot, then don't do IP banning.

[google.com...]

[jdMorgan]

Giacomo,

I block, ban, and cloak (for security purposes) quite successfully using combinations of IP addresses, user-agents, and other request-based variables. I also set up traps to catch vaguely-reported troublemakers and gather more information about them. But I can't even set up a trap with a 2xx.xxx.xxx.xxx address - That's a rather broad range. :(

Jim

[Deodato]

I set mozilla to mimic googlebot once in a while when I check on my competitors. As fiestagirl says, it's probably just someone doing the same - if you're not doing anything funky, I'd just ignore it. Oh, and have a go yourself - you might uncover some cheesy seo.

[jdMorgan]

I've used wannabrowser several times to do just that... Sometimes it's quite amusing... Almost as funny as the "expert" SEO company that wanted to pay to add an invisible link to carfac's site [webmasterworld.com]. :)

Jim

[Marval]

Saw a similar hit today from a "Google CHTML Proxy" although it did have what seemed to be part of their block.. 216.239.39.x ..oh and UP.Browser was part of the line

[Key_Master]

On my sites, I monitor visits from IP addresses using major search engine spider user-agents. In some cases, you're likely to find the cloaked spam you were looking for. The only problem is, I only spam cloaked agents and reporting your findings to the search engines will only make you look dumb. On top of that, your IP has been tagged as a potential competitor and subsequent visits from that IP will be recorded and closely monitored.

<agent =~ "^googlebot.*$">
skip if (ip =~ "^216\.239\.(3[2-9]¦[4-5][0-9]¦6[0-3])\..*$¦^64\.68\.8[0-7]\..*$");
append.file("/guard/cloaked-agent.txt");
write "[ip]¦[host]¦[agent]¦[document]¦[referrer]¦[time]¦[date]";
close.file;
mail.alert("Cloaked Agent");
<!-- insert optional spamming and misleading info here -->
exit;
</agent>

[xlcus]

I'd appreciate it if you would post the IP address - at least the first three digit groupings.

213.190.192.xxx

Looks like an ISP in Portugal

[jdMorgan]

xlcus,

Thanks! Just wanted to know in case this got more widespread...

Key_Master,

Cool idea! - I'm sure I would find visiting your site with a fake UA interesting and amusing, too. I'm more interested in the technology angle - I don't take any of this stuff TOO seriously. :)

Marval,
Hmmm... Now what's a "CHTML proxy"? I've seen their WAP proxy for wireless devices, but not CHTML.

Jim

[Yidaki]

jd Morgan, it's a google imode proxy. Read more about Google's CHTML proxy [appelsiini.net].

[jdMorgan]

yidaki,

Thanks!

Jim