homepage Welcome to WebmasterWorld Guest from 54.227.56.174
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Subscribe to WebmasterWorld

Home / Forums Index / Google / Google News Archive
Forum Library, Charter, Moderator: open

Google News Archive Forum

    
Fake Googlebot
Had me puzzled for a minute...
xlcus




msg:105587
 9:11 pm on Jan 21, 2003 (gmt 0)

I was looking through the logs of one of my sites for Googlebot visits tonight. I was happily scrolling through, noticing a few Freshbot visits here and there, and then suddenly saw a Googlebot visit from a 2xx.xxx.xxx.xxx address! :o

A few thoughts raced through my head for a while... Was this a surprise mid month crawl? Was it someone from Google checking out my site?

Then looked at the ip address a little closer... It looked a little unusual for Google. Checked it out a little more thoroughly, and it turned out to be a dial-up ip of some isp.

Someone's obviously having a laugh going round scaring paranoid webmasters with a fake user agent set in their browser! ;)

 

fiestagirl




msg:105588
 9:22 pm on Jan 21, 2003 (gmt 0)

It was probably someone surfing around looking for UA cloaking...

Giacomo




msg:105589
 10:58 pm on Jan 21, 2003 (gmt 0)

It's quite easy to spoof the user agent with Mozilla.

Dreamquick




msg:105590
 11:31 pm on Jan 21, 2003 (gmt 0)

Just as an aside 216.239.* is supposed to be google's "fresh" bot, but obviously if you have traced it to a dialup then this one could be ruled out - just felt I should point that not all 2XX.* traffic claiming to be googlebot was going to be fake.

- Tony

jdMorgan




msg:105591
 11:36 pm on Jan 21, 2003 (gmt 0)

xlcus,

I'd appreciate it if you would post the IP address - at least the first three digit groupings. As Dreamquick points out, you might save the casual reader from banning the real googlebot.

Thanks,
Jim

Giacomo




msg:105592
 11:41 pm on Jan 21, 2003 (gmt 0)

jdMorgan, if you're afraid of the risk of banning the real G'bot, then don't do IP banning.

[google.com...]

jdMorgan




msg:105593
 12:26 am on Jan 22, 2003 (gmt 0)

Giacomo,

I block, ban, and cloak (for security purposes) quite successfully using combinations of IP addresses, user-agents, and other request-based variables. I also set up traps to catch vaguely-reported troublemakers and gather more information about them. But I can't even set up a trap with a 2xx.xxx.xxx.xxx address - That's a rather broad range. :(

Jim

Deodato




msg:105594
 2:23 am on Jan 22, 2003 (gmt 0)

I set mozilla to mimic googlebot once in a while when I check on my competitors. As fiestagirl says, it's probably just someone doing the same - if you're not doing anything funky, I'd just ignore it. Oh, and have a go yourself - you might uncover some cheesy seo.

jdMorgan




msg:105595
 3:16 am on Jan 22, 2003 (gmt 0)

I've used wannabrowser several times to do just that... Sometimes it's quite amusing... Almost as funny as the "expert" SEO company that wanted to pay to add an invisible link to carfac's site [webmasterworld.com]. :)

Jim

Marval




msg:105596
 3:21 am on Jan 22, 2003 (gmt 0)

Saw a similar hit today from a "Google CHTML Proxy" although it did have what seemed to be part of their block.. 216.239.39.x ..oh and UP.Browser was part of the line

Key_Master




msg:105597
 4:03 am on Jan 22, 2003 (gmt 0)

On my sites, I monitor visits from IP addresses using major search engine spider user-agents. In some cases, you're likely to find the cloaked spam you were looking for. The only problem is, I only spam cloaked agents and reporting your findings to the search engines will only make you look dumb. On top of that, your IP has been tagged as a potential competitor and subsequent visits from that IP will be recorded and closely monitored.

<agent =~ "^googlebot.*$">
skip if (ip =~ "^216\.239\.(3[2-9][4-5][0-9]6[0-3])\..*$^64\.68\.8[0-7]\..*$");
append.file("/guard/cloaked-agent.txt");
write "[ip][host][agent][document][referrer][time][date]";
close.file;
mail.alert("Cloaked Agent");
<!-- insert optional spamming and misleading info here -->
exit;
</agent>

xlcus




msg:105598
 9:53 am on Jan 22, 2003 (gmt 0)

I'd appreciate it if you would post the IP address - at least the first three digit groupings.

213.190.192.xxx

Looks like an ISP in Portugal

jdMorgan




msg:105599
 5:46 pm on Jan 22, 2003 (gmt 0)

xlcus,

Thanks! Just wanted to know in case this got more widespread...

Key_Master,

Cool idea! - I'm sure I would find visiting your site with a fake UA interesting and amusing, too. I'm more interested in the technology angle - I don't take any of this stuff TOO seriously. :)

Marval,
Hmmm... Now what's a "CHTML proxy"? I've seen their WAP proxy for wireless devices, but not CHTML.

Jim

Yidaki




msg:105600
 6:35 pm on Jan 22, 2003 (gmt 0)

jd Morgan, it's a google imode proxy. Read more about Google's CHTML proxy [appelsiini.net].

jdMorgan




msg:105601
 6:43 pm on Jan 22, 2003 (gmt 0)

yidaki,

Thanks!

Jim

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Google / Google News Archive
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved