homepage Welcome to WebmasterWorld Guest from 54.197.94.241
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
fatlens.com using Googlebot agents?
brokaddr




msg:4445761
 2:43 am on Apr 26, 2012 (gmt 0)

For a few weeks now I've been noticing Fatbot using Google user agents. I do not mind if FatBot indexes my site, but I don't why/if they are spoofing or using Google agents to bypass filters set up for non-Google bots?

Page: /image/example.jpg
IP: 208.83.154.22
User Agent: GooglebotImage/1.0

I did email them about this and only got a reply that they were 'looking into it' - no follow-up after that.

 

wilderness




msg:4445771
 3:29 am on Apr 26, 2012 (gmt 0)

fake+google [google.com]

incrediBILL




msg:4445772
 3:29 am on Apr 26, 2012 (gmt 0)

You're probably right, they're probably trying to bypass filters.

If you're going to spoof to bypass filters it seems more logical to use a browser user agent than Googlebot which can be easily authenticated and fakes blocked in real-time. Although browser agents coming from hosting locations is a policy violation for many bot blockers so they would deny access to that as well.

Sounds like they're just going to have to live with getting blocked in a few places now doesn't it? :)

brokaddr




msg:4445777
 3:53 am on Apr 26, 2012 (gmt 0)

Thanks for the followups guys, I was concerned I was possibly mis-detecting something, as Fatbot is from an otherwise legit company.

Hope they get this looked after.. I'd have no problem adding their bot to the same whitelist Google is currently on if it ran its own user agent. I won't whitelist bots that partake in activity, such as this, however.

enigma1




msg:4448430
 1:19 pm on May 2, 2012 (gmt 0)

I did email them about this and only got a reply that they were 'looking into it' - no follow-up after that.

Did you tell them how to fix it? It's the "Googlebot-Image/1.0" UA they need to fake.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved