homepage Welcome to WebmasterWorld Guest from 54.211.230.186
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
fatlens.com using Googlebot agents?
brokaddr



 
Msg#: 4445759 posted 2:43 am on Apr 26, 2012 (gmt 0)

For a few weeks now I've been noticing Fatbot using Google user agents. I do not mind if FatBot indexes my site, but I don't why/if they are spoofing or using Google agents to bypass filters set up for non-Google bots?

Page: /image/example.jpg
IP: 208.83.154.22
User Agent: GooglebotImage/1.0

I did email them about this and only got a reply that they were 'looking into it' - no follow-up after that.

 

wilderness

WebmasterWorld Senior Member wilderness us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4445759 posted 3:29 am on Apr 26, 2012 (gmt 0)

fake+google [google.com]

incrediBILL

WebmasterWorld Administrator incredibill us a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month



 
Msg#: 4445759 posted 3:29 am on Apr 26, 2012 (gmt 0)

You're probably right, they're probably trying to bypass filters.

If you're going to spoof to bypass filters it seems more logical to use a browser user agent than Googlebot which can be easily authenticated and fakes blocked in real-time. Although browser agents coming from hosting locations is a policy violation for many bot blockers so they would deny access to that as well.

Sounds like they're just going to have to live with getting blocked in a few places now doesn't it? :)

brokaddr



 
Msg#: 4445759 posted 3:53 am on Apr 26, 2012 (gmt 0)

Thanks for the followups guys, I was concerned I was possibly mis-detecting something, as Fatbot is from an otherwise legit company.

Hope they get this looked after.. I'd have no problem adding their bot to the same whitelist Google is currently on if it ran its own user agent. I won't whitelist bots that partake in activity, such as this, however.

enigma1

WebmasterWorld Senior Member 5+ Year Member



 
Msg#: 4445759 posted 1:19 pm on May 2, 2012 (gmt 0)

I did email them about this and only got a reply that they were 'looking into it' - no follow-up after that.

Did you tell them how to fix it? It's the "Googlebot-Image/1.0" UA they need to fake.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved