homepage Welcome to WebmasterWorld Guest from 54.205.127.52
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
Yellow Pages bot impersonating Googlebot?
mrtonyg



 
Msg#: 4303575 posted 12:16 am on Apr 20, 2011 (gmt 0)

I just saw this on my log:

64.209.132.4 - - [19/Apr/2011:04:37:59 -0400] "GET /filename.html HTTP/1.1" 403 380 "-" "YPBot/Raven1.1.3 (compatible; Googlebot/2.1;+http://www.yellowpages.com/about/legal/crawl)"

Yellow Pages IP with both their bot and Googlebot names...?

Do the yellow pages and google have a partnership?

What is going on...should I allow them access?

Any advice would be appreciated.

 

topr8

WebmasterWorld Senior Member topr8 us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 4303575 posted 7:00 am on Apr 20, 2011 (gmt 0)

anything calling itself googlebot that doesn't pass a reverse dns test gets blocked by us.

mrtonyg



 
Msg#: 4303575 posted 1:08 am on Apr 21, 2011 (gmt 0)

I blocked them, but thought it was very odd that a relatively mainstream company would attempt to pull a fast one.

topr8

WebmasterWorld Senior Member topr8 us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 4303575 posted 9:41 am on Apr 25, 2011 (gmt 0)

... actually there is no hostname associated with the ip address you gave ...

quite possibly not yellow pages either!

phranque

WebmasterWorld Administrator phranque us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4303575 posted 10:52 am on Apr 25, 2011 (gmt 0)

Yellowpages.com SAVV-S265785-1 (NET-64-209-132-0-1) 64.209.132.0 - 64.209.132.255

topr8

WebmasterWorld Senior Member topr8 us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 4303575 posted 1:14 pm on Apr 25, 2011 (gmt 0)

ok so it is yellow pages, but no hostname set up!

dstiles

WebmasterWorld Senior Member dstiles us a WebmasterWorld Top Contributor of All Time 5+ Year Member



 
Msg#: 4303575 posted 7:14 pm on Apr 25, 2011 (gmt 0)

I have no bot recorded as having come from that range and certainly not a googlebot, which would get rejected on that IP range, or a YP bot.

There is no rDNS, as topr8 mentions, so it's due a blocking.

The parent of that IP range is Savvis, which is worth blocking anyway.

Staffa

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 4303575 posted 6:11 pm on Jul 15, 2011 (gmt 0)

Got a visit today from 208.93.105.250 - null.ev1.yellowpages.com
YPBot/Raven1.1.3 (compatible; Googlebot/2.1; http: //www.yellowpages.com/about/legal/crawl)

Took robots.txt and left, well good luck to them when it comes back ;o)
The 208. range was never in my good books

lucy24

WebmasterWorld Senior Member lucy24 us a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



 
Msg#: 4303575 posted 7:44 pm on Jul 15, 2011 (gmt 0)

Any possible relationship to this activity [webmasterworld.com]?

Staffa

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 4303575 posted 9:41 pm on Jul 15, 2011 (gmt 0)

No, none.
This is Ypages, the other thread is Ybook

g1smd

WebmasterWorld Senior Member g1smd us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4303575 posted 9:57 pm on Jul 15, 2011 (gmt 0)

There's a LOT of stuff in 208. much of it not good.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved