homepage Welcome to WebmasterWorld Guest from 23.21.9.44
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
Yellow Pages bot impersonating Googlebot?
mrtonyg




msg:4300825
 12:16 am on Apr 20, 2011 (gmt 0)

I just saw this on my log:

64.209.132.4 - - [19/Apr/2011:04:37:59 -0400] "GET /filename.html HTTP/1.1" 403 380 "-" "YPBot/Raven1.1.3 (compatible; Googlebot/2.1;+http://www.yellowpages.com/about/legal/crawl)"

Yellow Pages IP with both their bot and Googlebot names...?

Do the yellow pages and google have a partnership?

What is going on...should I allow them access?

Any advice would be appreciated.

 

topr8




msg:4301095
 7:00 am on Apr 20, 2011 (gmt 0)

anything calling itself googlebot that doesn't pass a reverse dns test gets blocked by us.

mrtonyg




msg:4301604
 1:08 am on Apr 21, 2011 (gmt 0)

I blocked them, but thought it was very odd that a relatively mainstream company would attempt to pull a fast one.

topr8




msg:4303559
 9:41 am on Apr 25, 2011 (gmt 0)

... actually there is no hostname associated with the ip address you gave ...

quite possibly not yellow pages either!

phranque




msg:4303574
 10:52 am on Apr 25, 2011 (gmt 0)

Yellowpages.com SAVV-S265785-1 (NET-64-209-132-0-1) 64.209.132.0 - 64.209.132.255

topr8




msg:4303601
 1:14 pm on Apr 25, 2011 (gmt 0)

ok so it is yellow pages, but no hostname set up!

dstiles




msg:4303759
 7:14 pm on Apr 25, 2011 (gmt 0)

I have no bot recorded as having come from that range and certainly not a googlebot, which would get rejected on that IP range, or a YP bot.

There is no rDNS, as topr8 mentions, so it's due a blocking.

The parent of that IP range is Savvis, which is worth blocking anyway.

Staffa




msg:4340101
 6:11 pm on Jul 15, 2011 (gmt 0)

Got a visit today from 208.93.105.250 - null.ev1.yellowpages.com
YPBot/Raven1.1.3 (compatible; Googlebot/2.1; http: //www.yellowpages.com/about/legal/crawl)

Took robots.txt and left, well good luck to them when it comes back ;o)
The 208. range was never in my good books

lucy24




msg:4340142
 7:44 pm on Jul 15, 2011 (gmt 0)

Any possible relationship to this activity [webmasterworld.com]?

Staffa




msg:4340186
 9:41 pm on Jul 15, 2011 (gmt 0)

No, none.
This is Ypages, the other thread is Ybook

g1smd




msg:4340190
 9:57 pm on Jul 15, 2011 (gmt 0)

There's a LOT of stuff in 208. much of it not good.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved