homepage Welcome to WebmasterWorld Guest from 54.167.179.48
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
On Dasher?
Megaclinium

5+ Year Member



 
Msg#: 4041748 posted 12:51 am on Dec 12, 2009 (gmt 0)

I'm periodically hit by bots with a UA of '-',
probably trying to make you not notice as next to referring page which could be a '-'.

However this last one originates from yahoo range.

68.180.211.xx

OrgName: Yahoo! Inc.
OrgID: YHOO
Address: 701 First Ave
City: Sunnyvale
StateProv: CA
PostalCode: 94089
Country: US

NetRange: 68.180.128.0 - 68.180.255.255

Any idea why they are doing this?

 

dstiles

WebmasterWorld Senior Member dstiles us a WebmasterWorld Top Contributor of All Time 5+ Year Member



 
Msg#: 4041748 posted 9:01 pm on Dec 12, 2009 (gmt 0)

68.180.211.nnn is not a yahoo bot range - at least, not slurp. As with most larger engines, a lot of their IP range has nothing directly to do with crawling.

If you do an rDNS lookup on the actual IP it should show you what it really is for. That particular C-net range comprises betas, shops, video search (which may be some kind of bot but not identified as such) but is mostly answers and mobile. Again, these may be some kind of bot but if so they're not being honest about it.

I get a lot of yahoo accesses with absolutely no headers but rDNS returns of things like ycar1.mobile.ch1.yahoo.com. Since they have no UA or other headers they get blocked.

keyplyr

WebmasterWorld Senior Member keyplyr us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4041748 posted 12:10 am on Dec 13, 2009 (gmt 0)

I allow blank referrers from the mobile Yahoo ranges which include the IP in OP. A rapidly increasing share of my traffic comes from mobile search including Yahoo.

Megaclinium

5+ Year Member



 
Msg#: 4041748 posted 3:10 am on Dec 14, 2009 (gmt 0)

Thanks! I see '-' from alot of other addresses but not from yahoo range.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved