homepage Welcome to WebmasterWorld Guest from 54.234.0.85
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
On Dasher?
Megaclinium




msg:4041750
 12:51 am on Dec 12, 2009 (gmt 0)

I'm periodically hit by bots with a UA of '-',
probably trying to make you not notice as next to referring page which could be a '-'.

However this last one originates from yahoo range.

68.180.211.xx

OrgName: Yahoo! Inc.
OrgID: YHOO
Address: 701 First Ave
City: Sunnyvale
StateProv: CA
PostalCode: 94089
Country: US

NetRange: 68.180.128.0 - 68.180.255.255

Any idea why they are doing this?

 

dstiles




msg:4042077
 9:01 pm on Dec 12, 2009 (gmt 0)

68.180.211.nnn is not a yahoo bot range - at least, not slurp. As with most larger engines, a lot of their IP range has nothing directly to do with crawling.

If you do an rDNS lookup on the actual IP it should show you what it really is for. That particular C-net range comprises betas, shops, video search (which may be some kind of bot but not identified as such) but is mostly answers and mobile. Again, these may be some kind of bot but if so they're not being honest about it.

I get a lot of yahoo accesses with absolutely no headers but rDNS returns of things like ycar1.mobile.ch1.yahoo.com. Since they have no UA or other headers they get blocked.

keyplyr




msg:4042139
 12:10 am on Dec 13, 2009 (gmt 0)

I allow blank referrers from the mobile Yahoo ranges which include the IP in OP. A rapidly increasing share of my traffic comes from mobile search including Yahoo.

Megaclinium




msg:4042584
 3:10 am on Dec 14, 2009 (gmt 0)

Thanks! I see '-' from alot of other addresses but not from yahoo range.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved