homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

On Dasher?

 12:51 am on Dec 12, 2009 (gmt 0)

I'm periodically hit by bots with a UA of '-',
probably trying to make you not notice as next to referring page which could be a '-'.

However this last one originates from yahoo range.


OrgName: Yahoo! Inc.
Address: 701 First Ave
City: Sunnyvale
StateProv: CA
PostalCode: 94089
Country: US

NetRange: -

Any idea why they are doing this?



 9:01 pm on Dec 12, 2009 (gmt 0)

68.180.211.nnn is not a yahoo bot range - at least, not slurp. As with most larger engines, a lot of their IP range has nothing directly to do with crawling.

If you do an rDNS lookup on the actual IP it should show you what it really is for. That particular C-net range comprises betas, shops, video search (which may be some kind of bot but not identified as such) but is mostly answers and mobile. Again, these may be some kind of bot but if so they're not being honest about it.

I get a lot of yahoo accesses with absolutely no headers but rDNS returns of things like ycar1.mobile.ch1.yahoo.com. Since they have no UA or other headers they get blocked.


 12:10 am on Dec 13, 2009 (gmt 0)

I allow blank referrers from the mobile Yahoo ranges which include the IP in OP. A rapidly increasing share of my traffic comes from mobile search including Yahoo.


 3:10 am on Dec 14, 2009 (gmt 0)

Thanks! I see '-' from alot of other addresses but not from yahoo range.

Global Options:
 top home search open messages active posts  

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved