Welcome to WebmasterWorld Guest from 54.163.40.152

Forum Moderators: Ocean10000 & incrediBILL

Message Too Old, No Replies

*missing* - user agent from Yahoo, or spoof?

     

brokaddr

5:38 am on Apr 26, 2012 (gmt 0)

5+ Year Member



For a recurring period of time, Yahoo (or what appears to be Yahoo) is sending these peculiar user agents - is this a spoof? They're viewing/indexing everything from pages to images.

One thing that stands out is the hostname, "mrs8.mobile.bf1.yahoo.com" - does Yahoo have some sort of mobile app unrelated to their indexing search engine that allows user to browse pages... I would assume a toolbar could potentially sent this kind of data?

I did try to research the user agent, but for obvious reasons came up empty handed.

Page Viewed: image/example.jpg
Referrer: None
IP: 98.139.242.230
User Agent: *missing*

lucy24

8:29 am on Apr 26, 2012 (gmt 0)

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



Tangential query: Why are they getting through at all? Most people categorically block blank UAs, with maybe a few loopholes such as robots.txt.

Edit: Oh, wait. Do you mean that "missing" is the actual UA string? Lordy, lordy, what will they think of next?

It's definitely a Yahoo IP. But then, they've got more of 'em than you can shake a stick at.

:: shuffling papers ::

8.12.144 Yahoo! Mobile
67.195 Yahoo! Slurp
69.147.64.0/18 yahoo of some sort
72.30 Slurp again
74.6 and more
98.136.0.0/14
-- which I've flagged as including 98.137.72 Slurp and 98.139.241 YahooCacheSystem
184.165 unspecified Yahoo
209.131.32.0/19 Yahoo mobile
209.191.64.0/18 more yahoo

Some of these I've never actually met in person, I just stumbled across them while filling in gaps between humans.

wilderness

3:32 pm on Apr 26, 2012 (gmt 0)

WebmasterWorld Senior Member wilderness is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



I did try to research the user agent, but for obvious reasons came up empty handed.


Pretty much the same response as your inquiry in the Apache forum, unless you provide the a "raw visitor log" line, nobody will be able to assist you.

IMO, the best thing you could is dump the PHP script you enjoy so much, unless your able to modify it to provide useful data and/or identities.

"host name" lookups are cpu server intensive and should be avoided.

keyplyr

10:00 pm on Apr 26, 2012 (gmt 0)

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



YahooCacheSystem used to use an actual blank UA, then changed to "YahooCacheSystem" and now has been migrating to "missing"

Seems they read these boards... so soon I expect a new, improved, UA.

brokaddr

2:55 am on Apr 27, 2012 (gmt 0)

5+ Year Member



Pretty much the same response as your inquiry in the Apache forum, unless you provide the a "raw visitor log" line, nobody will be able to assist you.


The user agent is *missing*. If it were blank, I'd of said it was blank. :)

then changed to "YahooCacheSystem" and now has been migrating to "missing"

So this is a legit user agent for Yahoo? Is there a purpose for them not identifying themselves as yahoo, like they do with the Slurp agent?

And why *missing*? Is there a purpose to this particular bot/user agent?

keyplyr

3:05 am on Apr 27, 2012 (gmt 0)

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month




Is there a purpose for them not identifying themselves...

Ya just gotta read this forum. There's lots of info on Yahho Cache System and what they do with the data they mine.

And why *missing*? Is there a purpose to this particular bot/user agent?

As I alluded to above, IMO they are changing the UA to avoid blocks.

brokaddr

3:13 am on Apr 27, 2012 (gmt 0)

5+ Year Member



Found, thank you: [webmasterworld.com...]
 

Featured Threads

Hot Threads This Week

Hot Threads This Month