Forum Moderators: open

Message Too Old, No Replies

Puzzled by this UA behavior

         

aristotle

1:34 pm on May 25, 2014 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



The following type of Latest Visitor log entry has been appearing occasionally in the logs of one of my sites:
Host: 96.19.157.44
/example-page.html
Http Code: 200 Date: May 23 20:22:21 Http Version: HTTP/1.0 Size in Bytes: 10830
Referer: [us.yhs4.search.yahoo.com...]
Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko/20100101 Firefox/12.0

/example-image.jpg
Http Code: 200 Date: May 23 20:22:21 Http Version: HTTP/1.0 Size in Bytes: 14212
Referer: http://www.example.com/example-page.html
Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko

Here is what I notice:
1. It happens with different IP's from various U.S. internet providers.

2. It's always a Yahoo search.

3. The page fetch UA is different from the image fetch UA -- The page fetch UA always has "/20100101 Firefox/12.0" appended, but the image fetch UA doesn't.

4. It's always Firefox/12.0

These seem to be real visitors, but it's very suspicious. Does anyone have an explanation?

keyplyr

2:05 am on May 26, 2014 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Sounds like droned machines (botnet) being used to harvest, in your case maybe image files. Here's why I say this:

1.) Http Version: HTTP/1.0 increases chance it is a bot being used as harvester... however there are some (usually old) platforms still using HTTP/1.0. A lot of webmasters just block HTTP/1.0 and forget about it, depending on your niche, not too much collateral damage.

2.) Every time I have investigated where the UA changes with different file types, it was a bot.

Another possibility: a network caching agent (either private ((example: corporate)) or public) but if these hits come from various networks, I still say they are droned machines (botnet.)

aristotle

10:50 am on May 26, 2014 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Thanks for the reply. At first I thought they were real visitors because they appear to come from Yahoo search. But that must be faked somehow.

keyplyr

7:00 pm on May 26, 2014 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month




UA and Referrer are easily spoofed when writing a bot.

lucy24

8:13 pm on May 26, 2014 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



I did a search for the exact string
Windows NT 6.1; WOW64; Trident/7.0; rv:11.0
Hadn't noticed it before, but it's got a disproportionate number of non-standard* search engine referers. Interesting.


* i.e. rarely Google or Bing.