Welcome to WebmasterWorld Guest from 54.205.88.118

Forum Moderators: Ocean10000 & incrediBILL

Strange user agent

   
9:08 am on Feb 23, 2012 (gmt 0)



Hi all,

New here, but been lurking for years. Thanks to all who post and have pre-answered so many of my questions without me needing to sign up!

I have a user agent claiming to be 'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)' hitting one of my sites search page with queries every 5 mins or so, 24/7. Ip's too numerous to mention, but seldom are any *blacklisted* anywhere. It's always searching for people by name, and never seems to go further than the search page (which only lists relevant pages, and doesn't display content).

Does anyone have a clue what this entity might be? I've put a sample of IP's, times and search terms at <snip>

Kev

[edited by: incrediBILL at 10:17 am (utc) on Feb 23, 2012]
[edit reason] NoPersonal URLs Please, include all data in post [/edit]

10:29 am on Feb 23, 2012 (gmt 0)

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month





Because of the numerous IPs and the frequency, this could be a botnet - many infected machines running a scheduled task.
10:52 am on Feb 23, 2012 (gmt 0)



Thanks, I did wonder about that. I guess there's little point guessing exactly what it's up to with all those names and variants - last seen searching for ...

Hayes richardson
Hayes richardsson
Hayes richerdson
Hayes richardsan
Hayes richardsen

... from 99.108.182.212 (it uses a different IP for each name and it's variants)

Kev
11:04 am on Feb 23, 2012 (gmt 0)

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Also, looking through my notes I found that almost all IPs using "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)" were bots and a large percentage came from either China, Russia or Asia Pacific. Personally, I block all China ranges on principal :)
12:34 pm on Feb 23, 2012 (gmt 0)

WebmasterWorld Senior Member wilderness is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Also, looking through my notes I found that almost all IPs using "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)" were bots and


RewriteCond %{HTTP_USER_AGENT} 5\.1\)$
8:39 pm on Feb 23, 2012 (gmt 0)

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month





I block these by other methods.
9:10 pm on Feb 23, 2012 (gmt 0)

WebmasterWorld Senior Member wilderness is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



you using a big fly swatter ;)
9:23 pm on Feb 23, 2012 (gmt 0)

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Who me? What "fly swatter" are you referring to?
9:35 pm on Feb 23, 2012 (gmt 0)

WebmasterWorld Senior Member wilderness is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



What "fly swatter" are you referring to?


I block these by other methods.
10:33 pm on Feb 23, 2012 (gmt 0)

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Sorry wilderness, I can't understand you.
1:12 am on Feb 24, 2012 (gmt 0)

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



Sorry wilderness, I can't understand you.


I can. (w himself will testify that this is not always the case.) If you don't block by UA, what do you block by?

fwiw, I recognized that UA instantly. I call it "MSIE generic". It's used by, among other things, all Chinese robots that don't have names-- and a few that do. Kinda doubt any human would use it.
1:27 am on Feb 24, 2012 (gmt 0)

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



If you don't block by UA, what do you block by?

As previously stated, I block China and most of the Asia Pacific by IP, so that covers most all using this UA. The few others I get with that UA are usually already blocked by IP. Occasionally, there are a few *legit* users like library's or military. These I deal with on a case-by-case basis.

A "big fly swatter?" well yeah... China is a big bug!
11:16 am on Feb 28, 2012 (gmt 0)



RewriteCond %{HTTP_USER_AGENT} 5\.1\)$


Would the above block a user with the below UA as it has "535.11" for it's Applewebkit and Safari version ? Or does the $ state that it must end at that point ?

Mozilla/5.0 (Windows NT 6.0) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.56 Safari/535.11 


Thank you
11:59 am on Feb 28, 2012 (gmt 0)

WebmasterWorld Senior Member wilderness is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Would the above block a user with the below UA as it has "535.11" for it's Applewebkit and Safari version ? Or does the $ state that it must end at that point ?


the ends with anchor (dollar sign) is the key here to make mod-rewrite only look and compare to that portion of the UA.

In your example it would be
5\.11$
35\.11$
535\.11$

Any of the three would work

\.11$ may also work, however over time, I've learned to be cautious about opening a line with an escape.
9:13 pm on Feb 28, 2012 (gmt 0)



Thank you wilderness. I was actually trying to be cautious. My poorly written question was meant to ask if by using the quoted rule I may accidentally block those other UA's but I can see from you reply that it would not.
Many thanks for your reply.
10:21 pm on Feb 28, 2012 (gmt 0)

WebmasterWorld Senior Member dstiles is a WebmasterWorld Top Contributor of All Time 5+ Year Member



Of course, that block will only work with this week's chrome browser. Next week the version numbers are likely to change (last week-ish it ended in 7).

And, of course, this will kill all versions of safari of that version number, chrome or not, unless tied to a specific IP or other header fields.
12:30 am on Feb 29, 2012 (gmt 0)

WebmasterWorld Senior Member tangor is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month



Keep it a bit more simple: MSIE 6\.0

After all, who's running that these days? Go for the nibble, not the over-large sledgehammer. .htaccess 6 and all those others lesser in value, too. These days I'm loving my 403's... and that tiny tiny (teeny tiny) 403 served and homemade script that strips those out for analysis on what remains.
2:14 pm on Feb 29, 2012 (gmt 0)

10+ Year Member



Just a heads up, if you're running NetDNA or similar content delivery network their pull cache uses this MSIE 6.0 user agent.
 

Featured Threads

Hot Threads This Week

Hot Threads This Month