Strange user agent

[KevinV]

Hi all,

New here, but been lurking for years. Thanks to all who post and have pre-answered so many of my questions without me needing to sign up!

I have a user agent claiming to be 'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)' hitting one of my sites search page with queries every 5 mins or so, 24/7. Ip's too numerous to mention, but seldom are any *blacklisted* anywhere. It's always searching for people by name, and never seems to go further than the search page (which only lists relevant pages, and doesn't display content).

Does anyone have a clue what this entity might be? I've put a sample of IP's, times and search terms at <snip>

Kev

[edited by: incrediBILL at 10:17 am (utc) on Feb 23, 2012]
[edit reason] NoPersonal URLs Please, include all data in post [/edit]

[keyplyr]

Because of the numerous IPs and the frequency, this could be a botnet - many infected machines running a scheduled task.

[KevinV]

Thanks, I did wonder about that. I guess there's little point guessing exactly what it's up to with all those names and variants - last seen searching for ...

Hayes richardson
Hayes richardsson
Hayes richerdson
Hayes richardsan
Hayes richardsen

... from 99.108.182.212 (it uses a different IP for each name and it's variants)

Kev

[keyplyr]

Also, looking through my notes I found that almost all IPs using "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)" were bots and a large percentage came from either China, Russia or Asia Pacific. Personally, I block all China ranges on principal :)

[wilderness]

Also, looking through my notes I found that almost all IPs using "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)" were bots and

RewriteCond %{HTTP_USER_AGENT} 5\.1\)$

[keyplyr]

I block these by other methods.

[wilderness]

you using a big fly swatter ;)

[keyplyr]

Who me? What "fly swatter" are you referring to?

[wilderness]

What "fly swatter" are you referring to?

I block these by other methods.

[keyplyr]

Sorry wilderness, I can't understand you.

[lucy24]

Sorry wilderness, I can't understand you.

I can. (w himself will testify that this is not always the case.) If you don't block by UA, what do you block by?

fwiw, I recognized that UA instantly. I call it "MSIE generic". It's used by, among other things, all Chinese robots that don't have names-- and a few that do. Kinda doubt any human would use it.

[keyplyr]

If you don't block by UA, what do you block by?

As previously stated, I block China and most of the Asia Pacific by IP, so that covers most all using this UA. The few others I get with that UA are usually already blocked by IP. Occasionally, there are a few *legit* users like library's or military. These I deal with on a case-by-case basis.

A "big fly swatter?" well yeah... China is a big bug!

[Seedy]

RewriteCond %{HTTP_USER_AGENT} 5\.1\)$

Would the above block a user with the below UA as it has "535.11" for it's Applewebkit and Safari version ? Or does the $ state that it must end at that point ?

Mozilla/5.0 (Windows NT 6.0) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.56 Safari/535.11 

Thank you

[wilderness]

Would the above block a user with the below UA as it has "535.11" for it's Applewebkit and Safari version ? Or does the $ state that it must end at that point ?

the ends with anchor (dollar sign) is the key here to make mod-rewrite only look and compare to that portion of the UA.

In your example it would be
5\.11$
35\.11$
535\.11$

Any of the three would work

\.11$ may also work, however over time, I've learned to be cautious about opening a line with an escape.

[Seedy]

Thank you wilderness. I was actually trying to be cautious. My poorly written question was meant to ask if by using the quoted rule I may accidentally block those other UA's but I can see from you reply that it would not.
Many thanks for your reply.

[dstiles]

Of course, that block will only work with this week's chrome browser. Next week the version numbers are likely to change (last week-ish it ended in 7).

And, of course, this will kill all versions of safari of that version number, chrome or not, unless tied to a specific IP or other header fields.

[tangor]

Keep it a bit more simple: MSIE 6\.0

After all, who's running that these days? Go for the nibble, not the over-large sledgehammer. .htaccess 6 and all those others lesser in value, too. These days I'm loving my 403's... and that tiny tiny (teeny tiny) 403 served and homemade script that strips those out for analysis on what remains.

[motorhaven]

Just a heads up, if you're running NetDNA or similar content delivery network their pull cache uses this MSIE 6.0 user agent.