pt-PT

[trintragula]

Anyone know what this is?

Mozilla/5.0 (Windows; U; Windows NT 5.1; pt-PT; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2 (.NET CLR 3.5.30729)

I've seen them from dozens of countries and some notorius server farms - some 300 distinct IPs in the last year. Which suggests a botnet. I even saw it from Portugal once. Portugese may be the language of Brazil, but they tend to use pt-BR...

All of today's (6) requests have a query string like:

topic=NNNN.0\'A=0

where I've replaced the actual topic numbers by NNNN. I would associate the escaped quote with a SQL injection attempt (perhaps a probe?), but it doesn't seem to be followed up.

Always auto-refers. Some missing headers.

The combination of Firefox and .NET seems unlikely in itself, but also appears in:

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2 (.NET CLR 3.5.30729; Diffbot/0.1; +http://www.diffbot.com)

Mozilla/5.0 (Windows; U; Windows NT 5.1; en; rv:1.9.0.13) Gecko/2009073022 Firefox/3.5.2 (.NET CLR 3.5.30729) SurveyBot/2.3 (DomainTools)

Mozilla/5.0 (Windows; U; Windows NT 5.1; ru; rv:1.9.0.19) Gecko/2010031422 Firefox/3.0.19 YB/3.5.1 (.NET CLR 3.5.30729)

Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 GTB6 (.NET CLR 3.5.30729)

in my robot dustbin.

The rv:1.9.1.2 is the same as DiffBot, which is a clue - reading back they apparently they have a history of using a lot of different UA strings, though this one is very samey.
Maybe there's some shared lineage there.

If I was blocking by user agent, this one would definitely be a candidate.

[lucy24]

If I was blocking by user agent

Well, even _I_ block FF < 3.6 :) (Technically, I only block 1 and 2. FF 3.0-3.5 and FF 5-8 get sent to an Old Browsers page where they can do no harm. Don't know what the deal is with 3.6 and 4, but some humans remain attached.)

[Pfui]

If I was blocking by user agent,

Why not block by UA in addition to using other safeguards?

In my experience, if a Real Person in Real Time uses any UA version less than --

MSIE 6
Firefox/24
Chrome/22

-- they need to upgrade or e-me before visiting. Sound tough? Nah. The vaaaaast majority of ancient browsers I see stating those versions are either bad bots or fakes/infecteds.

FWIW, also waylaid are ALL UAs containing the following language indicia:

ru <= includes 3 permutations: ru-ru ru; ru)
zh <= includes 2 permutations: zh-tw zh-CN

[trintragula]

I do use a few basic user agent patterns to catch things that actually look like bots - I just don't have a list. Things that look like Browsers get the benefit of the doubt - but only from that specific trap. Stealth bots will usually get caught other ways, even when they use more recent browser versions (and they do...)

A quick query shows 10 hits today from Firefox browser user agents less than version 9 (and ignoring the google favicon bot):
3 hits from pt-PT - the subject above - blocked by headers
1 hit from FF/5 - blocked by headers
1 hit from FF/3.1b2 - blocked by headers
3 hits from a spammer using FF/1.5.0.1. Spammers are often careful with headers, but fail in other ways.

and...

2 hits from

Mozilla/5.0 (Windows; U; Win98; en-US; rv:1.8.1.20) Gecko/20081217 Firefox/2.0.0.20

from a dsl/cable address in Australia (which is not an unlikely location for my site). Plausible URLs and refers from a search engine, with SE query terms that are right on for my site, and reaching two relevant pages - with believable time spacing. I think this was probably a human, though I would have thought anything that still runs Win98 would have a fan that squeaks. Maybe they're an otherwise innocent human who just likes pretending to use an old browser to see when they get stopped.

While my site is English-speaking and almost completely dominated by the half-dozen largest english speaking countries, I get real humans visiting from pretty much everywhere on the planet. Including Russia, and possibly China, though the latter is not confirmed.
I had someone sign up the other week from an African country - who turned out to be from England - was just traveling on business and signed up when he had a wifi signal available.

I've had someone log in since Christmas with FF/16, and generally I'd rather not upset my actual target audience if I can avoid it.

[Pfui]

Sounds like you're on top of things. Good.

I err on the if-in-doubt-don't side because I've got going on a half-million archived posts to protect that are seen as tantalizing, low-hanging fruit by both 'good' and bad bots. And then there are the usual site whackers, copyright violators, fckeditor attackers, WordPress exploiters, and on and on. Sigh.

[trintragula]

Your site is about 10x bigger than mine. It's definitely good to hear about what people need to do when their sites get bigger. Doubtless different issues start to dominate. Raising the bar for entry seems reasonable.
Valid use of old browsers is pretty rare, and there are other good reasons to encourage people to upgrade - not least the compatibility millstone. I dread to think what my site looks like on FF2.

[dstiles]

General notes to above...

Windows NT 5.1 is XP and is now officially obsolete. There are still people out there using it but it is now a serious health risk.

MSIE 6 has been obsolete for so long it's been found in tar pits! A secondary factor is that MSIE 6 does not support SSL above a certain level, that level now being removed from security-concious servers so MSIE 6 will not be able to access security-concious SSL sites (such as mine).

As a matter of course I now block: Firefox/[01245789]\.|Firefox/1[1345689]\.

Why do I not block V 3, 6, 10, 12, 17? Customers of an important site are still stupid enough to use them; although they often get blocked for other infringements.

Ok, and... V 12 is the latest version that runs on Windows 2000, two of which I still occasionally use to access my own sites. :)

V3.6 is common on old linux OSs that are never updated. I think the others are in similar circumstances on linux, mac and windows. Very dangerous and can be killed by so many things - anything before the newest browsers can be! (All four major browsers were hacked at an official hacking symposium last week - Firefox issued a new version a couple of days later).

Below are just a few important URLs to scare you...

[community.qualys.com...]
[community.qualys.com...]
[community.qualys.com...]
[community.qualys.com...]
[community.qualys.com...]
[scotthelme.co.uk...]

Although seriously Windows client oriented, some of the above can also kill linux and Mac!

[lucy24]

As a matter of course I now block: Firefox/[01245789]\.|Firefox/1[1345689]\.

Firefox/([01245789]|1[13-69])\.

Well, you knew I'd say it.

What's the deal with FF 6? Or, for that matter, 10 and 12?

I meet the occasional human using FF3.6 (including unmodified Camino, though personally I lie in this part of the UA string), but nothing in the 3\.[0-5] range. FF 3\.[789] don't seem to exist.

[trintragula]

I suspect there may be XP boxes with squeaky fans for a while yet, even after the POS and ATM systems are upgraded.

I do actually block MSIE 6 explicitly - every rule has to have an exception, and this is one I put in very early and have never seen fit to remove.

Firefox 21 and 22 seem to have been used a lot as cover for downloaders and botnets.

Security always scares me. I remember when we thought DES was the bee's knees.
I dare say Carrier Grade NAT is a problem for anyone security conscious. In the UK I gather some ISPs are trying to put off IPv6 by making customers share IPv4 address simultaneously :( Brings the old "party line" phone service of the seventies to mind - which also had security issues...


I have been seeing ...

Firefox/10.0 (Chrome)

shadowing a much more recent chrome browser. More than once. I'm not sure what's going on there. Maybe a plugin downloader of some kind?

[lucy24]

:: detour to raw logs ::

Yikes! Perhaps I should have constrained that search to a more recent time period, after people really did stop using FF 10! Although I do see a human FF10 as recently as last year, mixed in with a ###load of unwanted robots.

But I see what you mean. F'rinstance (tag end of a human visit)

95.252.88.abc - - [28/Feb/2015:16:54:45 -0800] "GET /fonts/images/keyboards/greek_opt.png HTTP/1.1" 200 2265 "http://example.com/fonts/custom_greek.html" "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.115 Safari/537.36" 
95.252.88.abc - - [28/Feb/2015:16:54:46 -0800] "HEAD /fonts/custom_greek.html HTTP/1.1" 200 293 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20100101 Firefox/10.0 (Chrome)" 
95.252.88.abc - - [28/Feb/2015:16:54:47 -0800] "GET /fonts/custom_greek.html HTTP/1.1" 200 5695 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20100101 Firefox/10.0 (Chrome)" 
95.252.88.abc - - [28/Feb/2015:16:54:48 -0800] "GET /favicon.ico HTTP/1.1" 200 662 "-" "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.115 Safari/537.36" 

Next time you notice one, see if they too started with the HEAD.

[trintragula]

It looks like I'm seeing a similar pattern:

[22/Mar/2015:12:45:55 -0700] GET example.com/index.php?board=7.30 HTTP/1.1 200 15289 http://example.com/index.php?board=7.0 Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.101 Safari/537.36
[22/Mar/2015:12:45:59 -0700] HEAD example.com/index.php?board=7.30 HTTP/1.1 200 364 - Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20100101 Firefox/10.0 (Chrome)
[22/Mar/2015:12:46:00 -0700] GET example.com/index.php?board=7.30 HTTP/1.1 200 13897 - Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20100101 Firefox/10.0 (Chrome)
[22/Mar/2015:12:46:08 -0700] GET example.com/index.php?topic=284.0 HTTP/1.1 200 22272 http://example.com/index.php?board=7.30 Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.101 Safari/537.36
[22/Mar/2015:12:46:10 -0700] HEAD example.com/index.php?topic=284.0 HTTP/1.1 200 530 - Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20100101 Firefox/10.0 (Chrome)
[22/Mar/2015:12:46:11 -0700] GET example.com/index.php?topic=284.0 HTTP/1.1 200 20761 - Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20100101 Firefox/10.0 (Chrome)
[22/Mar/2015:12:46:12 -0700] HEAD example.com/index.php?board=7.30 HTTP/1.1 200 364 - Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20100101 Firefox/10.0 (Chrome)
[22/Mar/2015:12:46:13 -0700] GET example.com/index.php?board=7.30 HTTP/1.1 200 13897 - Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20100101 Firefox/10.0 (Chrome)

_{(sorry, I lost the quotes somewhere....)}

This is from a DSL/cable address.
Sometimes the followup that looks like FF10 will be delayed. Or repeated.
I think there are quite a few things like this that are browser plugins or toolbars and sometimes malware.

[dstiles]

Lucy - yes, but I think my way is probably faster under IIS. :)