SNI and hostname Different

[yaashul]

I get following error whenever this useragent try and access my site.

IP_ADDRESS - - [05/Oct/2014:10:56:43 +0530] "GET / HTTP/1.1" 400 4845 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; CrystalSemanticsBot [crystalsemantics.com...]

Error -

Hostname IP_ADDRESS provided via SNI and hostname www.example.com provided via HTTP are different


I am using Apache 2.2.x (redhat 6.5) and my website on SSL. I tried blocking this bots but It always give http 400 (it shud be 403).


Any suggestions?

[not2easy]

They are still being blocked, but as to why, it may happen when a visitor is using a proxy. In case you wondered, a 400 response = Bad Request - there is a syntax error in the request, it is denied.

[lucy24]

Are you on shared hosting? It's possible that whatever the hosts do to generate a 400 lockout happens before the request reaches your htaccess, so their error kicks in before yours gets a chance.

For that matter, it can happen in your own server too. You just need to figure out what's generating the 400 error. (That is, what module or what stage in request handling.) Don't try to stop it; it's just useful to know.

You generally think of host-level lockouts as creating a 500-class error. But mine uses a 418 with mod_security. It looks pretty droll, but it does make it easier to distinguish intentional lockouts from bona fide errors.

[wilderness]

But mine uses a 418 with mod_security.

I'm thinking my host uses a 406, however they are so infrequent that I could be mistaken.

[yaashul]

I m on dedicated server.

[not2easy]

I see that Apache offers a wiki that covers this topic and you may find what you need there: [wiki.apache.org...]

Using name-based virtual hosts with SSL adds another layer of complication. Without the SNI extension, it's not generally possible (though a subset of virtual host might work). With SNI, it's necessary to consider the configuration carefully to ensure security is maintained.

They discuss this type of problem and solutions including server requirements.

I kind of wonder (if you aren't having issues with other UAs), whether the

Mozilla/4.0 (compatible; MSIE 6.0;

antiquity might be one of the problems. Unless you can get the bot to upgrade their UA, they are probably unable to connect to get a 403.

[lucy24]

:: detour to confirm hunch ::

The exact UA string

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)

is one of my mod_security triggers. In fact it's almost the only one I can find in recent days, except one case of the bare

Mozilla/5.0

and-that-was-all-she-wrote.

:: further study of error logs ::

Hm, now that's interesting. That particular offender made 15 requests, of which 13 made it to error logs. This one turned out to be part of the "nyet.gif" botnet, where they first try to PUT a file and then GET the same one. So the sequence goes:

403 for PUT method
404 for "nyet.gif"
two 418 for request matching "ofc_upload_image.php"
301 for "index.html"
two 403 for request ending in .php
418 for request matching very long name pattern that I won't paste in
two 403 for request ending in .php
404 again for "nyet.gif"
two more 418 for "ofc_upload_image.php"
301 for "index.html"
and a final 403 again for request in .php

It's illuminating to see just how many different ways a request can fail even when it's an unknown IP (this one turned out to be from Hong Kong, which is not yet subject to my universal China lockout, though it probably should be).

A 400 may be a smidgen less work for the server, because most humans don't generate 400 errors so there's no need for a custom error page. (Yes, contrary to widespread belief it is possible for a legitimate human with no ill intentions to get a perfectly natural 403.)

[yaashul]

I don't have problem with this UA Mozilla/4.0 (compatible; MSIE 6.0;

All other user agent and browser with ie6 working fine.

Lucy24 I also use Mod_Security.

[wilderness]

I don't have problem with this UA Mozilla/4.0 (compatible; MSIE 6.0;

You should.
It's a malformed UA and non-standard.

[yaashul]

I mean not with MSIE 6.0 user agent.

[wilderness]

The example that lucy provided:

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)

Is a malformed UA, and is used notoriously.

A simple "ends with SV1" kills that.