.htaccess and RewriteCond

How to ban USER_AGENT empty

[Maleville]

HI!

I have a .htaccess file at the root of my web site which ban bad robots as:

RewriteEngine on
RewriteCond %{HTTP_USER_AGENT} .*eCatch* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*EmailCollector* [NC,OR]
# and so on
RewriteCond %{HTTP_USER_AGENT} .*Zeus*
RewriteRule ^.*$ [other-domain.com...] [L,R]

which works properly.

I would want to ban {HTTP_USER_AGENT} which are empty, without identification.

How can I do
Tkank you if some one can aswer.

[jdMorgan]

Block blank user-agent:

RewriteCond %{HTTP_USER_AGENT} ^$ [OR]

Block "faked" blank user-agent:

RewriteCond %{HTTP_USER_AGENT} ^-$ [OR]

Combined:

RewriteCond %{HTTP_USER_AGENT} ^-?$ [OR]

Ref: Introduction to mod_rewrite [webmasterworld.com]

Jim

[Maleville]

Thank you very much jdMorgan. I am going to try this trick.
But what do you mean by "faked" blank user-agent. Have you an exemple?

[jdMorgan]

Maleville,

> But what do you mean by "faked" blank user-agent. Have you an exemple?

No, I can't show you a log file example, because that is the trick.

Here's what a blank referrer looks like in the logs ... "-"
But what if the referer is non-blank and actually consists of a string equal to "-"? It will then pass through blank-referer blocks, but look like a blank referer in the logs!

Jim

[Maleville]

Thank you for the explanation.
But, is it possible that a "faked" blank user-agent looks like this "_" or "¦" or "=" and so on? Is the "-" sign the obliged way? If it is not the case can we and how to make a string composed with strange signs?

[jdMorgan]

Maleville,

I don't know about using other characters, but most of them would not be legal in a URL.

The essence of the "faked blank referrer" using "-" is that it is not blank, but is displayed in exactly the same way that a truly-blank referrer would be displayed in raw log files.

Jim

[Maleville]

Thank's for the answer.

[keyplyr]

What would be some of the pros and cons of blocking empty and faked-empty UAs?

[jdMorgan]

keyplr,

Almost all blank UA's are troublemakers. All known instances of the UA = "-" faked blank referer have been troublemakers.

I don't block blank referers because of the proxy/firewall/internet security software/browser settings issues, but I do invoke a more restrictive set of rules if the UA and referer are both blank - that is a red flag. The faked blank referer will get a 403 if I ever see one.

This ploy was brought my my attention by a problem posted here last year: It appeared that the poster's blank referrer block was failing, because he blocked UA = ^$ but the requests were still being honored as evidenced by his raw logs showing a 200-OK response for a UA of "-", which is the standard log file entry for a blank UA. In a rather inspired leap of imagination, he asked himself, "Yeah, but what happens if the UA is actually a hyphen?" With a simple test using a spoofed UA of "-", he proved that this was the case - the log entry looked the same as that for a blank UA, but showed a 200-OK response.

Jim