Blocking blocked referrers

[maxgoldie]

I have a lot of blocked referrers in my logs. Is there a way to redirect blocked referrers?

[jdMorgan]

I'm not sure I understand your question.

You can send a redirect response to blocked referrers, but most of them won't bother to follow a redirect (It is up to the client to decide to follow a redirect or not).

Perhaps what you are asking is if you can get rid of these entries in your log files. In that case, the answer is yes if you have server config privileges and no otherwise. If you have httpd.conf access, you can set up custom logging (see Apache mod_log_config), and just drop those 403 log entries.

If you have no admin privileges, then the best you can do is reduce the bandwidth wasted on these requests by sending a very short 403-Forbidden response. You can do this as follows:

Create a subdirectory and place a very short custom 403 page in it.

Create a .htaccess file in that subdirectory that declares this 403 page as the ErrorDocument for that subdirectory.

Also in that .htaccess file, forbid access to anthing other than the custom 403 page in that subdirectory.

In your main .htaccess file, rewrite (not redirect) all unwelcome requests to that subdirectory.

In this way, any unwelcome request is rewritten to the subdirectory, but all files except the 403 error page in that subdirectory are forbidden, so the error handler is invoked, and serves the very-short 403 error page.

I hope that helps.

Jim

[maxgoldie]

So, in the case of the last suggestion, the visitor (who is a blocked referrer) is redirected to a 403 page? Or does this just fool the logs into not recording it?

[jdMorgan]

No, the visitor is served a very short 403-Forbidden response, which minimizes your "bandwidth loss," but the entry will still appear in your logs.

Basically, the only way to avoid logging is to use mod_log_config in httpd.conf, or to block the request at the firewall by IP address, before it even reaches your server.

I suppose you could also write a script to remove these entries from your raw logs, or modify your 'stats' script to ignore them. Depends on what the end-goal is.

Jim

[Key_Master]

You can save some additional bandwidth and not have to create a custom 403 page by entering the following line in your .htaccess file.

ErrorDocument 403 "403 Forbidden"

The visitor will simply see the short message in quotes.

[maxgoldie]

A lot of the referrers I get are blocked by Agnitum and Norton. If I put something like this:

RewriteCond %{HTTP_REFERER} (Field\sblocked¦ProxyWay¦Outpost¦Agnitum¦anonym¦steganos¦Norton) [NC]
RewriteRule!^.*\.403\.php$ - [F,L]

would that do the trick?

[jdMorgan]

Yes, probably, but you don't need [L], it's implicit with [F], and your pattern can be shortened for faster processing:

RewriteRule !\.403\.php$ - [F]

Jim

[topsites]

I have two things to say regarding this topic...
My first question is I would like to know how to ban a referrer based on a keyword rather than an entire domain name. The reason for this is I got a slew of spamrefs and the list got too long to be practical... However, using keyword-based banning would shorten the list dramatically as most of these domains belong to few people, meaning they registered a slew of similar-named domains, ALL of which have certain words in common.
An example of a flexible ban-command would be:
RewriteCond %{HTTP_REFERER} ^http://(www\.)?spamdomainer.com
Which makes the 'www' optional and doesn't require one listing for each (one with, the other without the w's). But, it's not complete towards keyword-based and my knowledge in the area is limited.
Any ideas here would be appreciated.

For the re-direct issue, it can be done but where to re-direct it to? Redirect the hits to someone legit like Google, and that's really uncool... There is a way to bounce the hits back to the spammer, but I did this once and next thing I knew, they took revenge on me and I am not sure how it was done but the entire .htaccess procedure was bypassed and I really had my hands full emptying my access log daily, line by line. In the meantime, I lost the command-line but I speak from experience when I say the best thing to do is simply absorb the hits.

In the end I also disabled the 'origin' part of my public statistics, while it leaves my stats page minus one function, it also gives me peace and quiet as I no longer have to worry about this issue. I am sorry, but it appears to be the only permanent solution to the headache.

Peace out.

[maxgoldie]

My whole problem is similar to this, I get a lot of "blocked referrer" spam and "Field Blocked By Outpost", and stuff like this:
user agent string: XXXX:+++++++++++++++++++

So I just blocked all of the empty referrers like this:

SetEnvIf Referer ^$ empty_referer
Deny from env=3Dempty_referer

Hopefully, that will take care of that pain Norton (NIS) and Agnitum/Outpost

Then I blocked all that ++++++++++ noise like:

# An empty string or a dash is not a valid UA string.
RewriteCond %{HTTP_USER_AGENT} ^-?$
RewriteRule!\.403\.php$ - [F]
# ------------------------------------------------------------------------------
# A user-agent (or a referer) which string is :
# XXXX:+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
# and another which is a series of dashes
RewriteCond %{HTTP_USER_AGENT} ^xx+:\++$ [NC,OR]
RewriteCond %{HTTP_REFERER} ^--+$ [NC,OR]
RewriteCond %{HTTP_REFERER} ^xx+:\++$ [NC]
RewriteRule!\.403\.php$ - [F]

I have to do this, the amount of spam lately is crazy. I consider all those who come to my site as blocked referrers either are spam, or have something to hide.