Can't block reverse.theplanet?

Trying to block them with htaccess and they come right through

[Jon in Michigan]

Hi,

I'm really new to using htaccess and I'm trying to stop the barage of hits to my site by reverse.theplanet.com. I've been using an .htaccess file to block comment and referer spammers for quite a while with pretty good success.

But reverse.theplanet.com seems to come right through without issue.

For example, I get a hit from DNS = 18.70-86-95.reverse.theplanet.com, IP = 70.86.95.18.

In my .htaccess file I have tried the following combinations:

SetEnvIfNoCase User-Agent "^reverse.theplanet.com*" banned
order allow,deny
allow from all
deny from env=banned

<Limit GET POST HEAD>
order deny,allow
allow from all
deny from 70.86.95.18
deny from reverse.theplanet.com
</Limit>

RewriteCond %{HTTP_USER_AGENT} .*reverse.theplanet.* [NC,OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?reverse.theplanet.*$ [NC,OR]
RewriteCond %{REMOTE_HOST} ^(http://)?(www\.)?.*(-¦.)reverse.theplanet.com(-¦.).*$ [NC,OR]
RewriteCond %{REMOTE_HOST} reverse\.theplanet\.com [NC,OR]
RewriteCond %{HTTP_USER_AGENT} reverse\.theplanet\.com [NC,OR]
RewriteCond %{HTTP_HOST} ^http://[0-9a-z_.\-]*reverse.theplanet.com[NC,OR]

Its very frustrating. If anyone has any suggestions for this I would very grateful. Thanks!

[jdMorgan]

Jon,

Welcome to WebmasterWorld!

An important point, if you're not aware of it, is that blocking by using .htaccess is not going to stop these requests from being logged by your server. What should change is the response to those requests. Instead of a 200-OK, they will get a 403-Forbidden. I'm bringing this up because some of your code should have worked, but maybe the results are not what you expected.

Another point is that you'll need to identify whether these are requests from reverse.theplanet, or if they are referrals from pages of sites hosted on reverse.theplanet. In the former case, it is reverse.theplanet servers or ISP service subscribers attempting access, while in the second case, it is people visiting reverse.theplanet, and being referred (by links) to the pages, images, or scripts on your site. This determines the server variable that must be tested in order to deny access.

It is doubtful (though I'm not sure) that these visitors are using a browser called "reverse.theplanet" so I'm pretty sure that you should not be using HTTP_USER_AGENT to block requests. You're most likely going to be using the REMOTE_HOST or HTTP_REFERER variables.

This mod_access should work for requests from reverse.theplanet servers and ISP service subscribers:

Order Allow,Deny
Allow from all
Deny from reverse.theplanet

This mod_setenvif and mod_access code should work if visitors are referred (linked) from reverse.theplanet:

SetEnvIfNoCase Referer reverse\.theplanet\.com banned
Order Allow,Deny
Allow from all
Deny from env=banned

The following mod_rewrite code should work for either:

Options +FollowSymLinks
RewriteEngine on
RewriteCond %{HTTP_REFERER} ^http://(www\.)?reverse\.theplanet [NC,OR]
RewriteCond %{REMOTE_HOST} reverse\.theplanet\.com [NC]
RewriteRule .* - [F]

Note that in this last code block, a RewriteRule is required in order to actually do anything, and RewriteConds make the RewriteRule act conditionally. Also, the last RewriteCond must never have an [OR] flag on it. Some hosts don't allow use of mod_rewrite, and the Options directive at the top may not be needed on some servers.

All of this code would need to be modified if you use a custom 403 error document; The code would need to be modified to allow the server to serve that error document, even to blocked remote hosts and referrers. Otherwise, you'l get a loop: The access is blocked, the server tries to serve the custom 403 error document, and that access is blocked, so it tries to serve the custom error document, ad infinitum. I don't want to confuse the subject before you get it working, just be aware that you may want to disable your 403 custom error document (if any) while you are trying to get this working.

Comment: Blocking by Remote_Host is horribly inefficient. In order to do this, your server must perform a double reverse-DNS lookup. This means is actually has to send a request to the DNS system (DNS servers) and wait for a response before it can decide what to do. Any delay in the DNS system, or worse--a failure) will affect your server's response time. So if possible, block by IP address range instead of by hostnames. If you must block by hostnames, then try to limit the scope of the directives: Do you really need to block all files, as the code above will do, or can you narrow the scope to a smaller number of URLs/files that need to be protected? Anything you can do to reduce the number of reverse DNS lookups will improve your server performance and reliability.

Note the the Apache variables for referrers are indeed misspelled - Use variables named "referer" to block referrers.

For more information, see Apache mod_rewrite [httpd.apache.org], mod_access [httpd.apache.org], and mod_setenvif [httpd.apache.org]. There are also some good references cited in our forum charter [webmasterworld.com], and a couple of tutorials in the Apache Forum section of the WebmasterWorld Library [webmasterworld.com]. It's worthwhile to read the docs in order to save time and frustration; The chances of success without this research are pretty much zero.

Jim

[Jon in Michigan]

Thanks, Jim. It does seem odd that some of those should have worked.

I did some digging and elimination of lines in the .htaccess file to figure out what was going on. I put in a block from another page I run and found that I could refer right on in despite the block in the htaccess file for it. So I went back and deleted EVERYTHING except that line and it blocked me - but with a *custom error age*. I think the guy who runs the server must have added it recently. I wonder if it is causing problems. Hmmmm.

Thanks for your help and links to some of the tutorials. I'll go back and read through them again and hopefully I can straighten it out.

[jdMorgan]

It may be soemthing as simple as the order of the new code before/after/within the new code. The code will need to be located near the beginning of any access-control or URL-modification code in the existing file. Otherwise, a preceding directive may take action on the requested URL before the new access-control code even gets to run.

Jim

[Jon in Michigan]

Curiouser and curiouser.

I found a typo and fixed it but the problem still stayed until I accidently left [NC,OR] in the last line when I just wanted [NC]. Then it worked! This is crazy.

Still reading the tutorial...