Challenging htaccess ban

Ban REFERER using htaccess

[emm386]

I have a site that is receiving thousands of GET requests from a referring Domain/URL that is a blank ASPX page if you go directly to it. The Domain home page is blank except for “Under Construction”.

Analytics and server log entries show the referrer as:

://obfuscated-domain.com/ieupdateobject/sk.aspx?client=obfuscated-client-number&URL=obfuscated-keyword

The IP address of the GET always varies, and appears to be actual surfers that have been erroneously redirected to our site. Interestingly enough, the keywords appended to the referrer are very relevant to our site. However, the traffic is not as they immediately bounce.

I have attempted to ban the traffic with the following htaccess entry, which does not work:

RewriteCond %{HTTP_REFERER} obfuscated-domain\.com [NC]
RewriteRule .* - [F]

I’m having a hard time wrapping my brain around what obfuscated-domain.com is trying to accomplish, but more importantly, how to ban them.

Thoughts and suggestions very much appreciated.

[jdMorgan]

Do you have any other working rewriterules in this .htaccess file?

How, specifically, does "this not work"? (The problem with "it does not work" is that the only sensible reply that can be formulated to it is, "That's because it's broken.")

Are you still seeing 200-OK responses in your raw server access log file?

Jim

[emm386]

Jim, yes there are other rewrite rules that are working, here is the current htaccess”

Options +FollowSymLinks
RewriteEngine on
RewriteCond %{HTTP_HOST} ^ourdomain.com [NC]
RewriteRule ^(.*)$ [ourdomain.com...] [L,R=301]

Sorry, by “does not work” I meant it does not block the referrer traffic. The traffic still arrives on our home page with a response code 200.

I tested this rule:

RewriteCond %{HTTP_REFERER} obfuscated-domain\.com [NC]
RewriteRule .* - [F]

by placing a link to our site on another website of ours, and I did receive a 403 when I accessed our site via the new link, so I assume our rewrite engine is functioning normally.

One other thing that has me puzzled, and may be part of the picture. If you access the referral url without the appended source string you get a blank page aspx, however if you access the referral url with the appended string you are redirected to our site.

This takes you to a blank page:
://obfuscated-domain.com/ieupdateobject/sk.aspx

This redirects you to our home page:
://obfuscated-domain.com/ieupdateobject/sk.aspx?client=obfuscated-client-number&URL=obfuscated-keyword

Thanks!
Mark

[jdMorgan]

Reading the above, I get "It works, but it doesn't work." You tested it by placing a link on one of your other sites, blocking that site as a referrer, and it worked. But conversely, it also doesn't work.

This is confusing. A thorough step-by-step description of each test, the expected results, the actual results, and an exposition on the differences between the expected and actual results is recommended as a way to get useful answers here -- We cannot "look over your shoulder" or know what you are doing and seeing without a verbose and complete description... even for a very simple problem like this one.

You may be getting fooled by your browser cache, so be sure to completely flush (delete) your browser cache between tests -- A "force page reload" often isn't enough.

Otherwise, your browser will likely display a previously-cached version of the page (or the server's error response) instead of sending a new request to your server. If it does not send a new request to your server, then of course your server-side code will have no effect on what is displayed.

This obfuscated-domain sounds like an NPH proxy to me, so it is a very good idea to block it, especially if the original request is coming from a search engine. They may be trying yet another 302-hijacking ploy -- a topic well covered here last year (try a search).

Jim

[emm386]

Thanks! I Completely flushed cache, then reloaded the htaccess file as follows:

Options +FollowSymLinks
RewriteEngine on
RewriteCond %{HTTP_HOST} ^ourdomain.com [NC]
RewriteRule ^(.*)$ [ourdomain.com...] [L,R=301]

RewriteCond %{HTTP_REFERER} obfuscated-domain\.com [NC]
RewriteRule .* - [F]

Traffic from the referrer is now receiving a 403 page.

I'd still like to figure out what they were up to, I’ll research NPH proxy/302 information.

Thanks again Jim.
Mark

[jdMorgan]

You might be happier with the more-specific:

Options +FollowSymLinks
RewriteEngine on
#
RewriteCond %{HTTP_REFERER} ^https?://([^./]+\.)*obfuscated-domain\.com [NC]
RewriteRule ^ - [F]
#
RewriteCond %{HTTP_HOST} !^(www\.example\.com)?$
RewriteRule ^(.*)$ http://www.example.com/$1 [R=301,L]

This redirects any requested hostname that is not *exactly* "www.example.com" to the canonical domain, and blocks referrers where the obfuscated domain is the referrer (and not, for example, just part of the query string).

Unlike your original, this domain canonicalization mod covers https and non-canonical case errors, FQDN-format hostnames, and appended port numbers.

As shown, it also allows for blank Host headers (e.g. from true HTTP/1.0 clients).

Finally, since there's little use in redirecting non-canonical referrals from obfuscated-domain, I reversed the rule order. Other tweaks for efficiency/standardization only.

Jim

[emm386]

Just installed it.
Many thanks Jim - That’s above and beyond that call of duty. I owe you one.
Mark