Banning Site Referrals

Not working.

[Tiebreaker]

Hi

I'm using the following code in my htaccess file to try and ban referrals from a certain domain (both the .com and the .co.uk site) but it isn't working


RewriteEngine on
RewriteCond %{HTTP_REFERER} domain\.com [NC,OR]
RewriteCond %{HTTP_REFERER} domain\.co.uk
RewriteRule .* - [F]


I've tested it out by trying to ban referrals from another of my own domains - editing the code, obviously.

I place a link on that site but it works fine - I want people to see a 403 error when they click a link to my site.

Any ideas what I may be missing here?

I used a very similar code in the htaccess file which is in my images directory, to ban hotlinking - and that works perfectly.

[topr8]

are you sure you are sending referer headers?
i understand that norton (and other things) can turn them off.

also perhaps try

RewriteRule .* - [F,L]

[Tiebreaker]

Sorry, I just realised I needed to clear my cache - it was working all the time!


Just another question though - I assume if I change the code as below, using a wildcard instead of .com or .co.uk, that will block referrals from all of their different domain extensions with just the one simple line?


RewriteEngine on
RewriteCond %{HTTP_REFERER} domain\.* [NC]
RewriteRule .* - [F]

[jdMorgan]

This is a bit faster and more robust, as it specifies that the blocked domain be the referring domain, rather than just appearing anywhere in the referrer string (e.g. as part of a query string sent in a request from an "approved" domain). It's also a bit more specific, and won't block a referral from a domain whose name includes the blocked domain's name as a substring: For example, if you block "foo.com", this code won't block "not-foo.com" as well, as your code above would do.

RewriteEngine on
RewriteCond %{HTTP_REFERER} ^([^./]+\.)*domain\. [NC]
RewriteRule .* - [F]

No need to use [L] with the [F] flag, as it is already implied.

Jim

[Tiebreaker]

Thanks :-)

Just checking though - does that code block different extensions - eg domain.com, domain.co.uk, domain.ca etc?

Because I notice that the wildcard after the domain is not in your code.

[jdMorgan]

I do make errors when posting here, but they tend to be much more spectacular than that... :)

You don't need a wild-card because the pattern has no end-anchor ("$") on it...
Neither do you need a wild-card at the beginning of a pattern that is not start-anchored with a "^".

The patterns "^.*xyz.*$" and "xyz" match exactly the same thing. The second pattern is much shorter. Imagine that it costs you a dollar to parse each character in the pattern... Which will you choose?

These match processing for these two patterns is not done in exactly the same way, though: The first pattern will match the last "xyz" in the input string, while the second pattern will match the first. This is because of the "greedy" matching behavior of the ".*" pattern: The leading ".*" will initially match the entire input pattern, leaving the "xyz" and ttrailing ".*" patterns to starve. Then the matching engine will back off one character and retry the match, this time satisfying both the leading and trailing ".*" subpatterns, but still leaving the "xyz" subpattern starving. The engine will "back off" from the end of the input string one character at a time until it can satisfy all three subpatterns.

On the other hand, the "xyz" pattern is matched by starting at the beginning of the input string, and scanning forward until the first "x" is found. If the character following this "x" is not a "y", then scanning resumes to the next "x", and the "following-y" test is repeated. When both match, the next character is tested to see if it's a "z". If not, then we start scanning for the next "x" -- Lather, rinse, repeat.

When you have a simple question like this, it will likely be faster to test the code and see, rather than waiting for an answer here. You might also want to try a search for "regular expression tester" to find a good tool to answer such questions... I strongly suggest that you such tools only to test; If you rely on them to develop regex patterns, then you will never learn regular expressions thoroughly, and the result will be that you'll always have difficulty writing, reading, and understanding patterns.

Jim

[Tiebreaker]

Thanks for your help.

Yes, I should really try to learn this stuff in detail myself - another job on the "To Do" list :-)

[jdMorgan]

If you intend to use mod_rewrite, then that job should be the first on your "to-do list." Otherwise, you may make a lot of "search rankings repair" work for yourself later, or perhaps put yourself out of business...

Server config code should not be altered unless the function and side-effects are fully understood.

Jim