Welcome to WebmasterWorld Guest from 54.145.209.107

Forum Moderators: Ocean10000 & incrediBILL & phranque

REFERER Overkill?

REFERER, Overkill, block

   
3:24 am on Mar 2, 2014 (gmt 0)



REFERER Overkill?

Working on a new site that Starts w/ .fr the REFERER zaps the CSS in IE only...

I always thought it blocked the domains ending in .fr?

I have also seen some iPads & sometimes rr.com, comcast.net & myvzw.com being blocked.


# Referrer Blocker - .se search bug?
RewriteEngine On
RewriteCond %{HTTP_REFERER} \.at [NC,OR]
RewriteCond %{HTTP_REFERER} \.biz [NC,OR]
RewriteCond %{HTTP_REFERER} \.br [NC,OR]
RewriteCond %{HTTP_REFERER} \.by [NC,OR]
RewriteCond %{HTTP_REFERER} \.cn [NC,OR]
RewriteCond %{HTTP_REFERER} \.cy [NC,OR]
RewriteCond %{HTTP_REFERER} \.cz [NC,OR]
RewriteCond %{HTTP_REFERER} \.de [NC,OR]
RewriteCond %{HTTP_REFERER} \.es [NC,OR]
RewriteCond %{HTTP_REFERER} \.eu [NC,OR]
# RewriteCond %{HTTP_REFERER} \.fr [NC,OR]
RewriteCond %{HTTP_REFERER} \.hk [NC,OR]
RewriteCond %{HTTP_REFERER} \.in [NC,OR]
RewriteCond %{HTTP_REFERER} \.info [NC,OR]
RewriteCond %{HTTP_REFERER} \.ir [NC,OR]
RewriteCond %{HTTP_REFERER} \.it [NC,OR]
RewriteCond %{HTTP_REFERER} \.jp [NC,OR]
RewriteCond %{HTTP_REFERER} \.kr [NC,OR]
RewriteCond %{HTTP_REFERER} \.kw [NC,OR]
RewriteCond %{HTTP_REFERER} \.lt [NC,OR]
RewriteCond %{HTTP_REFERER} \.lu [NC,OR]
RewriteCond %{HTTP_REFERER} \.lv [NC,OR]
RewriteCond %{HTTP_REFERER} \.mx [NC,OR]
RewriteCond %{HTTP_REFERER} \.nl [NC,OR]
RewriteCond %{HTTP_REFERER} \.pk [NC,OR]
RewriteCond %{HTTP_REFERER} \.pl [NC,OR]
RewriteCond %{HTTP_REFERER} \.pw [NC,OR]
RewriteCond %{HTTP_REFERER} \.py [NC,OR]
RewriteCond %{HTTP_REFERER} \.ro [NC,OR]
RewriteCond %{HTTP_REFERER} \.ru [NC,OR]
RewriteCond %{HTTP_REFERER} \.sk [NC,OR]
RewriteCond %{HTTP_REFERER} \.su [NC,OR]
RewriteCond %{HTTP_REFERER} \.to [NC,OR]
RewriteCond %{HTTP_REFERER} \.tr [NC,OR]
RewriteCond %{HTTP_REFERER} \.ua [NC,OR]
RewriteCond %{HTTP_REFERER} \.uy [NC,OR]
RewriteCond %{HTTP_REFERER} \.vg [NC,OR]
RewriteCond %{HTTP_REFERER} \.vn [NC,OR]
RewriteCond %{HTTP_REFERER} \.xxx [NC]
RewriteRule .* - [F]
4:03 am on Mar 2, 2014 (gmt 0)

WebmasterWorld Administrator phranque is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



it will block any request that has ".fr" anywhere in the HTTP Referer header url, not just for the .ccTLD
(assuming that RewriteCond isn't commented out)
5:06 am on Mar 2, 2014 (gmt 0)



I know (deny from .fr) works too, but I also want to do it from the bad REFERER like some-forum.fr
8:03 am on Mar 2, 2014 (gmt 0)

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



You need to end-anchor each TLD pattern.

There's a simpler way to do OR operations: \.(xx|yy|zz)$
9:52 am on Mar 2, 2014 (gmt 0)

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



...and
\.(aa|af|aj|az|ba|bb|bc|bd|be|cg|da|db|dc|dd|de|dh|dk|dm|dr|ds|dt|du|dz|ep|...)$

simplifies to
\.(a[afjz]|b[a-e]|cg|d[a-ehkmr-uz]|ep|...)$

and processes massively faster.
12:57 pm on Mar 2, 2014 (gmt 0)

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



Working on a new site that Starts w/ .fr the REFERER zaps the CSS in IE only...

I've looked at that upside-down, backward and sideways, and still can't make sense of it.

You need to end-anchor each TLD pattern.

But the referer is the whole page, not just the domain. Wouldn't it have to be ($|/) at the end?

With that many bad referers, seems like it would be easier to whitelist:

RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !\.(com|org|net|uk|au)($|/)


... and constrain the rule to requests for pages:
(\.html|/|^)$

substituting whatever extensions you actually use. Non-page requests would have a different set of rules with even tighter restrictions.
1:10 pm on Mar 2, 2014 (gmt 0)

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Err, yes, end the pattern with a slash or indeed whitelist rather than blacklist.
12:19 am on Mar 3, 2014 (gmt 0)



How does this look?

# Referrer Blocker
RewriteEngine On
RewriteCond %{HTTP_REFERER} \.(af|al|at|au|be|bg|br|by|ch|cl|cn|cz|de|dk|ee|es|eu)(/|$) [NC]
RewriteEngine On
RewriteCond %{HTTP_REFERER} \.(fi|fr|gb|gr|hk|id|in|ir|it|jp|kr|kz|lt|lv)(/|$) [NC]
RewriteEngine On
RewriteCond %{HTTP_REFERER} \.(me|mu|my|mx|ng|pa|ph|pk|pl|ng|nl|no|ro|rs|ru)(/|$) [NC]
RewriteEngine On
RewriteCond %{HTTP_REFERER} \.(se|sg|si|sk|su|th|tn|tr|ua|ug|uk|uy|ve|vn|ye)(/|$) [NC]
RewriteRule .* - [F]
6:11 am on Mar 3, 2014 (gmt 0)

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



RewriteEngine On
is needed only once.

You can simplify the code...
RewriteCond %{HTTP_REFERER} \.(a[fltu]|b[egry]|c[hlnz]|d[ek]|e[esu])(/|$) [NC,OR]

The first three conditions need OR at the end.
8:14 am on Mar 3, 2014 (gmt 0)

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



Come to think of it, are there any countries you don't want to lock out?

RewriteCond %{HTTP_REFERER} \.[a-z][a-z](/|$)
RewriteCond %{HTTP_REFERER} !\.(c[ao]|nz)(/|$)
12:03 am on Mar 5, 2014 (gmt 0)



All but the USA ;)
12:23 am on Mar 5, 2014 (gmt 0)



I forgot to say thank to ALL for the advice 8)
2:29 am on Mar 5, 2014 (gmt 0)

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



All but the USA

If you mean it, then you really do want

RewriteCond %{HTTP_REFERER} \.[a-z][a-z](/|$)


to exclude all country codes everywhere. (What? Even .gl?!) I guess .us is theoretically possible, but they're surely up to no good.
3:22 am on Mar 5, 2014 (gmt 0)



OR

<Files>
deny from all
allow from .us
</Files>

All fun & games aside my main pests are .cn, .ru, .su, .de, .ua & .nl w/ webbost bots
7:20 am on Mar 5, 2014 (gmt 0)

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Are you sure it is REFERER you should be testing?
 

Featured Threads

Hot Threads This Week

Hot Threads This Month