homepage Welcome to WebmasterWorld Guest from 54.145.183.126
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Code, Content, and Presentation / Apache Web Server
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL & phranque

Apache Web Server Forum

    
REFERER Overkill?
REFERER, Overkill, block
EastTexas



 
Msg#: 4650485 posted 3:24 am on Mar 2, 2014 (gmt 0)

REFERER Overkill?

Working on a new site that Starts w/ .fr the REFERER zaps the CSS in IE only...

I always thought it blocked the domains ending in .fr?

I have also seen some iPads & sometimes rr.com, comcast.net & myvzw.com being blocked.


# Referrer Blocker - .se search bug?
RewriteEngine On
RewriteCond %{HTTP_REFERER} \.at [NC,OR]
RewriteCond %{HTTP_REFERER} \.biz [NC,OR]
RewriteCond %{HTTP_REFERER} \.br [NC,OR]
RewriteCond %{HTTP_REFERER} \.by [NC,OR]
RewriteCond %{HTTP_REFERER} \.cn [NC,OR]
RewriteCond %{HTTP_REFERER} \.cy [NC,OR]
RewriteCond %{HTTP_REFERER} \.cz [NC,OR]
RewriteCond %{HTTP_REFERER} \.de [NC,OR]
RewriteCond %{HTTP_REFERER} \.es [NC,OR]
RewriteCond %{HTTP_REFERER} \.eu [NC,OR]
# RewriteCond %{HTTP_REFERER} \.fr [NC,OR]
RewriteCond %{HTTP_REFERER} \.hk [NC,OR]
RewriteCond %{HTTP_REFERER} \.in [NC,OR]
RewriteCond %{HTTP_REFERER} \.info [NC,OR]
RewriteCond %{HTTP_REFERER} \.ir [NC,OR]
RewriteCond %{HTTP_REFERER} \.it [NC,OR]
RewriteCond %{HTTP_REFERER} \.jp [NC,OR]
RewriteCond %{HTTP_REFERER} \.kr [NC,OR]
RewriteCond %{HTTP_REFERER} \.kw [NC,OR]
RewriteCond %{HTTP_REFERER} \.lt [NC,OR]
RewriteCond %{HTTP_REFERER} \.lu [NC,OR]
RewriteCond %{HTTP_REFERER} \.lv [NC,OR]
RewriteCond %{HTTP_REFERER} \.mx [NC,OR]
RewriteCond %{HTTP_REFERER} \.nl [NC,OR]
RewriteCond %{HTTP_REFERER} \.pk [NC,OR]
RewriteCond %{HTTP_REFERER} \.pl [NC,OR]
RewriteCond %{HTTP_REFERER} \.pw [NC,OR]
RewriteCond %{HTTP_REFERER} \.py [NC,OR]
RewriteCond %{HTTP_REFERER} \.ro [NC,OR]
RewriteCond %{HTTP_REFERER} \.ru [NC,OR]
RewriteCond %{HTTP_REFERER} \.sk [NC,OR]
RewriteCond %{HTTP_REFERER} \.su [NC,OR]
RewriteCond %{HTTP_REFERER} \.to [NC,OR]
RewriteCond %{HTTP_REFERER} \.tr [NC,OR]
RewriteCond %{HTTP_REFERER} \.ua [NC,OR]
RewriteCond %{HTTP_REFERER} \.uy [NC,OR]
RewriteCond %{HTTP_REFERER} \.vg [NC,OR]
RewriteCond %{HTTP_REFERER} \.vn [NC,OR]
RewriteCond %{HTTP_REFERER} \.xxx [NC]
RewriteRule .* - [F]

 

phranque

WebmasterWorld Administrator phranque us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4650485 posted 4:03 am on Mar 2, 2014 (gmt 0)

it will block any request that has ".fr" anywhere in the HTTP Referer header url, not just for the .ccTLD
(assuming that RewriteCond isn't commented out)

EastTexas



 
Msg#: 4650485 posted 5:06 am on Mar 2, 2014 (gmt 0)

I know (deny from .fr) works too, but I also want to do it from the bad REFERER like some-forum.fr

g1smd

WebmasterWorld Senior Member g1smd us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 4650485 posted 8:03 am on Mar 2, 2014 (gmt 0)

You need to end-anchor each TLD pattern.

There's a simpler way to do OR operations: \.(xx|yy|zz)$

g1smd

WebmasterWorld Senior Member g1smd us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 4650485 posted 9:52 am on Mar 2, 2014 (gmt 0)

...and
\.(aa|af|aj|az|ba|bb|bc|bd|be|cg|da|db|dc|dd|de|dh|dk|dm|dr|ds|dt|du|dz|ep|...)$
simplifies to
\.(a[afjz]|b[a-e]|cg|d[a-ehkmr-uz]|ep|...)$
and processes massively faster.

lucy24

WebmasterWorld Senior Member lucy24 us a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



 
Msg#: 4650485 posted 12:57 pm on Mar 2, 2014 (gmt 0)

Working on a new site that Starts w/ .fr the REFERER zaps the CSS in IE only...

I've looked at that upside-down, backward and sideways, and still can't make sense of it.

You need to end-anchor each TLD pattern.

But the referer is the whole page, not just the domain. Wouldn't it have to be ($|/) at the end?

With that many bad referers, seems like it would be easier to whitelist:

RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !\.(com|org|net|uk|au)($|/)


... and constrain the rule to requests for pages:
(\.html|/|^)$
substituting whatever extensions you actually use. Non-page requests would have a different set of rules with even tighter restrictions.

g1smd

WebmasterWorld Senior Member g1smd us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 4650485 posted 1:10 pm on Mar 2, 2014 (gmt 0)

Err, yes, end the pattern with a slash or indeed whitelist rather than blacklist.

EastTexas



 
Msg#: 4650485 posted 12:19 am on Mar 3, 2014 (gmt 0)

How does this look?

# Referrer Blocker
RewriteEngine On
RewriteCond %{HTTP_REFERER} \.(af|al|at|au|be|bg|br|by|ch|cl|cn|cz|de|dk|ee|es|eu)(/|$) [NC]
RewriteEngine On
RewriteCond %{HTTP_REFERER} \.(fi|fr|gb|gr|hk|id|in|ir|it|jp|kr|kz|lt|lv)(/|$) [NC]
RewriteEngine On
RewriteCond %{HTTP_REFERER} \.(me|mu|my|mx|ng|pa|ph|pk|pl|ng|nl|no|ro|rs|ru)(/|$) [NC]
RewriteEngine On
RewriteCond %{HTTP_REFERER} \.(se|sg|si|sk|su|th|tn|tr|ua|ug|uk|uy|ve|vn|ye)(/|$) [NC]
RewriteRule .* - [F]

g1smd

WebmasterWorld Senior Member g1smd us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 4650485 posted 6:11 am on Mar 3, 2014 (gmt 0)

RewriteEngine On is needed only once.

You can simplify the code...
RewriteCond %{HTTP_REFERER} \.(a[fltu]|b[egry]|c[hlnz]|d[ek]|e[esu])(/|$) [NC,OR]
The first three conditions need OR at the end.

lucy24

WebmasterWorld Senior Member lucy24 us a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



 
Msg#: 4650485 posted 8:14 am on Mar 3, 2014 (gmt 0)

Come to think of it, are there any countries you don't want to lock out?

RewriteCond %{HTTP_REFERER} \.[a-z][a-z](/|$)
RewriteCond %{HTTP_REFERER} !\.(c[ao]|nz)(/|$)

EastTexas



 
Msg#: 4650485 posted 12:03 am on Mar 5, 2014 (gmt 0)

All but the USA ;)

EastTexas



 
Msg#: 4650485 posted 12:23 am on Mar 5, 2014 (gmt 0)

I forgot to say thank to ALL for the advice 8)

lucy24

WebmasterWorld Senior Member lucy24 us a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



 
Msg#: 4650485 posted 2:29 am on Mar 5, 2014 (gmt 0)

All but the USA

If you mean it, then you really do want

RewriteCond %{HTTP_REFERER} \.[a-z][a-z](/|$)

to exclude all country codes everywhere. (What? Even .gl?!) I guess .us is theoretically possible, but they're surely up to no good.

EastTexas



 
Msg#: 4650485 posted 3:22 am on Mar 5, 2014 (gmt 0)

OR

<Files>
deny from all
allow from .us
</Files>

All fun & games aside my main pests are .cn, .ru, .su, .de, .ua & .nl w/ webbost bots

g1smd

WebmasterWorld Senior Member g1smd us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 4650485 posted 7:20 am on Mar 5, 2014 (gmt 0)

Are you sure it is REFERER you should be testing?

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Code, Content, and Presentation / Apache Web Server
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved