homepage Welcome to WebmasterWorld Guest from 54.204.141.129
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Code, Content, and Presentation / Apache Web Server
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL & phranque

Apache Web Server Forum

    
REFERER Overkill?
REFERER, Overkill, block
EastTexas




msg:4650487
 3:24 am on Mar 2, 2014 (gmt 0)

REFERER Overkill?

Working on a new site that Starts w/ .fr the REFERER zaps the CSS in IE only...

I always thought it blocked the domains ending in .fr?

I have also seen some iPads & sometimes rr.com, comcast.net & myvzw.com being blocked.


# Referrer Blocker - .se search bug?
RewriteEngine On
RewriteCond %{HTTP_REFERER} \.at [NC,OR]
RewriteCond %{HTTP_REFERER} \.biz [NC,OR]
RewriteCond %{HTTP_REFERER} \.br [NC,OR]
RewriteCond %{HTTP_REFERER} \.by [NC,OR]
RewriteCond %{HTTP_REFERER} \.cn [NC,OR]
RewriteCond %{HTTP_REFERER} \.cy [NC,OR]
RewriteCond %{HTTP_REFERER} \.cz [NC,OR]
RewriteCond %{HTTP_REFERER} \.de [NC,OR]
RewriteCond %{HTTP_REFERER} \.es [NC,OR]
RewriteCond %{HTTP_REFERER} \.eu [NC,OR]
# RewriteCond %{HTTP_REFERER} \.fr [NC,OR]
RewriteCond %{HTTP_REFERER} \.hk [NC,OR]
RewriteCond %{HTTP_REFERER} \.in [NC,OR]
RewriteCond %{HTTP_REFERER} \.info [NC,OR]
RewriteCond %{HTTP_REFERER} \.ir [NC,OR]
RewriteCond %{HTTP_REFERER} \.it [NC,OR]
RewriteCond %{HTTP_REFERER} \.jp [NC,OR]
RewriteCond %{HTTP_REFERER} \.kr [NC,OR]
RewriteCond %{HTTP_REFERER} \.kw [NC,OR]
RewriteCond %{HTTP_REFERER} \.lt [NC,OR]
RewriteCond %{HTTP_REFERER} \.lu [NC,OR]
RewriteCond %{HTTP_REFERER} \.lv [NC,OR]
RewriteCond %{HTTP_REFERER} \.mx [NC,OR]
RewriteCond %{HTTP_REFERER} \.nl [NC,OR]
RewriteCond %{HTTP_REFERER} \.pk [NC,OR]
RewriteCond %{HTTP_REFERER} \.pl [NC,OR]
RewriteCond %{HTTP_REFERER} \.pw [NC,OR]
RewriteCond %{HTTP_REFERER} \.py [NC,OR]
RewriteCond %{HTTP_REFERER} \.ro [NC,OR]
RewriteCond %{HTTP_REFERER} \.ru [NC,OR]
RewriteCond %{HTTP_REFERER} \.sk [NC,OR]
RewriteCond %{HTTP_REFERER} \.su [NC,OR]
RewriteCond %{HTTP_REFERER} \.to [NC,OR]
RewriteCond %{HTTP_REFERER} \.tr [NC,OR]
RewriteCond %{HTTP_REFERER} \.ua [NC,OR]
RewriteCond %{HTTP_REFERER} \.uy [NC,OR]
RewriteCond %{HTTP_REFERER} \.vg [NC,OR]
RewriteCond %{HTTP_REFERER} \.vn [NC,OR]
RewriteCond %{HTTP_REFERER} \.xxx [NC]
RewriteRule .* - [F]

 

phranque




msg:4650493
 4:03 am on Mar 2, 2014 (gmt 0)

it will block any request that has ".fr" anywhere in the HTTP Referer header url, not just for the .ccTLD
(assuming that RewriteCond isn't commented out)

EastTexas




msg:4650498
 5:06 am on Mar 2, 2014 (gmt 0)

I know (deny from .fr) works too, but I also want to do it from the bad REFERER like some-forum.fr

g1smd




msg:4650503
 8:03 am on Mar 2, 2014 (gmt 0)

You need to end-anchor each TLD pattern.

There's a simpler way to do OR operations: \.(xx|yy|zz)$

g1smd




msg:4650506
 9:52 am on Mar 2, 2014 (gmt 0)

...and
\.(aa|af|aj|az|ba|bb|bc|bd|be|cg|da|db|dc|dd|de|dh|dk|dm|dr|ds|dt|du|dz|ep|...)$
simplifies to
\.(a[afjz]|b[a-e]|cg|d[a-ehkmr-uz]|ep|...)$
and processes massively faster.

lucy24




msg:4650518
 12:57 pm on Mar 2, 2014 (gmt 0)

Working on a new site that Starts w/ .fr the REFERER zaps the CSS in IE only...

I've looked at that upside-down, backward and sideways, and still can't make sense of it.

You need to end-anchor each TLD pattern.

But the referer is the whole page, not just the domain. Wouldn't it have to be ($|/) at the end?

With that many bad referers, seems like it would be easier to whitelist:

RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !\.(com|org|net|uk|au)($|/)


... and constrain the rule to requests for pages:
(\.html|/|^)$
substituting whatever extensions you actually use. Non-page requests would have a different set of rules with even tighter restrictions.

g1smd




msg:4650519
 1:10 pm on Mar 2, 2014 (gmt 0)

Err, yes, end the pattern with a slash or indeed whitelist rather than blacklist.

EastTexas




msg:4650582
 12:19 am on Mar 3, 2014 (gmt 0)

How does this look?

# Referrer Blocker
RewriteEngine On
RewriteCond %{HTTP_REFERER} \.(af|al|at|au|be|bg|br|by|ch|cl|cn|cz|de|dk|ee|es|eu)(/|$) [NC]
RewriteEngine On
RewriteCond %{HTTP_REFERER} \.(fi|fr|gb|gr|hk|id|in|ir|it|jp|kr|kz|lt|lv)(/|$) [NC]
RewriteEngine On
RewriteCond %{HTTP_REFERER} \.(me|mu|my|mx|ng|pa|ph|pk|pl|ng|nl|no|ro|rs|ru)(/|$) [NC]
RewriteEngine On
RewriteCond %{HTTP_REFERER} \.(se|sg|si|sk|su|th|tn|tr|ua|ug|uk|uy|ve|vn|ye)(/|$) [NC]
RewriteRule .* - [F]

g1smd




msg:4650652
 6:11 am on Mar 3, 2014 (gmt 0)

RewriteEngine On is needed only once.

You can simplify the code...
RewriteCond %{HTTP_REFERER} \.(a[fltu]|b[egry]|c[hlnz]|d[ek]|e[esu])(/|$) [NC,OR]
The first three conditions need OR at the end.

lucy24




msg:4650669
 8:14 am on Mar 3, 2014 (gmt 0)

Come to think of it, are there any countries you don't want to lock out?

RewriteCond %{HTTP_REFERER} \.[a-z][a-z](/|$)
RewriteCond %{HTTP_REFERER} !\.(c[ao]|nz)(/|$)

EastTexas




msg:4651190
 12:03 am on Mar 5, 2014 (gmt 0)

All but the USA ;)

EastTexas




msg:4651196
 12:23 am on Mar 5, 2014 (gmt 0)

I forgot to say thank to ALL for the advice 8)

lucy24




msg:4651210
 2:29 am on Mar 5, 2014 (gmt 0)

All but the USA

If you mean it, then you really do want

RewriteCond %{HTTP_REFERER} \.[a-z][a-z](/|$)

to exclude all country codes everywhere. (What? Even .gl?!) I guess .us is theoretically possible, but they're surely up to no good.

EastTexas




msg:4651216
 3:22 am on Mar 5, 2014 (gmt 0)

OR

<Files>
deny from all
allow from .us
</Files>

All fun & games aside my main pests are .cn, .ru, .su, .de, .ua & .nl w/ webbost bots

g1smd




msg:4651260
 7:20 am on Mar 5, 2014 (gmt 0)

Are you sure it is REFERER you should be testing?

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Code, Content, and Presentation / Apache Web Server
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved