Welcome to WebmasterWorld Guest from 54.242.115.55

Forum Moderators: Ocean10000 & phranque

Message Too Old, No Replies

Help with .htaccess and HTTP Referrer

Trying to generate collapsed HTTP Referrers using regular expressions

     
11:48 pm on Sep 23, 2004 (gmt 0)

New User

10+ Year Member

joined:Sept 8, 2004
posts:3
votes: 0


I am attempting to figure out how to write some rules in my .htaccess file so that I can block referral spam. At the same time, I don't want 300 lines in my .htaccess file. Could someone please take a look at this and tell me if I have the syntax correct?

Thanks in advance

Pat

----

RewriteEngine On

RewriteCond %{REQUEST_FILENAME} .*jpg$¦.*gif$¦.*png)$ [NC]
RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !example\.ca [NC]
RewriteCond %{HTTP_REFERER} !images\.google\. [NC]
RewriteCond %{HTTP_REFERER} !google\. [NC]
RewriteCond %{HTTP_REFERER} !search\?q=cache [NC]
RewriteRule (.*) /showpic.php?pic=$1

RewriteCond %{HTTP_REFERER} ^http://(www\.)?([a-z0-9_-]+\.)*paris([a-z0-9_-]+\.)*astoria([a-z0-9_-]+\.)*(com¦org¦net¦ws¦name¦biz¦us¦info).*$ [NC,OR]

RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*(free¦buy-¦download-¦dvd-¦#*$!).*(com¦org¦net¦ws¦name¦biz¦us¦info).*$ [NC,OR]

RewriteCond %{HTTP_REFERER} ^http://(www\.)?([a-z0-9_-]+\.)?dvd([a-z0-9_-]+\.)?rental([a-z0-9_-]+\.)?(com¦org¦net¦ws¦name¦biz¦us¦info).*$ [NC,OR]

RewriteCond %{HTTP_REFERER} ^http://(www\.)?example.*txcowg.*$ [NC,OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?myspace.*1546057.*$ [NC,OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?myspace.*1984292.*$ [NC]

RewriteRule ^.* - [F,L]

order allow,deny
allow from all
deny from 210.0.193.11 69.57.152.89 209.167.50.22 211.99.213.19 211.99.213.9 217.218.127.73
deny from 69.20.93.201 211.152.14.98 69.20.11.76 193.194.84.195 205.177.122.2 212.253.2.203
deny from 213.223.186.252 203.101.85.97 69.50.191.30 66.144.44.2 69.61.11.163
ErrorDocument 403 "403 Forbidden - Get lost

[edited by: jdMorgan at 12:18 am (utc) on Sep. 24, 2004]
[edit reason] Side scroll & other problems, see TOS. [/edit]

2:53 pm on Sept 24, 2004 (gmt 0)

Senior Member

WebmasterWorld Senior Member jdmorgan is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Mar 31, 2002
posts:25430
votes: 0


Pat,

Welcome to WebmasterWorld!

We don't normally do code reviews here, but these examples may be useful to others.

Here are some suggestions. Original lines are followed by a suggested replacement and a comment line.


RewriteCond %{REQUEST_FILENAME} .*jpg$¦.*gif$¦.*png$ [NC]
RewriteCond %{REQUEST_URI} \.(jpg¦gif¦png)$ [NC]
# Specify leading period on filetype and improve efficiency

RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} .
# Shorter equivalent; "Not blank" == "contains any character"

RewriteCond %{HTTP_REFERER} !example\.ca [NC]

RewriteCond %{HTTP_REFERER} !images\.google\. [NC]
# Delete the line above. It is redundant with the following line
RewriteCond %{HTTP_REFERER} !google\. [NC]

RewriteCond %{HTTP_REFERER} !http://.*google\. [NC]
# but make it a little more specific

RewriteCond %{HTTP_REFERER} !search\?q=cache [NC]

RewriteCond %{REQUEST_URI} !/showpic\.php$
# Added to prevent "infinite rewrite loop"

RewriteRule (.*) /showpic.php?pic=$1

RewriteCond %{HTTP_REFERER} ^http://(www\.)?([a-z0-9_-]+\.)*paris([a-z0-9_-]+\.)*astoria([a-z0-9_-]+\.)*(com¦org¦net¦ws¦name¦biz¦us¦info).*$ [NC,OR]
RewriteCond %{HTTP_REFERER} ^http://.*paris\ astoria.*\.(com¦org¦net¦ws¦name¦biz¦us¦info) [NC,OR]
# Unless specific matches are needed on "[a-z0-9_-]+\." etc., you can simplify these for ease of maintenance. Some loss in efficency, though. No need for end-anchor. You may not really need the tld list on the end of these Conds.

RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*(free¦buy-¦download-¦dvd-¦#*$!).*(com¦org¦net¦ws¦name¦biz¦us¦info).*$ [NC,OR]
RewriteCond %{HTTP_REFERER} ^http://.*(free¦buy-¦download-¦dvd-¦#\*\$!).*\.(com¦org¦net¦ws¦name¦biz¦us¦info) [NC,OR]
# Escape special characters * and $, no need for end-anchor

RewriteCond %{HTTP_REFERER} ^http://(www\.)?([a-z0-9_-]+\.)?dvd([a-z0-9_-]+\.)?rental([a-z0-9_-]+\.)?(com¦org¦net¦ws¦name¦biz¦us¦info).*$ [NC,OR]RewriteCond %{HTTP_REFERER} ^http://.*dvd.+rental.*\.(com¦org¦net¦ws¦name¦biz¦us¦info) [NC,OR]
# Simplified & removed end-anchor

RewriteCond %{HTTP_REFERER} ^http://(www\.)?example.*txcowg [NC,OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?myspace.*1546057 [NC,OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?myspace.*1984292 [NC]
# Removed unnecessary end-anchors

RewriteRule ^.* - [F,L]
RewriteRule .* - [F]
# Removed unnecessary start anchor, [L] is implied by [F] and is therefore redundant
3:13 pm on Sept 24, 2004 (gmt 0)

New User

10+ Year Member

joined:Sept 8, 2004
posts:3
votes: 0


All I can say is, wow. Thanks very much for such a thorough treatment of this code. As a non-coder, I was able to generate most of my .htaccess file from posts on the Webmaster World forums.

This little bit of extra attention is really appreciated.

Pat