homepage Welcome to WebmasterWorld Guest from 54.161.166.171
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Visit PubCon.com
Home / Forums Index / Code, Content, and Presentation / Apache Web Server
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL & phranque

Apache Web Server Forum

    
Using mod rewrite to block traffic from a particular domain referral?
georgec




msg:1496749
 9:06 am on Jun 21, 2003 (gmt 0)

Hi:
I need some help blocking all traffic from a particular domain referral, lets say xxx.com. I know this requires mod rewrite to first detect the referral itself and act accordingly, but don't know where to go from there. Can anyone help?

The problem I'm facing is that starting recently, a sex related domain has been showing up big time in our referral logs. The referral URL itself is some strange cgi redirect script I can't make heads or tails from (redirects to a 404 page), and it's obvious they're not sending actual visitors to my site. I suspect they may simply be hot linking or accessing some non html files on our server, jacking up the referral logs in the process. Using mod rewrite would seem to be the only solution, as something like IP tables blocks the domain itself, but not traffic redirected from and not necessarily originating from the offending domain (referrals).

Thanks much.

 

georgec




msg:1496750
 9:22 am on Jun 21, 2003 (gmt 0)

Never mind, figured it out. For those who are interested:

RewriteEngine on
Options +FollowSymlinks
RewriteCond %{HTTP_REFERER} xxx\.com
RewriteRule .* - [F,L]

georgec




msg:1496751
 11:24 pm on Jun 21, 2003 (gmt 0)

Just a question on syntax- when is it necessary to backslash in a .htaccess file? For example, if the domain I wish to ban above is:

something.xxx.com

Do I need to specifiy it as:

something\.xxx\.com?

I couldn't confirm that it even makes a difference one way or the other.

jdMorgan




msg:1496752
 11:35 pm on Jun 21, 2003 (gmt 0)

georgec,

In a mod_rewrite/regular expressions context, an unescaped period means "any single character," so yes, you should precede periods in domain names, IP addresses, and file extensions with a backslash.

Ref: Regular expressions tutorial [etext.lib.virginia.edu]

Also, you don't need an "L" in your [F,L] flag - "L" is built into "F".

Ref: Apache mod_rewrite [httpd.apache.org]

Jim

berli




msg:1496753
 3:01 am on Jun 22, 2003 (gmt 0)

What is the Options +FollowSymlinks part for?

Can it be safely omitted?

jdMorgan




msg:1496754
 3:13 am on Jun 22, 2003 (gmt 0)

It can be safely omitted, unless you get a 500-server error when you omit it. :)

It is needed if the user acccount was not configured with FollowSymLinks in its <directory> section in httpd.conf (server configuration).

Jim

georgec




msg:1496755
 3:26 am on Jun 22, 2003 (gmt 0)

Thanks jdMorgan for the explanation.

georgec




msg:1496756
 1:02 am on Jul 2, 2003 (gmt 0)

Another follow up question if you don't mind. I need to add another rogue referrer to my .htaccess file. Here's naturally what I gather I should do:

RewriteEngine on
Options +FollowSymlinks
RewriteCond %{HTTP_REFERER} xxx\.com
RewriteCond %{HTTP_REFERER} anotherdomain\.com
RewriteRule .* - [F]

Are there any "flags" I need to add to the end of the two lines containing the domains, like [NC] or [OR]? I've seen them in documentations, but can't figure out when they're needed, such as in this case.

Thanks for the help.

Wizcrafts




msg:1496757
 1:46 am on Jul 2, 2003 (gmt 0)

Georgec

Yes, you do need to ad [OR} after the first line, allowing a space after the expression, ie:

RewriteCond %{HTTP_REFERER} xxx\.com [OR]
RewriteCond %{HTTP_REFERER} anotherdomain\.com
RewriteRule .* - [F]

Also, if there is a possibility that the referer will have some uppercase letters in it, add NC to the switch, as in [NC,OR]

I hope this helps
Wiz

georgec




msg:1496758
 8:21 am on Jul 2, 2003 (gmt 0)

Great Wiz, thanks. I figured I probably need to add [OR] when it comes to blocking (and not accepting) a list of domains.

Thx.

tschild




msg:1496759
 8:57 pm on Jul 2, 2003 (gmt 0)

Approaching your problem from another direction: One possible reason for strange referrer entries of this kind is that they want their URL to show up in a publicly viewable statistics page. Are your web server statistics password-protected or are they viewable to the public?

claus




msg:1496760
 9:21 pm on Jul 2, 2003 (gmt 0)

allow me to add that it's normally more accurate to ban an IP address than a host name/referrer, as the latter can more easily be manipulated.

/claus

georgec




msg:1496761
 7:15 am on Jul 3, 2003 (gmt 0)

Tschild, our server doesn't generate web stats, just log files I download to then process offline.

claus, thanks for the note. I've considered banning by their IP instead, though in this case it would seem referer is more appropriate. As mentioned, it's some strange sex site.

claus




msg:1496762
 10:57 am on Jul 3, 2003 (gmt 0)

ohyes, it's easier too se what it means. IPs are just numbers. But, consider for a moment how easy it is to go from, say, www.widgetsex.com to xxx.widgetsex.com on the same IP - it'doesn't take hours to make that change ;)
/claus

arielmeadow




msg:1496763
 6:24 pm on Jul 29, 2003 (gmt 0)

I tried using this code in my htaccess file, and I seem to have hit an error.

<Limit GET POST>
Order Allow,Deny
Allow from all
</Limit>

rewriteEngine on
Options +FollowSymlinks
rewriteCond %{HTTP_REFERER} offendingsite\.com
rewriteRule .* - [F]

ErrorDocument 403 [mysite.com...]

When I test by clicking the link on the offending site, I get this error message:

"Redirection limit for this URL exceeded. Unable to load the requested page. This may be caused by cookies that are blocked."

I'm thinking it doesn't have anything to do with cookies, and is instead related to my coding. Oops. Any advice?

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Code, Content, and Presentation / Apache Web Server
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved