Welcome to WebmasterWorld Guest from 54.226.62.251

Forum Moderators: Ocean10000 & incrediBILL & phranque

Message Too Old, No Replies

Using mod rewrite to block traffic from a particular domain referral?

     

georgec

9:06 am on Jun 21, 2003 (gmt 0)

10+ Year Member



Hi:
I need some help blocking all traffic from a particular domain referral, lets say xxx.com. I know this requires mod rewrite to first detect the referral itself and act accordingly, but don't know where to go from there. Can anyone help?

The problem I'm facing is that starting recently, a sex related domain has been showing up big time in our referral logs. The referral URL itself is some strange cgi redirect script I can't make heads or tails from (redirects to a 404 page), and it's obvious they're not sending actual visitors to my site. I suspect they may simply be hot linking or accessing some non html files on our server, jacking up the referral logs in the process. Using mod rewrite would seem to be the only solution, as something like IP tables blocks the domain itself, but not traffic redirected from and not necessarily originating from the offending domain (referrals).

Thanks much.

georgec

9:22 am on Jun 21, 2003 (gmt 0)

10+ Year Member



Never mind, figured it out. For those who are interested:

RewriteEngine on
Options +FollowSymlinks
RewriteCond %{HTTP_REFERER} xxx\.com
RewriteRule .* - [F,L]

georgec

11:24 pm on Jun 21, 2003 (gmt 0)

10+ Year Member



Just a question on syntax- when is it necessary to backslash in a .htaccess file? For example, if the domain I wish to ban above is:

something.xxx.com

Do I need to specifiy it as:

something\.xxx\.com?

I couldn't confirm that it even makes a difference one way or the other.

jdMorgan

11:35 pm on Jun 21, 2003 (gmt 0)

WebmasterWorld Senior Member jdmorgan is a WebmasterWorld Top Contributor of All Time 10+ Year Member



georgec,

In a mod_rewrite/regular expressions context, an unescaped period means "any single character," so yes, you should precede periods in domain names, IP addresses, and file extensions with a backslash.

Ref: Regular expressions tutorial [etext.lib.virginia.edu]

Also, you don't need an "L" in your [F,L] flag - "L" is built into "F".

Ref: Apache mod_rewrite [httpd.apache.org]

Jim

berli

3:01 am on Jun 22, 2003 (gmt 0)

10+ Year Member



What is the Options +FollowSymlinks part for?

Can it be safely omitted?

jdMorgan

3:13 am on Jun 22, 2003 (gmt 0)

WebmasterWorld Senior Member jdmorgan is a WebmasterWorld Top Contributor of All Time 10+ Year Member



It can be safely omitted, unless you get a 500-server error when you omit it. :)

It is needed if the user acccount was not configured with FollowSymLinks in its <directory> section in httpd.conf (server configuration).

Jim

georgec

3:26 am on Jun 22, 2003 (gmt 0)

10+ Year Member



Thanks jdMorgan for the explanation.

georgec

1:02 am on Jul 2, 2003 (gmt 0)

10+ Year Member



Another follow up question if you don't mind. I need to add another rogue referrer to my .htaccess file. Here's naturally what I gather I should do:

RewriteEngine on
Options +FollowSymlinks
RewriteCond %{HTTP_REFERER} xxx\.com
RewriteCond %{HTTP_REFERER} anotherdomain\.com
RewriteRule .* - [F]

Are there any "flags" I need to add to the end of the two lines containing the domains, like [NC] or [OR]? I've seen them in documentations, but can't figure out when they're needed, such as in this case.

Thanks for the help.

Wizcrafts

1:46 am on Jul 2, 2003 (gmt 0)

10+ Year Member



Georgec

Yes, you do need to ad [OR} after the first line, allowing a space after the expression, ie:


RewriteCond %{HTTP_REFERER} xxx\.com [OR]
RewriteCond %{HTTP_REFERER} anotherdomain\.com
RewriteRule .* - [F]

Also, if there is a possibility that the referer will have some uppercase letters in it, add NC to the switch, as in [NC,OR]

I hope this helps
Wiz

georgec

8:21 am on Jul 2, 2003 (gmt 0)

10+ Year Member



Great Wiz, thanks. I figured I probably need to add [OR] when it comes to blocking (and not accepting) a list of domains.

Thx.

tschild

8:57 pm on Jul 2, 2003 (gmt 0)

10+ Year Member



Approaching your problem from another direction: One possible reason for strange referrer entries of this kind is that they want their URL to show up in a publicly viewable statistics page. Are your web server statistics password-protected or are they viewable to the public?

claus

9:21 pm on Jul 2, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



allow me to add that it's normally more accurate to ban an IP address than a host name/referrer, as the latter can more easily be manipulated.

/claus

georgec

7:15 am on Jul 3, 2003 (gmt 0)

10+ Year Member



Tschild, our server doesn't generate web stats, just log files I download to then process offline.

claus, thanks for the note. I've considered banning by their IP instead, though in this case it would seem referer is more appropriate. As mentioned, it's some strange sex site.

claus

10:57 am on Jul 3, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



ohyes, it's easier too se what it means. IPs are just numbers. But, consider for a moment how easy it is to go from, say, www.widgetsex.com to xxx.widgetsex.com on the same IP - it'doesn't take hours to make that change ;)
/claus

arielmeadow

6:24 pm on Jul 29, 2003 (gmt 0)

10+ Year Member



I tried using this code in my htaccess file, and I seem to have hit an error.

<Limit GET POST>
Order Allow,Deny
Allow from all
</Limit>

rewriteEngine on
Options +FollowSymlinks
rewriteCond %{HTTP_REFERER} offendingsite\.com
rewriteRule .* - [F]

ErrorDocument 403 [mysite.com...]

When I test by clicking the link on the offending site, I get this error message:

"Redirection limit for this URL exceeded. Unable to load the requested page. This may be caused by cookies that are blocked."

I'm thinking it doesn't have anything to do with cookies, and is instead related to my coding. Oops. Any advice?

 

Featured Threads

Hot Threads This Week

Hot Threads This Month