Forum Moderators: phranque
-------------------------------------------------
2xx.3.xx.30 - - [03/Nov/2003:16:46:06 -0500] "GET http://ads.example.com/creati/55-1? HTTP/1.0" 404 4854 "http://www.example.org" "Mozilla/4.0 (compatible; MSIE 5.5; Windows 98)"
2xx.93.x.66 - - [03/Nov/2003:16:50:08 -0500] "GET http://fo.ads.example.com//ad/id=ni&rw=4 HTTP/1.1" 404 4921 "http://www.example.com/" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; YComp 5.0.2.6)"
2xx.93.x.66 - - [03/Nov/2003:16:52:25 -0500] "GET http://fo.ads.example.net//ad/id=ni&rw=4 HTTP/1.1" 404 4921 "http://www.example.com/" "Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0)"
-------------------------------------------------
The request “ads.example.com” has nothing to do with my web site/pages. My proxy is turned off long time ago. The 404 error code tell there is no real danger. But I don’t like the fact that now my log is very hard to read when I want to check the broken links.
I read some previous posts here (they are dated 2/24/2003). Following the instruction I put the following two lines in .htaccess (in document root)
RewriteEngine on
RewriteRule ^/?http:// - [F]
But the problem is I still got these strange entries with error code 404, not 403. I know someone used this method before and it seems working fine. In fact, when I type in the URL http://www.mydomain/http://anything in my browser, I do got a message saying, Forbidden: You don’t have permission to access /http://anything on this server.
Can anyone give me a clue? Do I miss anything?
Thanks a lot.
Jin
[edited by: jdMorgan at 12:32 am (utc) on Nov. 7, 2003]
[edit reason] No real URLs, please. Examplified & delinked [/edit]
Anyway, I change the original method to the following and got the results I want.
========================
#for http://ads.example.com/creati/
RewriteCond %{HTTP_REFERER} ^http://www.example.com.*$ [NC,OR]
#for http://ads1.example.com/affilgo/
RewriteCond %{HTTP_REFERER} ^http://www.example.net.*$ [NC,OR]
...
...
RewriteCond %{HTTP_REFERER} ^http://www.example.org.*$ [NC]
RewriteRule .* - [F]
========================
It is working. A few days ago, in my daily log file I had 946 “404” error records come from these illegal requests and had 3 “403” errors. Yesterday the numbers changed to 19 and 233, respectively. The traffic basically remains the same.
But there are something I don’t like it:
1. The referrer website need to be selected carefully. It is easy to make mistakes to block normal traffic.
2. For each illegal request, normally there are several referrer websites. I have to select and check these referrer websites one by one. It is time-consuming.
3. Compared with original method, one line for each illegal request, now I have to setup multiple RewriteConds for each one. Considering these will be used to check every request, this overhead is probably too much (e.g. there are 30 lines for http://www.example.com/search.jsp alone and the number is still growing).
Is there any other potential problems for this method? Is there any better way to solve the problem.
Any suggestions/comments will be appreciated.
Jin
[edited by: jdMorgan at 12:46 am (utc) on Nov. 7, 2003]
[edit reason] Examplified & de-linked URLs [/edit]
You're right, blocking by referrer doesn't work well. Here's part of what I use:
RewriteCond %{THE_REQUEST} ^(GET¦HEAD¦POST)\ /?http:// [NC]
RewriteCond %{THE_REQUEST} !^(GET¦HEAD¦POST)\ /?http://(www\.)?yourdomain\.com/
RewriteCond %{THE_REQUEST} !^(GET¦HEAD¦POST)\ /?http://127\.xxx\.0\.123/
RewriteRule .* - [F]
Jim
<edit>You'll need to replace the "¦" characters above with solid vertical pipe characters from your keyboard before using this code.</edit>
Great! It is exactly what I need. I put it in my .htaccess file this morning and ALL those illegal requests got “403”.
Thank you very much.
Jin
