mod rewrite skipping some concurrent requests

[milanmk]

I have setup the following rewrite rule in VirtualHost to only allow homepage requests (Thanks Jim).

RewriteEngine on
RewriteRule!^/(index\.html)?$ http://example.com/ [R=301,L]

However, some requests are redirected and rests are still able to generate 404 errors.

Here are the access log entries:

[02:08:03] "GET //phpads/adxmlrpc.php HTTP/1.1" [b]301[/b] 235
[02:08:03] "GET //Ads/adxmlrpc.php HTTP/1.1" 404 223
[02:08:03] "GET //ads/adxmlrpc.php HTTP/1.1" 404 223
[02:08:03] "GET //adxmlrpc.php HTTP/1.1" 404 219
[02:08:03] "GET //Ads/adxmlrpc.php HTTP/1.1" [b]301[/b] 235
[02:08:03] "GET //ads/adxmlrpc.php HTTP/1.1" 404 223
[02:08:03] "GET //adserver/adxmlrpc.php HTTP/1.1" 404 228
[02:08:04] "GET //ads/adxmlrpc.php HTTP/1.1" [b]301[/b] 235
[02:08:04] "GET //phpAdsNew/adxmlrpc.php HTTP/1.1" 404 229
[02:08:04] "GET //phpadsnew/adxmlrpc.php HTTP/1.1" 404 229
[02:08:05] "GET //phpads/adxmlrpc.php HTTP/1.1" 404 226
[02:08:05] "GET //Ads/adxmlrpc.php HTTP/1.1" 404 223
[02:08:06] "GET //ads/adxmlrpc.php HTTP/1.1" 404 223

Any idea why mod_rewrite is not able to redirect all the non-homepage requests?

Milan

[jdMorgan]

Simply put, because all of those are malformed requests from exploit scanners, looking for security holes in common PHP scripts.

I'd suggest adding a rule ahead of your other rules:

RewriteRule adxmlrpc\.php$ - [F]

This returns a 403-Forbidden response to any requests which end with "adxmlrpc.php"

[added] (The requests are malformed because they contain two leading slashes.) [/added]

Jim

[edited by: jdMorgan at 2:58 pm (utc) on Mar. 25, 2007]

[milanmk]

Yes, I am aware that all these are malformed requests and I am getting almost 200+ such other requests from scanners per hour. I have my own logging script for this and what I really want is to redirect all these requests to that logging script instead of generating 404/403 errors in my error log.

My concern is why Apache is not able to redirect all such requests.

Milan

[jdMorgan]

Did you try the code or not?

Jim

[milanmk]

Yes. I blocked all but homepage requests with 403 but I think I will need to wait for sometime while such requests start from scanners.

RewriteEngine on
RewriteRule!^/(index\.html)?$ - [F]

As such even the above redirect was working when I manually put such malformed url in browser but it misses out some when scanners are doing it concurrently.

Let us see whether Apache can handle the new rewrite. I will update you shortly.

Milan

[jdMorgan]

"Concurrently" should affect nothing -- Each Apache thread (started by an incoming request) runs on its own, and each will process the mod_rewrite code. If all of these requests are not caught, then either the code does not match the problem precisely, or the code is not located in the path to the requested resource.

Basically, it should be impossible to serve any request without running the code, and as long as the code is in the right place in the filesystem, it will be executed for all requests.

Jim

[milanmk]

I see your point here.

Still do not have any scanners yet. I think I will put some overhead to the server with RewriteLog and try to find the exact problem with the code.

Milan