Forum Moderators: phranque

Message Too Old, No Replies

How to block libwww-perl user agent other than W3C version

How to block all libwww user agents other than the W3C version

         

wdfw

7:04 pm on Feb 23, 2011 (gmt 0)

10+ Year Member



We currently block the libwww-perl user agent in our .htaccess. Its main use appears to be for script kiddies to sniff around for injection vulnerabilities.

Unfortunately, however, W3C uses it for their online link checker.

Is it possible to construct a RewriteCond that will block all libwww-perl user agents UNLESS it contains "W3C-checklink"?

Thanks!

tangor

9:50 pm on Feb 23, 2011 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Put that as a "pass" first then kill all libwww which follow, however, that's a pretty easy thing for the script kiddies to spoof... so might want to look at other criteria.

wdfw

10:35 pm on Feb 23, 2011 (gmt 0)

10+ Year Member



Thanks Tangor. As I am still quite the noob with mod_rewrite, would you mind providing the code?

Here is a snippet from our .htaccess

<IfModule mod_rewrite.c>
RewriteEngine on
RewriteCond %{REQUEST_URI} !^/security.php
RewriteCond %{HTTP_USER_AGENT} (.*)Havij(.*) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} (*)libwww-perl(*) [NC,OR]
etc., etc.,
RewriteRule .* [#*$!.#*$!.com...] [R=301,L]
</IfModule>

wilderness

10:45 pm on Feb 23, 2011 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



The version numbers may require correction before use.

#reads EXCEPT checklink OR except request for security.php
RewriteCond %{HTTP_USER_AGENT} !^/W3C-checklink/4\.5 [4/.160] libwww-perl/5\.823 [OR]
RewriteCond %{REQUEST_URI} !^/security.php