Welcome to WebmasterWorld Guest from 50.19.156.133

Forum Moderators: Ocean10000 & incrediBILL & phranque

Message Too Old, No Replies

Syntax for Banning bots using httpd.conf

     

Imaster

8:37 am on Jul 3, 2007 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Hello,

I am trying to ban all the bad bots from all websites on my server by using httpd.conf

I checked out one thread [webmasterworld.com...] which mentions using "BrowserMatch"

My question was should I use :
#BrowserMatch or
#browsermatchNoCase or
# SetEnvIf or
# SetEnvIfNoCase

Which one of the above should I use?

Additionally, I checked up the post "a close to perfect .htaccess file" which mentions using the following syntax types in .htaccess:

RewriteCond %{HTTP_USER_AGENT} ^BlackWidow [OR]
RewriteCond %{HTTP_USER_AGENT} ^Bot\ mailto:craftbot@yahoo.com [OR]
RewriteCond %{HTTP_USER_AGENT} ^ChinaClaw [OR]
RewriteCond %{HTTP_USER_AGENT} ^DISCo [OR]
RewriteCond %{HTTP_USER_AGENT} ^Download\ Demon [OR]

So for a 2 word spider I used ^Download\ Demon , but that is giving error in httpd.conf. Should something else be used?

browsermatch "^DISCo" Getout
BrowserMatch "^Download\ Demon" Getout
#
<Files *>
Order Deny,Allow
Deny from env=Getout
Allow from all
</Files>

jdMorgan

2:10 am on Jul 4, 2007 (gmt 0)

WebmasterWorld Senior Member jdmorgan is a WebmasterWorld Top Contributor of All Time 10+ Year Member



You can use any of the mod_setenvif directives; Some have more flexibility, some are easier to use. So the choice is a matter of need and personal style.

I assume this is the line that gave you problems:


BrowserMatch "^Download\ Demon" Getout

But that line should work fine. You could also try:

BrowserMatch ^Download\ Demon Getout

-or-

BrowserMatch "Download Demon" Getout

If neither works any better, then please tell us what is reported in your server error log -- The information logged there is often quite specific and useful.

You may already know this, but just in case: "^" is a regular expressions start-anchor. If it is used, the pattern will match only if the user-agent starts with the specified string. See the regular-expressions tutorial cited in our forum charter if you need more info.

Jim

 

Featured Threads

Hot Threads This Week

Hot Threads This Month