Welcome to WebmasterWorld Guest from 54.147.237.64

Forum Moderators: Ocean10000 & incrediBILL & phranque

Message Too Old, No Replies

Syntax for Banning bots using httpd.conf

     
8:37 am on Jul 3, 2007 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Feb 3, 2003
posts:961
votes: 0


Hello,

I am trying to ban all the bad bots from all websites on my server by using httpd.conf

I checked out one thread [webmasterworld.com...] which mentions using "BrowserMatch"

My question was should I use :
#BrowserMatch or
#browsermatchNoCase or
# SetEnvIf or
# SetEnvIfNoCase

Which one of the above should I use?

Additionally, I checked up the post "a close to perfect .htaccess file" which mentions using the following syntax types in .htaccess:

RewriteCond %{HTTP_USER_AGENT} ^BlackWidow [OR]
RewriteCond %{HTTP_USER_AGENT} ^Bot\ mailto:craftbot@yahoo.com [OR]
RewriteCond %{HTTP_USER_AGENT} ^ChinaClaw [OR]
RewriteCond %{HTTP_USER_AGENT} ^DISCo [OR]
RewriteCond %{HTTP_USER_AGENT} ^Download\ Demon [OR]

So for a 2 word spider I used ^Download\ Demon , but that is giving error in httpd.conf. Should something else be used?

browsermatch "^DISCo" Getout
BrowserMatch "^Download\ Demon" Getout
#
<Files *>
Order Deny,Allow
Deny from env=Getout
Allow from all
</Files>

2:10 am on July 4, 2007 (gmt 0)

Senior Member

WebmasterWorld Senior Member jdmorgan is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Mar 31, 2002
posts:25430
votes: 0


You can use any of the mod_setenvif directives; Some have more flexibility, some are easier to use. So the choice is a matter of need and personal style.

I assume this is the line that gave you problems:


BrowserMatch "^Download\ Demon" Getout

But that line should work fine. You could also try:

BrowserMatch ^Download\ Demon Getout

-or-

BrowserMatch "Download Demon" Getout

If neither works any better, then please tell us what is reported in your server error log -- The information logged there is often quite specific and useful.

You may already know this, but just in case: "^" is a regular expressions start-anchor. If it is used, the pattern will match only if the user-agent starts with the specified string. See the regular-expressions tutorial cited in our forum charter if you need more info.

Jim

 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members