homepage Welcome to WebmasterWorld Guest from 54.204.182.118
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Code, Content, and Presentation / Apache Web Server
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL & phranque

Apache Web Server Forum

    
Block Bots With Empty User-Agents?
hiccupsheadache



 
Msg#: 4268077 posted 6:26 pm on Feb 16, 2011 (gmt 0)

Hi guys,

Please can someone tell me how to block bots with empty user agents with .htaccess.

Thanks so much for any help!

my site has been disabled - and i cannot have it turned back on until i stop the problem i am having with bots using my bandwidth.

Many thanks!

 

wilderness

WebmasterWorld Senior Member wilderness us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4268077 posted 12:40 am on Feb 17, 2011 (gmt 0)

There should be many examples of the required lines in the archives.

blank+ua [google.com]

hiccupsheadache



 
Msg#: 4268077 posted 7:42 am on Feb 17, 2011 (gmt 0)

thanks wilderness -- but im totally usless with .htaccess files - and whenever i have to extract just a snippet of code from a full file, it never works :-(

What i was really hopeing for was just an example of a htaccess that ONLY blocks empty user agent bots. Then i could see which lines are really needed.

Thanks again to anyone reading who can help.

Regards.

wilderness

WebmasterWorld Senior Member wilderness us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4268077 posted 5:33 pm on Feb 17, 2011 (gmt 0)

FWIW, denying blank User Agents is one of the simplest and least technical additions that may be added to htaccess.

As Jim explained to you in a 2010 inquiry, reviewing the Forum Library, as well as this previous thread link that I supplied, will provide a solution example for you.

jdMorgan

WebmasterWorld Senior Member jdmorgan us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 4268077 posted 2:11 am on Feb 18, 2011 (gmt 0)

Add above your existing rewriterules:

RewriteCond %{HTTP_USER_AGENT} ^-?$
RewriteRule ^ - [F]

This will block blank user-agents and those consisting of a single hyphen, which are "blank user-agent spoofers" -- and even worse than actually-blank ones...

If you don't already have any other working rules, then you'll probably need to precede the above rule with either both of these lines or only the second one. I can't tell you, because it depends on your host's default configuration. You will have to test on your server to find out.

Options +FollowSymLinks -MultiViews
RewriteEngine on


Jim

hiccupsheadache



 
Msg#: 4268077 posted 7:13 pm on Feb 20, 2011 (gmt 0)

I'm sorry for being a total idiot!

wilderness -- I never even saw the link you posted! Sorry. I thought it was just a sig link. haha. I should read more carefully :)

Jim -- Once again, you are a true life saver! Your posts are always fantastic. Cheers!

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Code, Content, and Presentation / Apache Web Server
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved