Welcome to WebmasterWorld Guest from 50.19.53.104

Forum Moderators: Ocean10000 & incrediBILL & phranque

Message Too Old, No Replies

Block Bots With Empty User-Agents?

     

hiccupsheadache

6:26 pm on Feb 16, 2011 (gmt 0)

5+ Year Member



Hi guys,

Please can someone tell me how to block bots with empty user agents with .htaccess.

Thanks so much for any help!

my site has been disabled - and i cannot have it turned back on until i stop the problem i am having with bots using my bandwidth.

Many thanks!

wilderness

12:40 am on Feb 17, 2011 (gmt 0)

WebmasterWorld Senior Member wilderness is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



There should be many examples of the required lines in the archives.

blank+ua [google.com]

hiccupsheadache

7:42 am on Feb 17, 2011 (gmt 0)

5+ Year Member



thanks wilderness -- but im totally usless with .htaccess files - and whenever i have to extract just a snippet of code from a full file, it never works :-(

What i was really hopeing for was just an example of a htaccess that ONLY blocks empty user agent bots. Then i could see which lines are really needed.

Thanks again to anyone reading who can help.

Regards.

wilderness

5:33 pm on Feb 17, 2011 (gmt 0)

WebmasterWorld Senior Member wilderness is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



FWIW, denying blank User Agents is one of the simplest and least technical additions that may be added to htaccess.

As Jim explained to you in a 2010 inquiry, reviewing the Forum Library, as well as this previous thread link that I supplied, will provide a solution example for you.

jdMorgan

2:11 am on Feb 18, 2011 (gmt 0)

WebmasterWorld Senior Member jdmorgan is a WebmasterWorld Top Contributor of All Time 10+ Year Member



Add above your existing rewriterules:

RewriteCond %{HTTP_USER_AGENT} ^-?$
RewriteRule ^ - [F]

This will block blank user-agents and those consisting of a single hyphen, which are "blank user-agent spoofers" -- and even worse than actually-blank ones...

If you don't already have any other working rules, then you'll probably need to precede the above rule with either both of these lines or only the second one. I can't tell you, because it depends on your host's default configuration. You will have to test on your server to find out.

Options +FollowSymLinks -MultiViews
RewriteEngine on


Jim

hiccupsheadache

7:13 pm on Feb 20, 2011 (gmt 0)

5+ Year Member



I'm sorry for being a total idiot!

wilderness -- I never even saw the link you posted! Sorry. I thought it was just a sig link. haha. I should read more carefully :)

Jim -- Once again, you are a true life saver! Your posts are always fantastic. Cheers!
 

Featured Threads

Hot Threads This Week

Hot Threads This Month