homepage Welcome to WebmasterWorld Guest from 54.167.174.90
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Code, Content, and Presentation / Apache Web Server
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL & phranque

Apache Web Server Forum

    
htaccess to ban all but standard browsers?
Prevent spam-bots, No need for search engines
kapow

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 256 posted 5:08 pm on Jun 20, 2003 (gmt 0)

Is there a simple (short) .htaccess I can use to ban all but standard browser users.

The site is for internal use within a company (visitors may want to access it from home or a hotel). They don't need search engines or anyone else to find it. Users will all know the url.

My main concern is the legions of spam-bots out there (I will encode the email addresses). There is no need to cater for any unusual browsers either.

So I am wondering if/how to make a .htaccess to ban everything except say: IE, Netscape and Opera

I've tried to follow the detailed thread on creating an htaccess file for sites that require search engines
[webmasterworld.com...]
but it gets confusing (I know very little about htaccess).

 

grahamstewart

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 256 posted 5:09 am on Jun 21, 2003 (gmt 0)

The trouble is.. whats a "standard" browser?

If you want to ban everything except Internet Explorer then that will be easy enough. But evolt.org currently list around one hundred different browsers that some one somewhere will regard as 'standard'.

Evolt Browser Archive [browsers.evolt.org]

jdMorgan

WebmasterWorld Senior Member jdmorgan us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 256 posted 6:18 am on Jun 21, 2003 (gmt 0)

Actually, I think a larger problem is that many of the spambots spoof as common browsers, so you have to block by IP address or IP ranges. So that blows the original "short .htaccess" requirement right out the window.

Then there is the "blank user-agent" problem. This will either cause problems with users who have products like Norton Internet Security installed, or it will represent a 'hole" in the "security," depending on whether you allow or disallow blank user-agents. So again, you have to fall back to IP control if this is a concern.

I have been experimenting with the idea of a "block unless allowed" .htaccess, and even though it is much shorter than a version that blocks specific bad-bots, it is still about 15kB in size with all the IP stuff in it. And just as things were going well, MSN comes out with a brand-new 'bot that was not "allowed" and it ran smack into a 403-Forbidden. Luckily it came back, but that is the downside of the "allow" method. It is definitely NOT what you'd want for a "set and forget" general-purpose Web site with low maintenance.

So my short answer to the original post would be, "No, there is not a short .htaccess that will do that, assuming you are on the internet and not on a tightly-firewalled-off intranet."

Create an allow list, add some IP blocking, and install key_master's bad-bot trap - for starters.

Jim

DrDoc

WebmasterWorld Senior Member drdoc us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 256 posted 7:55 pm on Jun 21, 2003 (gmt 0)

Why not just skip the entire "ban bad user" idea and set up a password system?

kapow

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 256 posted 9:11 pm on Jun 22, 2003 (gmt 0)

MSN comes out with a brand-new 'bot

The site does not require search engine ranking at all.

set up a password system?

Have explored that idea - Would rather keep it easy for visitors from the owner/business to access it. They will forget passwords, many of them may only view the site a couple of times a year.

one hundred different browsers that some one somewhere will regard as 'standard'

All potential visitors from the business that the site is for (about 7000 in total) have ordinary browsers ie Internet Explorer or Netscape.

Can someone tell me how to write a ban thing that bans all that is not IE or Netscape? I will live with the bad-bots that disguise them selves - for now.

Key_Master

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 256 posted 9:16 pm on Jun 22, 2003 (gmt 0)

Why not use Javascript to write a cookie to the browser. No cookie or JavaScript, no access.

jdMorgan

WebmasterWorld Senior Member jdmorgan us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 256 posted 9:46 pm on Jun 22, 2003 (gmt 0)

kapow,

This should allow only MS Internet Explorer and Netscape 4.x and above browsers.
It will not block bad-bots spoofing as either.
Use at your own risk.

Options +FollowSymLinks
RewriteEngine on
RewriteCond %{HTTP_USER_AGENT} !^Mozilla/[3-9]\.[0-9]\ \(compatible\;\ MSIE
RewriteCond %{HTTP_USER_AGENT} !^Mozilla/4\.[0-9]{1,2}\ \[en\]
RewriteCond %{HTTP_USER_AGENT} !^Mozilla/[5-9]\.[0-9]\ \([^\)]*\)\ Gecko/[^\ ]*\ Netscape/
RewriteRule .* - [F]

Jim

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Code, Content, and Presentation / Apache Web Server
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved