Welcome to WebmasterWorld Guest from 54.162.239.134

Forum Moderators: Ocean10000 & incrediBILL & phranque

Message Too Old, No Replies

htaccess to ban all but standard browsers?

Prevent spam-bots, No need for search engines

     
5:08 pm on Jun 20, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Is there a simple (short) .htaccess I can use to ban all but standard browser users.

The site is for internal use within a company (visitors may want to access it from home or a hotel). They don't need search engines or anyone else to find it. Users will all know the url.

My main concern is the legions of spam-bots out there (I will encode the email addresses). There is no need to cater for any unusual browsers either.

So I am wondering if/how to make a .htaccess to ban everything except say: IE, Netscape and Opera

I've tried to follow the detailed thread on creating an htaccess file for sites that require search engines
[webmasterworld.com...]
but it gets confusing (I know very little about htaccess).

5:09 am on Jun 21, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



The trouble is.. whats a "standard" browser?

If you want to ban everything except Internet Explorer then that will be easy enough. But evolt.org currently list around one hundred different browsers that some one somewhere will regard as 'standard'.

Evolt Browser Archive [browsers.evolt.org]

6:18 am on Jun 21, 2003 (gmt 0)

WebmasterWorld Senior Member jdmorgan is a WebmasterWorld Top Contributor of All Time 10+ Year Member



Actually, I think a larger problem is that many of the spambots spoof as common browsers, so you have to block by IP address or IP ranges. So that blows the original "short .htaccess" requirement right out the window.

Then there is the "blank user-agent" problem. This will either cause problems with users who have products like Norton Internet Security installed, or it will represent a 'hole" in the "security," depending on whether you allow or disallow blank user-agents. So again, you have to fall back to IP control if this is a concern.

I have been experimenting with the idea of a "block unless allowed" .htaccess, and even though it is much shorter than a version that blocks specific bad-bots, it is still about 15kB in size with all the IP stuff in it. And just as things were going well, MSN comes out with a brand-new 'bot that was not "allowed" and it ran smack into a 403-Forbidden. Luckily it came back, but that is the downside of the "allow" method. It is definitely NOT what you'd want for a "set and forget" general-purpose Web site with low maintenance.

So my short answer to the original post would be, "No, there is not a short .htaccess that will do that, assuming you are on the internet and not on a tightly-firewalled-off intranet."

Create an allow list, add some IP blocking, and install key_master's bad-bot trap - for starters.

Jim

7:55 pm on Jun 21, 2003 (gmt 0)

WebmasterWorld Senior Member drdoc is a WebmasterWorld Top Contributor of All Time 10+ Year Member



Why not just skip the entire "ban bad user" idea and set up a password system?
9:11 pm on Jun 22, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



MSN comes out with a brand-new 'bot

The site does not require search engine ranking at all.

set up a password system?

Have explored that idea - Would rather keep it easy for visitors from the owner/business to access it. They will forget passwords, many of them may only view the site a couple of times a year.

one hundred different browsers that some one somewhere will regard as 'standard'

All potential visitors from the business that the site is for (about 7000 in total) have ordinary browsers ie Internet Explorer or Netscape.

Can someone tell me how to write a ban thing that bans all that is not IE or Netscape? I will live with the bad-bots that disguise them selves - for now.

9:16 pm on Jun 22, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Why not use Javascript to write a cookie to the browser. No cookie or JavaScript, no access.
9:46 pm on Jun 22, 2003 (gmt 0)

WebmasterWorld Senior Member jdmorgan is a WebmasterWorld Top Contributor of All Time 10+ Year Member



kapow,

This should allow only MS Internet Explorer and Netscape 4.x and above browsers.
It will not block bad-bots spoofing as either.
Use at your own risk.


Options +FollowSymLinks
RewriteEngine on
RewriteCond %{HTTP_USER_AGENT} !^Mozilla/[3-9]\.[0-9]\ \(compatible\;\ MSIE
RewriteCond %{HTTP_USER_AGENT} !^Mozilla/4\.[0-9]{1,2}\ \[en\]
RewriteCond %{HTTP_USER_AGENT} !^Mozilla/[5-9]\.[0-9]\ \([^\)]*\)\ Gecko/[^\ ]*\ Netscape/
RewriteRule .* - [F]

Jim
 

Featured Threads

Hot Threads This Week

Hot Threads This Month