Blocking all bots except the big three

is this possible?

[dhiggerdhigger]

My site has visitors almost exclusively from Google, Yahoo and MSN, and from inbound links. The activity of other web robots can take up quite a bit of bandwidth.

I was wondering about allowing all user agent strings which come from browsers or bots I choose to allow (whether bots respect robots.txt or not).

1. Is this possible?
2. Is this a bad idea?

[topr8]

most badboy bots will come knocking at your door with a regular browser ua

[jdMorgan]

Yes, it's possible. Whether it's a good idea depends on your site. Some solutions are a good fit for some sites but not for others.

Many sites use a combination of firewall block lists, HTTP-user-agent and IP-address whitelisting and blacklisting, bad-bot trapping, and access-rate limiting to control site access. All of these subjects have been well-covered here, and these previous discussions should be readily available by searching this site.

Jim