msgraph - 3:42 am on May 4, 2002 (gmt 0)

I agree with John. When you have lot's of domains running on a server it can really slow things down to just do DNS/IP matchups.

Plus some of the SE's are adding new IPs every few months. I'd rather have a failsafe and take my chances than have the search engine catch me with my pants down.

One of the best methods I have experienced had both options going. When the browser visits, a UA match is performed. For example, Mozilla to the left and the rest to the right.

If Mozilla has a term that matches something from the SE then it is given the proper page.

The "rest" go off by UA/IP match.

The bots can afford a slight slowdown but we're not going to give that slowdown to every single user that enters the site.

As for the getting caught, I'd rather get caught by some user who is just trying to steal a page than a SE robot. I'm not going to take that risk at all.