Lord_Majestic - 7:56 pm on Dec 2, 2005 (gmt 0)

IMO the best most honest policy is this: you either close access to search engines or you allow them (non-abusive bots only of course) to roam free - doing anything else is likely to push site beyond line in sand that separates good content sites and spammy junkies.