Page is a not externally linkable
- Search Engines
-- Search Engine Spider and User Agent Identification
---- What's the Best Way to Keep All Spiders/Bots Out?


inbound - 5:42 pm on Oct 3, 2011 (gmt 0)


tangor, it's the kind of measures that you'll put in place to stop abuse of the search you offer that's interesting to me.

The issue that I can see is people wanting to run automated queries that will cost me money and give a poorer experience to others using the site legitimately.


Thread source:: http://www.webmasterworld.com/search_engine_spiders/4369764.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com