Page is a not externally linkable
- Search Engines
-- Search Engine Spider and User Agent Identification
---- Are these considered as bad bots ?

pkKumar - 4:21 am on Jan 8, 2013 (gmt 0)


can these be blocked with robots.txt, whether they will follow robots.txt, if not then you will say block in .htaccess but mine is windows server shared hostng. Site is in asp.net. What's the alternative ?

Thread source:: http://www.webmasterworld.com/search_engine_spiders/4533935.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com