Page is a not externally linkable
- Search Engines
-- Sitemaps, Meta Data, and robots.txt
---- SonicWALL Firewall Blocks Spiders


mavherick - 9:04 pm on May 4, 2005 (gmt 0)


Who's the target audience for your robots.txt? Well mannered bots right? We all know the bad bots won't listen anyway right? So I would consider that akin to banning users who visit your site map. Weird decision if you ask me.


Thread source:: http://www.webmasterworld.com/robots_txt/631.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com