Page is a not externally linkable
- Search Engines
-- Sitemaps, Meta Data, and robots.txt
---- Do we need robots.txt?


willtell - 9:21 pm on Jun 6, 2002 (gmt 0)


Ciml, could you elaborate on your comment about sending 403's. We are going to stop certain spam sites at the network card level.


Thread source:: http://www.webmasterworld.com/robots_txt/112.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com