Page is a not externally linkable
- Search Engines
-- Sitemaps, Meta Data, and robots.txt
---- Fear and Loathing of Robots.txt

mivox - 6:06 pm on Jun 22, 2001 (gmt 0)


For Mac users, BBEdit and BBEdit Lite can be set to save everything in Unix format. My BBEdit-created robots.txt just validated with flying colors.

Thread source:: http://www.webmasterworld.com/robots_txt/20.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com