Page is a not externally linkable
- Search Engines
-- Sitemaps, Meta Data, and robots.txt
---- Put your robots.txt on a diet
cabowabo - 1:42 pm on Jun 13, 2003 (gmt 0)Thread source:: http://www.webmasterworld.com/robots_txt/3.htm
Great info, as a newbie, I have just one question: Why would you want to disallow Grub? Don't they feed Wisenut which is owned by LookSmart?
Brought to you by WebmasterWorld: http://www.webmasterworld.com