- - Search Engines
- -- Sitemaps, Meta Data, and robots.txt
- ---- Put your robots.txt on a diet
cabowabo - 1:42 pm on Jun 13, 2003 (gmt 0)
Great info, as a newbie, I have just one question: Why would you want to disallow Grub? Don't they feed Wisenut which is owned by LookSmart?
Thread source: http://www.webmasterworld.com/robots_txt/3.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com