Page is a not externally linkable
- Search Engines
-- Sitemaps, Meta Data, and robots.txt
---- Robots.txt


Froggyman - 1:34 am on Apr 3, 2001 (gmt 0)


I agree. I don't have a robots.txt on my favorite site and spiders are always throwing parties there. Today, UA: ScoutMaster (62.133.27.210) was the guest of honor. Maybe a new one???

BTW- The site is well indexed by most search engines, big and small. I don't even have to resubmit anymore. Makes my job easier.


Thread source:: http://www.webmasterworld.com/robots_txt/88.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com