- - Search Engines
- -- Sitemaps, Meta Data, and robots.txt
- ---- I've been looking at the WW robots.txt ...
sem4u - 1:32 pm on Feb 4, 2004 (gmt 0)
|Are we talking about deleted posts that may be archived, etc? |
I think so. Also, any threads that may need to be deleted at a later date.
Thread source: http://www.webmasterworld.com/robots_txt/255.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com