Page is a not externally linkable
- Search Engines
-- Sitemaps, Meta Data, and robots.txt
---- robots.txt


DerekJPreston - 3:17 pm on Sep 19, 2003 (gmt 0)


Google support the presence of robots.txt files in other directories, I believe, but only as a way of (temporarily) removing files. You have to tell them where the buried robots.txt files are located. By default, they like other robots only look in the root of each site. (www.yoursite.com/robots.txt)

Regards,

Derek J. Preston

[edited by: agerhart at 6:02 pm (utc) on Sep. 19, 2003]
[edit reason] no sigs please [/edit]


Thread source:: http://www.webmasterworld.com/robots_txt/1.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com