Page is a not externally linkable
- Search Engines
-- Sitemaps, Meta Data, and robots.txt
---- basic robots.txt question


kapow - 3:22 pm on Aug 2, 2001 (gmt 0)


When I buy a domain name for a new client - the host I use (pair.com) has a form that asks which folder the name should point to. It would now be a lot of work to change them all to a sub folder of a clients folder, plus - I don't know if the host system allows that (I can ask) e.g.

clients/furniture
clients/nasa well not yet :)

It would be easier if I could just use some kind of 'Allow only this folder' instruction in my robots.txt


Thread source:: http://www.webmasterworld.com/robots_txt/108.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com