Page is a not externally linkable
- Search Engines
-- Sitemaps, Meta Data, and robots.txt
---- Robots.txt question


JVB_Mktg - 7:22 pm on Mar 2, 2003 (gmt 0)


How can I have a bot ignore every directory in my site except one?

Do I have to disalow all of them 1 by 1 ie.

User-agent: bot
Disallow: /folder1/

User-agent: bot
Disallow: /folder2/

User-agent: bot
Disallow: /folder3/

User-agent: bot
Disallow: /folder4/ etc.. or is there an easier way.

I only need it to check www.mydomain.com/midi and exclude it from every other place.

TIA,

Javi


Thread source:: http://www.webmasterworld.com/robots_txt/138.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com