Page is a not externally linkable
- Search Engines
-- Sitemaps, Meta Data, and robots.txt
---- robots.txt


jdMorgan - 6:09 pm on Apr 11, 2003 (gmt 0)


spinweb,

That is not valid robots.txt syntax.

What you need to do is to split those domains out, and give each one its own robots.txt. This can be done, even though they all go to the same hosting account. Then simply disallow all robots from those domains you don't want indexed.

The method used to do this will depend on your server, e.g. Apache or IIS. It's fairly easy with Apache if you can use mod_rewrite.

Jim


Thread source:: http://www.webmasterworld.com/robots_txt/36.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com