Page is a not externally linkable
- Search Engines
-- Sitemaps, Meta Data, and robots.txt
---- Robots.txt for mirror sites


jdMorgan - 8:59 pm on Aug 6, 2002 (gmt 0)


ScottM,

On a mirror site I have for use if the primary server fails, I use

User-agent: *
Disallow: /

This disallows all robots from indexing the duplicated content. I do not promote the mirror site
in any way, except for a single link on the main site on a page that is also disallowed. The mirror
is for back-up purposes only.

No banned-due to duplicate-content problems for me!

Jim


Thread source:: http://www.webmasterworld.com/robots_txt/144.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com