Page is a not externally linkable
- Microsoft
-- Bing Search Engine News
---- Does msnbot obey meta tag noindex?


morags - 9:46 am on Oct 17, 2006 (gmt 0)


Further to the thread below - but slightly different:

How do you prevent msnbot from indexing a page? This page has "<meta name="robots" content="noindex,nofollow">" in the head. ALL bots are also excluded from this page via robots.txt. And yet the page IS indexed and appears on search.live.com.

Google and Yahoo ignore this page as expected. How do I prevent msnbot from indexing the page? Do I need to specifically mention msnbot in robots.txt?

I am becoming very worried about possible duplicate content problems.

Thanks.


Thread source:: http://www.webmasterworld.com/msn_microsoft_search/3123875.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com