Welcome to WebmasterWorld Guest from 54.163.68.15

Forum Moderators: mack

Message Too Old, No Replies

Microsoft Details Robots.txt

   
4:40 pm on Jun 3, 2008 (gmt 0)

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month Best Post Of The Month



From the Live Search webmaster Blog [blogs.msdn.com]

The de-facto standard for managing this is the Robots Exclusion Protocol (REP) introduced back in the early 1990's. Over the years, the REP has evolved to support more than "exclusion" directives; it now supports directives controlling what content gets included, how the content is displayed, and how frequently the content is crawled.

[edited by: tedster at 6:58 pm (utc) on June 3, 2008]
[edit reason] fix side scroll [/edit]

4:55 pm on Jun 3, 2008 (gmt 0)

5+ Year Member



Oh, sweet, they are going to obey wildcards by end of June! Thanks, MSN! That is great to hear.
11:14 pm on Jun 3, 2008 (gmt 0)

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Will it be like Google operates though?

[webmasterworld.com...]