Welcome to WebmasterWorld Guest from 50.17.114.227

Forum Moderators: mack

Message Too Old, No Replies

Microsoft Details Robots.txt

     
4:40 pm on Jun 3, 2008 (gmt 0)

Administrator from US 

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 21, 1999
posts:38047
votes: 11


From the Live Search webmaster Blog [blogs.msdn.com]

The de-facto standard for managing this is the Robots Exclusion Protocol (REP) introduced back in the early 1990's. Over the years, the REP has evolved to support more than "exclusion" directives; it now supports directives controlling what content gets included, how the content is displayed, and how frequently the content is crawled.

[edited by: tedster at 6:58 pm (utc) on June 3, 2008]
[edit reason] fix side scroll [/edit]

4:55 pm on June 3, 2008 (gmt 0)

Preferred Member

5+ Year Member

joined:June 8, 2006
posts:556
votes: 0


Oh, sweet, they are going to obey wildcards by end of June! Thanks, MSN! That is great to hear.
11:14 pm on June 3, 2008 (gmt 0)

Senior Member

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:July 3, 2002
posts:18903
votes: 0


Will it be like Google operates though?

[webmasterworld.com...]

 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members