| Microsoft Details Robots.txt
|
Brett_Tabke
msg:3665896 | 4:40 pm on Jun 3, 2008 (gmt 0) |
From the Live Search webmaster Blog [blogs.msdn.com] | The de-facto standard for managing this is the Robots Exclusion Protocol (REP) introduced back in the early 1990's. Over the years, the REP has evolved to support more than "exclusion" directives; it now supports directives controlling what content gets included, how the content is displayed, and how frequently the content is crawled. |
|
[edited by: tedster at 6:58 pm (utc) on June 3, 2008]
[edit reason] fix side scroll [/edit]
|
WiseWebDude
msg:3665908 | 4:55 pm on Jun 3, 2008 (gmt 0) |
Oh, sweet, they are going to obey wildcards by end of June! Thanks, MSN! That is great to hear.
|
g1smd
msg:3666195 | 11:14 pm on Jun 3, 2008 (gmt 0) |
Will it be like Google operates though? [webmasterworld.com...]
|
|
|
