Page is a not externally linkable
- Microsoft
-- Bing Search Engine News
---- MSN Live Needs Serious Work


blend27 - 8:28 pm on Nov 18, 2007 (gmt 0)


I just 403'd msnbot-Products/1.0+(+http://search.msn.com/msnbot.htm).

Reason:

User-agent: *

#pages and files in this folder are not allowed to be seen by spiders, contains high resolution images
Disallow: /folder1/

#page contains contents to UPS Rate look up Script with in my site, a pop window
Disallow: /page1.cfm*

#page has been disallowed for 4 years, specificaly to ban obusive, rouge bots, spider trap and is listed on first page in Live.com when site comand is issued
Disallow: /page2.cfm

#Same as above
Disallow: /page3.cfm

MSNDude this is something that your team should think of.


Thread source:: http://www.webmasterworld.com/msn_microsoft_search/3484324.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com