Page is a not externally linkable
blend27 - 8:28 pm on Nov 18, 2007 (gmt 0)
Reason: User-agent: * #pages and files in this folder are not allowed to be seen by spiders, contains high resolution images #page contains contents to UPS Rate look up Script with in my site, a pop window #page has been disallowed for 4 years, specificaly to ban obusive, rouge bots, spider trap and is listed on first page in Live.com when site comand is issued #Same as above MSNDude this is something that your team should think of.
I just 403'd msnbot-Products/1.0+(+http://search.msn.com/msnbot.htm).
Disallow: /folder1/
Disallow: /page1.cfm*
Disallow: /page2.cfm
Disallow: /page3.cfm