Page is a not externally linkable
- Search Engines
-- Sitemaps, Meta Data, and robots.txt
---- Mozilla/4.* (Compatible


Seedy - 4:26 pm on Mar 14, 2012 (gmt 0)


I've noticed MSN sometimes uses Mozilla/4.* plus one or two ISP bots like StalkStalk and a couple of educational institutions.

Other than that the rest of its' users seem to be bad news.

Does anyone know of any other reasons not to block 'Mozilla/4.* (Compatible' please, perhaps to add as exclusions?

Thank you


Thread source:: http://www.webmasterworld.com/robots_txt/4429094.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com