Page is a not externally linkable
- Search Engines
-- Sitemaps, Meta Data, and robots.txt
---- Mozilla/4.* (Compatible
Seedy - 10:50 pm on Mar 15, 2012 (gmt 0)Thread source:: http://www.webmasterworld.com/robots_txt/4429094.htm
I wouldn't even consider blocking /5 but I just can't help but notice the very high proportion of /4 users that aren't legitimate visitors.
I will do as you suggest though and read up more.
Brought to you by WebmasterWorld: http://www.webmasterworld.com