Page is a not externally linkable
- Search Engines
-- Search Engine Spider and User Agent Identification
---- UNTRUSTED in Nokia User Agent


keyplyr - 9:04 pm on Jan 26, 2013 (gmt 0)



I have blocked all UAs containing "nutch" for well over 10 years without any adverse affect.

This generic bot can be used by any unaccountable agent for any unknown purpose, and the accountable ones should customize and rename so their bot UA reflects they are on the level IMO.


Thread source:: http://www.webmasterworld.com/search_engine_spiders/4440895.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com