Page is a not externally linkable
- Search Engines
-- Search Engine Spider and User Agent Identification
---- British Library scraper


jmccormac - 9:12 am on Aug 16, 2012 (gmt 0)


A British Library record for a website might be a very powerful thing if it came taking copyright over an infringement but the idea of the BL approaching the web as it does with print is interesting. Possibly one to whitelist.

Regards...jmcc


Thread source:: http://www.webmasterworld.com/search_engine_spiders/4484756.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com