bl.uk lddc bot

[keyplyr]

UA: bl.uk_lddc_bot/3.3.0-LBS-2016-02 (+http://www.bl.uk/aboutus/legaldeposit/websites/websites/faqswebmaster/index.html)
Protocol: HTTP/1.0
Robots.txt: Yes
Host: British Library (bl.uk)
194.66.224.0 - 194.66.239.255
194.66.224.0/20
Parent: Janet
194.66.0.0 - 194.66.255.255
194.66.0.0/16

The British Library and other Legal Deposit Libraries are entitled to copy UK-published material from the internet for archiving under legal deposit.

Archived threads:
[webmasterworld.com...]
[webmasterworld.com...]

[dstiles]

Entitled, maybe, but I've blocked them for years; along with archive.org.

[keyplyr]

I've blocked them for years; along with archive.org.

As do I. Odd they would attempt to scrape sites in the US. We have no "entitlement" to allow them to copy websites.

archive.org does it but AFAIK they are not "entitled" by any law and early on I had to file a C&D to have my sites removed. Since then I block all their ranges & UAs.