PrivateSearch

[keyplyr]

UA: PrivateSearch/0.1.0 (Polite Nutch Crawler; grierforensics.com)
Protocol: HTTP/1.0
Robots.txt: Yes
Host: AWS
52.0.0.0 - 52.31.255.255
52.0.0.0/11

Web forensics research

[tangor]

Chuckles (on my side) as I've nuked "nutch" for many years. Not sure I'd change my mind for "polite nutch".

[keyplyr]

IMO there's nothing inherently bad about Nutch; it's just a simple crawler anyone can freely use. It's what they use it for that I care about. I filter "nutch" and other generic UAs and let through the ones I like. So far I allow about 30.