Contacts-Crawler again

[lucy24]

Previously observed here [webmasterworld.com]. One feature has changed; one hasn't.

138.201.49.195 - - [26/May/2017:07:28:19 -0700] "GET /robots.txt HTTP/1.1" 301 528 "-" "Contacts-Crawler/0.1 (+http://www.scrapinghub.com)" 
138.201.49.195 - - [26/May/2017:07:28:24 -0700] "GET /robots.txt HTTP/1.1" 200 739 "-" "Contacts-Crawler/0.1 (+http://www.scrapinghub.com)" 
138.201.49.195 - - [26/May/2017:07:28:28 -0700] "GET / HTTP/1.1" 403 1766 "-" "Contacts-Crawler/0.1 (+http://www.scrapinghub.com)" 

Rhetorical question: What are the chances that someone whose UA string contains the element "scraping" will be a good, benign and desirable visitor?

A bit later in the day they asked for the sitemap, which mercifully isn't subject to the "Allow from all" exemption, so no dice. At the time of these visits they were not denied in robots.txt; now they are, so I'll see if they ever hammer at the door again.

[keyplyr]

Rhetorical question: What are the chances that someone whose UA string contains the element "scraping" will be a good, benign and desirable visitor?

Rhetorical answer: Bot runners are typically not webmasters (except myself) and not often of the defensive mindset we've developed. Also, while "scraping" seems alarming to find in our logs for some of us, others don't seem to mind the term.

For my website interests, there are only a half-dozen beneficial agents with some version of "scrape" in the UA and I rarely see the term in referrers, except this case.

Agreed, it seems self-defeating to name yourself a "scraper." I always wondered why some referred to themselves as "hackers"... seemed to me like calling yourselves "theives."

[jonasjacek]

I don't want to get philosophical but 'hackers', in it original meaning, does not (necessarily) have a negative connotation.
Just saying.

[keyplyr]

Well that depends on what you believe the "original meaning" to be; original according to whom or what source?

Regardless, it's what it means to you that's important.