Forum Moderators: open

Message Too Old, No Replies

CRMNLCrawlAgent

         

keyplyr

7:42 pm on Aug 11, 2018 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



UA: CRMNLCrawlAgent/Nutch-1.15-SNAPSHOT
Protocol: HTTP/1.0
Robots.txt: Yes
Host: sentia.com (vellance.com)
79.99.184.0 - 79.99.191.255
79.99.184.0/21

Canada Montreal Centre De Recherche Informatique De Montreal
crim.ca

Previous UAs:
CRIM Crawler/Nutch-2.3 (Crawler du Centre de Recherche Informatique de Montr\xc3\xa9al (CRIM))
Mozilla/5.0 (compatible; heritrix/3.2.0 +http://www.crim.ca)

lucy24

12:21 am on Aug 12, 2018 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



You gotta admit, CRMNL or CRIM is a pretty droll UA name. Can't call it a linguistic oversight, since it would look exactly the same in French.

NickMNS

4:07 am on Aug 13, 2018 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



@Lucy24 was using the word "droll" supposed to be a pun? Drole (with an accent over the o) is French for funny or strange, it's where the English word is derived from.

keyplyr

4:27 am on Aug 13, 2018 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Back on topic... also seen coming from Canadian ISP

UA: CRMNLCrawlAgent/Nutch-1.15-SNAPSHOT
Protocol: HTTP/1.0
Robots.txt: Yes
Host: rogers.com
142.146.0.0 - 142.146.255.255
142.146.0.0/16

lucy24

5:24 am on Aug 13, 2018 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



it's where the English word is derived from.
Heh, that’s funny, I always assumed we stole it from German. Dictionary says it was originally MDu, of all things. (This is the second consecutive time I've looked something up and found it traced back to Middle Dutch--a language I never formally knew existed, though if you think about it, it obviously had to. Fancy that.)

So, it’s a criminal robot that, Nutch-like, reads robots.txt. Huh. And crawls from ordinary public Canadian ISPs?

keyplyr

5:37 am on Aug 13, 2018 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



You're obviously not from Pennsylvania.

It could be a renamed organic Nutch. A lot of them do that. Nutch is free, easy to use and returns clean results. I ran it for a while.