Forum Moderators: open
IP: 207.31.251.130 belonging to Verio Inc, in Colorado, US. got robots.txt then index.page:
207.31.251.130 - - [09/Oct/2003:09:08:00 -0700] "GET /robots.txt HTTP/1.1" 200 1173 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)"
207.31.251.130 - - [09/Oct/2003:09:08:01 -0700] "GET / HTTP/1.1" 200 11128 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)"
207.31.251.130 - - [09/Oct/2003:09:08:17 -0700] "GET /scripts/script1.js HTTP/1.1" 200 73 "-" "JScript Processor"
207.31.251.130 - - [09/Oct/2003:09:08:17 -0700] "GET /scripts/script2.js HTTP/1.1" 200 388 "-" "JScript Processor"
....etc.
Sounds like a bot to me though, reading robots.txt first then going for disallowed scripts. JScript Processor must be a separate component used to parse javascripts since most crawlers choke on JS.
Corex Technologies (NETBLK-COR-207-31-251-128)
810 Memorial Drive
Boston, MA 02139Netname: COR-207-31-251-128
Netblock: 207.31.251.128 - 207.31.251.159
Maintainer: A018
It's a company producing business card scanners, so I can't see a good reason for them to run a spider through my site. Apart from that, they try to hide behind a generic UA most of the time, and rip the pages (plus *.js and *.css files) at a speed of more than one per second.