Interesting site ripper/bot like critter

Try blocking this one (other than IP...)

[tangor]

Ripped whole site, one day---very fast, from one ip. The UA is the oddity...

173.192.34.nn


Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0; Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1) ; Media Center PC 5.0)

Even tried a few path modifications which dated back some 13 years...

[keyplyr]

173.192.0.0/15

I've had this Softlayer ranged blocked for ages. Sorry it got you.

[Pfui]

Saw it on 07-03 on a business site:

173.192.34.**-static.reverse.softlayer.com
Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0; Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1) ; Media Center PC 5.0)

robots.txt? Yes BUT ignored
Crawl Rate: 1 page/sec

You can block the IP/CIDR, ditto on the UA (the space before the last semi-colon).

FWIW, go to projecthoneypot.org, plug in the IP (um... 100-5=nn) and check out the third note about a follow-up switch-hitter also from Softayer (74.86.0.0/16).