Monthly Robots.txt Request From securityspace.com

[incrediBILL]

Every month, like clockwork since 2007, I get this same exact request from securityspace.com.

67.19.79.218 - - [20/Nov/2007:01:31:04 -0600] "GET /robots.txt HTTP/1.0" 200 27 "-" "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT)"
.
.
.
67.19.79.218 - - [26/Jul/2011:14:51:06 +0000] "GET /robots.txt HTTP/1.0" 200 203 "-" "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT)"
67.19.79.218 - - [23/Aug/2011:22:50:31 +0000] "GET /robots.txt HTTP/1.0" 200 203 "-" "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT)"
67.19.79.218 - - [25/Sep/2011:07:50:48 +0000] "GET /robots.txt HTTP/1.0" 200 203 "-" "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT)"
67.19.79.218 - - [22/Oct/2011:08:34:13 +0000] "GET /robots.txt HTTP/1.0" 200 203 "-" "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT)"

IP never asks for anything other than robots.txt and goes away.

Anyone got a clue why?

[Pfui]

No clue, sorry. Just that last April, its kin hit a low-traffic, minimally-guarded site, asked for robots.txt and then behaved properly:

ns2.securityspace.net [projecthoneypot.org...]
Mozilla/4.0 (compatible; MSIE 5.01; Windows NT)

ns2.securityspace.net = ns2.securityspace.com = 66.36.230.78 [robtex.com...]

I've not noticed it on my largest, busiest site, probably because I automatically block Host names containing variations of the word secure (...thanks to trouble from Host names like: esecuredata, insecure, internetsecure, safesecureweb, secure-ss, securecomputing, securenet, securepoint, securepop, secureprivatenetwork, securemote, secureserver, secureserverdot, securesites, security-lab, tvsecure).

[Pfui]

Bill, your IP tweaked (and/or screwed up:) its UA very slightly along the way. From [projecthoneypot.org...]

Mozilla /4.0 (compatible; MSIE 5.01; Windows NT)
Mozilla/4.0 (compatible; MSIE 5.01; Windows NT)

[tangor]

I've cut many of my problem UAs by targeting "MSIE 5.01" ... there's no reason anyone should be running that these days.

[incrediBILL]

I cut MSIE 5.01 as well, I'm just curious why they're checking my robots.txt once a month.

Are they compiling stats on allowed and denied bots perhaps?

Kinds of a cool idea if you think of it, see who's blocking what and run some stats to get the feel of the webmaster mind share when it comes to allowed crawlers.

Of course more webmasters don't even know what a robots.txt file is so who am I kidding.