Has anyone had this User Agent in their logs?
Mozilla/4.0 (compatible; MSIE 4.0; Windows NT; ....../1.0 )
(yes it does have the ...... as part of the user agent).
It hit my site (without reading robots.txt that I could see) from a variety of IP addresses at about 6 second intervals, it grabbed pages and images (including those dissalowed in robots.txt)
It would occasionaly add the following query strings to requests for folders
?D=A
?D=D
?M=A
?M=D
?N=A
?S=D
<added>(Figured out that this is because the default folder views have links to sort the contents either Ascending or Descending on Name, Last Modified, Size and Description)</added>
And it also requested images from /sys-icons/ (which I haven't got links to anywhere)