Forum Moderators: DixonJones

Message Too Old, No Replies

Mozilla/4.0 (compatible; MSIE 4.0; Windows NT; ./1.0 )

From Multiple IP's, no robots.txt, grabbed pages & images

         

Dijkgraaf

9:58 pm on Nov 17, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Has anyone had this User Agent in their logs?
Mozilla/4.0 (compatible; MSIE 4.0; Windows NT; ....../1.0 )
(yes it does have the ...... as part of the user agent).
It hit my site (without reading robots.txt that I could see) from a variety of IP addresses at about 6 second intervals, it grabbed pages and images (including those dissalowed in robots.txt)
It would occasionaly add the following query strings to requests for folders
?D=A
?D=D
?M=A
?M=D
?N=A
?S=D
<added>(Figured out that this is because the default folder views have links to sort the contents either Ascending or Descending on Name, Last Modified, Size and Description)</added>
And it also requested images from /sys-icons/ (which I haven't got links to anywhere)

Hetta

4:07 pm on Nov 24, 2005 (gmt 0)

10+ Year Member



Aye, it grabbed 800 files on Nov20. dunno what that is.

bull

7:56 pm on Nov 25, 2005 (gmt 0)