Mozilla/4.0 (compatible; MSIE 4.0; Windows NT; ....../1.0 )
spider comes from MS IP-addresses
RonPK
11:22 am on Oct 16, 2003 (gmt 0)
It requests pages and pictures, and ignores instructions in robots.txt and meta tags. It comes from several numbers in the 131.107 range and 207.46.225.xxx: Microsoft.
Wilderness, you're pointing to a message that points to a thread about MSNBOT. If you're implying that ....../1.0 is the successor of MSNBOT/0.1, I think you might be right.
wilderness
5:00 pm on Oct 16, 2003 (gmt 0)
Ron, Ms can call or deem the UA by whatever they they desire (and they nearly have,) what's significant is that all the activity comes from the same IP ranges.
