Just in case you didn't know. It's been cralling a lot over the last 24 hours. IP Address: 209.67.247.232 Hostname: rdcrawler3.uswal.alltheweb.com User agent: Mozilla/4.0 (compatible; MSIE 4.01; Win32)
msgraph
9:46 pm on Nov 29, 2000 (gmt 0)
Getting the same spider on about a 100 of my domains. All within a 10 minute period. Definitely not a common user.
Edited by: msgraph
msgraph
9:46 pm on Nov 29, 2000 (gmt 0)
Getting the same spider on about a 100 of my domains. All within a 10 minute period. Definitely not a common user.
cirelle
4:43 pm on Dec 7, 2000 (gmt 0)
had about 50 domains hit 1 each between 11-28 and 11-29 none since.
c
PeteU
2:11 am on Jan 5, 2001 (gmt 0)
its at it again
littleman
2:18 am on Jan 5, 2001 (gmt 0)
Yes, it certainly is! Thanks for the headsup, Pete.
PeteU
2:26 am on Jan 5, 2001 (gmt 0)
heh, Littleman, somehow I knew you'd be the first to reply ;) seems like its a very organized crawl, its pulling alphabetically, all my hosts starting with 'a' are being requested nd only index pages
PeteU
2:30 am on Jan 5, 2001 (gmt 0)
oops..
Air
2:57 am on Jan 5, 2001 (gmt 0)
Yup, seeing it here too..
cirelle
9:05 pm on Jan 5, 2001 (gmt 0)
saw that one on 1/4 at about 10:30pm wasn't alphabetical though
