Wondering myself what and where and how to access what he's indexing. Checked all logical, found nothing.
papamaku
9:31 am on Feb 11, 2004 (gmt 0)
been hit hard the last couple of days from:
65.54.188.31
Powdork
9:35 am on Feb 11, 2004 (gmt 0)
Yesterday she grabbed about 1/3 of my site and it was the deep 1/3.
domingo
4:33 pm on Feb 11, 2004 (gmt 0)
I am facing the same situation, msnbot 0.11 spending most of the time on site. I have a question
1. Where will the results be used
If nowhere, is it feasible to ban it for now
asinah
3:29 pm on Feb 12, 2004 (gmt 0)
I noted it as well for the past 5 days. Got crawled about 450,000 pages already and another 1.5 million pages to go.
I do hope that they will add one day the crawled pages to their index, otherwsie it is a waste of bandwidth as they have taken around 7gb per day for the past 5 days.
Robino
3:31 pm on Feb 12, 2004 (gmt 0)
65.54.188.17 has been on one of my sites for 38 hours.
And, it has a $200,000 shopping cart in progress with over 300 items in it.