| Welcome to WebmasterWorld Guest from 184.108.40.206 |
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
|Become a Pro Member|
404ing with strange requests
Any idea what M$N was up to here? I've had several requests today from M$N with similar addresses.
Agent: msnbot/0.11 (+http://search.msn.com/msnbot.htm)
Well something...unidentified from Microsoft is out pulling pages...IP 220.127.116.11. Anyone else seen this one?
msnbot has been deep crawling my sites in the last few weeks.
Wondering myself what and where and how to access what he's indexing. Checked all logical, found nothing.
been hit hard the last couple of days from:
Yesterday she grabbed about 1/3 of my site and it was the deep 1/3.
I am facing the same situation, msnbot 0.11 spending most of the time on site. I have a question
1. Where will the results be used
If nowhere, is it feasible to ban it for now
I noted it as well for the past 5 days. Got crawled about 450,000 pages already and another 1.5 million pages to go.
I do hope that they will add one day the crawled pages to their index, otherwsie it is a waste of bandwidth as they have taken around 7gb per day for the past 5 days.
18.104.22.168 has been on one of my sites for 38 hours.
And, it has a $200,000 shopping cart in progress with over 300 items in it.
I hope Mr. Gates has room on his credit card.
All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved