Welcome to WebmasterWorld Guest from 54.147.217.76

Forum Moderators: mack

Message Too Old, No Replies

Microsoft crawling websites?

I found a microsoft bot crawling my pages.

   
4:30 am on Jul 17, 2003 (gmt 0)

10+ Year Member



Does anyone know what microsoft is doing with the following bot?

Microsoft URL Control - 6.00.8862

It has been crawling quite fast through my site and I am curious to know if this is the crawling process for the microsoft search engine.

4:41 am on Jul 17, 2003 (gmt 0)

WebmasterWorld Senior Member jdmorgan is a WebmasterWorld Top Contributor of All Time 10+ Year Member



akande,

Microsoft URL Control - 6.00.8862 is NOT a Microsoft robot, it is a "control" that allows programs running on Windows to access the web. You might want to check over in the Search Engine Spider Identification forum or do a site search for more info. This user-agent is on many "block lists" as a possible e-mail harvester or site-grabber.

teeceo, I don't think MS will publish their MSNBOT-based index until it is well-tested. So it's going to be awhile, IMO.

Jim

4:46 am on Jul 17, 2003 (gmt 0)

WebmasterWorld Administrator mack is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



from what I understand Microsoft URL Control is a bad bot typicaly a form mail sniffer.

It is nothing to do with msn search.

The msn search true bot is Msnbot

Ban it in htaccess.

RewriteCond %{HTTP_USER_AGENT} ^microsoft.url [OR]

Mack.

added what jdMorgan said :)

8:20 pm on Jul 17, 2003 (gmt 0)

10+ Year Member



Thanks for the post replies.