Welcome to WebmasterWorld Guest from 54.146.217.179

Forum Moderators: Ocean10000 & incrediBILL

Message Too Old, No Replies

MSIECrawler

     
1:11 pm on Dec 2, 2007 (gmt 0)

Preferred Member

5+ Year Member Top Contributors Of The Month

joined:Aug 16, 2006
posts: 397
votes: 1


I got a HUGE amount of hits from this in the last 24 hours

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; MSIECrawler)

Can anyone please tell me what it is?

1:20 pm on Dec 2, 2007 (gmt 0)

Senior Member from CA 

WebmasterWorld Senior Member encyclo is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Aug 31, 2003
posts:9063
votes: 2


MSIECrawler is usually (unless someone's spoofing it) the user agent for Internet Explorer when downloading your site for offline browsing. See:

[msdn2.microsoft.com...]

Also: [google.ca...]

It should respect robots.txt:

User-Agent:MSIECrawler
Disallow:/
1:59 pm on Dec 2, 2007 (gmt 0)

Preferred Member

5+ Year Member Top Contributors Of The Month

joined:Aug 16, 2006
posts: 397
votes: 1


Thank you for your reply!