Forum Moderators: open
MSIECrawler is not a true web robot. It is usually to be found in site logs if someone bookmarks a page whilst offline. Internet Explorer will then download the page and all links related to it, including links, images, JavaScript and Style sheets, when the user is next online.
MSIECrawler is not a true web robot
I didn't feel it needed an explantion ;)
It's in most UA deny lists.
For the benefit of those not aware?
There are a variety of tools similar to this for keeping bookmarks current.
The majority of browser users are not even sure how to confgure such things and are even less aware that their computer is verifying links in the background, unbeknown to their current activity.
From a webmasters point-of-view, we have a tool (bot) using or bandwidth unnecessarily. Perhaps even for a page the visitor will never return to.
Don