I didn't ask it to... what a pain in the nas, they could send an e-mail asking us, instead of their little pain in the nas creature sniffing my system, and the damn thing opened a socket too! little piece of crap! Someone to tell AV to sort themselves out.
Dont think it would really be practicle for Av to send an email to every site they intend to crawl.
The whole point of the crawler is to index web sites that can later be searchable by AV users. If your complaining about a legitamite search engine spider then you really cant be planning for any real traffic. You aint going to get real traffic without search engine listings and you aint going to get those without being crawled.
Effectively, it's an ecom site for locals, simply becuase shipping the products overseas isn't viable, we basically import the goods and sell them locally, also I don't mind say the system lookingonce, taking the url and buggering off, that's fine by me, but hogging the system for days is another thing........