Crescent Internet ToolPak HTTP OLE Control v.1.0

Should I ban it in my robots.txt file?

[merlin78]

Hi all,

I was wondering what people generally think about the Crescent Internet Toolpak? In one week I was hit with 81,000 hits from it and was wondering whether I should ban its access through the robots.txt file or should I not be concerned at all? Will banning it restrict any visitors from accessing the site?

Any help would be much appreciated.

[littleman]

Hi Merlin,
The thing is that rouge spiders will seldom obey robots.txt, you would be better off banning the bugger using a .htaccess file which will deny it entrance to your server.