experienced - 5:32 am on May 21, 2011 (gmt 0)
is there any way to protect my site from the scraper bots. they are eating my website bandwidth and i am very tired of this now. almost 600% bandwidth is used by the spam bots in comparision of my real user viewed bandwidth.
is there any way that i can just allow, top engines to crawl my sites. so i dont get any ok,ok bot even.
i have been trying htaccess to block ip but it is very hard as ip gets changed regualrly.