Page is a not externally linkable
- WebmasterWorld
-- Webmaster General
---- bandwidth Issue


experienced - 5:32 am on May 21, 2011 (gmt 0)


is there any way to protect my site from the scraper bots. they are eating my website bandwidth and i am very tired of this now. almost 600% bandwidth is used by the spam bots in comparision of my real user viewed bandwidth.

is there any way that i can just allow, top engines to crawl my sites. so i dont get any ok,ok bot even.

i have been trying htaccess to block ip but it is very hard as ip gets changed regualrly.


Thread source:: http://www.webmasterworld.com/webmaster/4315608.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com