Forum Moderators: phranque

Message Too Old, No Replies

aranhabot

driving me mad...and costing me money

         

esllou

1:47 pm on Mar 22, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



didn't know where else to put this....

the last three months, "aranhabot" has used up between 8 and 9% of my total bandwidth....I am using 22gb a month so this is not small cheese!

have altered my robots.txt and added them to my bad bot .htaccess script too.

who the hell are they? sommat to do with alexa/amazon I heard once. did a search on google for them and out of four billion pages in their index, only one reference to them....and that here at WebmasterWorld and only in one sentence!

UrbanEcho

1:55 pm on Mar 22, 2004 (gmt 0)

10+ Year Member



This may help....

http://www.webmasterworld.com/forum11/1515.htm [webmasterworld.com]

esllou

9:50 pm on Mar 22, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



thanks, but that was the reference I had seen. anyone know of anything else about it? can I stop it using .htaccess or robots.txt?