Forum Moderators: goodroi

Message Too Old, No Replies

Mainseek bot

this bot is ripping through our site

         

ssgumby

1:31 pm on Oct 13, 2006 (gmt 0)

10+ Year Member



Anyone know what this BOT is? It does NOT obey robots.txt. It is grabbing every file on my site and has been doing so for days now.

Any idea how I can block this completely?

Thanks

goodroi

6:36 pm on Oct 14, 2006 (gmt 0)

WebmasterWorld Administrator 10+ Year Member Top Contributors Of The Month



if a bot is ignoring the robots.txt, you can try to stop it by blocking the ip address it is coming from.

ps they discussed mainseek a little more over here [webmasterworld.com...]