Welcome to WebmasterWorld Guest from 107.20.104.110

Forum Moderators: open

Message Too Old, No Replies

overzealous bot

teoma bot wrecked my server

     
5:55 pm on Aug 27, 2003 (gmt 0)

Full Member

10+ Year Member

joined:Jan 27, 2003
posts:266
votes: 0


Teoma/Ask jeeves has problems! Until further notice, they are banned from my sites! Their spider has problems... it requests too much, and I suspect it requests files in such a way as to crash the server.
5:58 pm on Aug 27, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:June 5, 2001
posts:2728
votes: 8


Check the programming for your site, if a simple spider crashes your server the pages being served have bugs in it. If its standard HTML blame the web server. If its coding then check the code and stress test the server. Think if your site was on CNN or slashdot one day, it would be down in less than a second from the traffic. :D

If it does pass varaibles or does anything server side check for alternative methods of doing so, if your using a premade package check for updates, fixes and patches.

10:11 pm on Aug 30, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:July 11, 2003
posts:955
votes: 0


Ask Jeeves/Teoma Bot works fine on my site, In fact , It requests less than Googlebot and FAST!
11:58 am on Sept 8, 2003 (gmt 0)

Full Member

10+ Year Member

joined:Jan 27, 2003
posts:266
votes: 0


The site's are not perfect, I know this. However, no other bots caused the same problems. In fact, google requests far more pages. It actually does not really matter. I have had no problems since I blocked the teoma robot. And, I do not seem to be missing the 10 visitors a day I used to receive from them.
 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members