Welcome to WebmasterWorld Guest from 54.158.240.147

Forum Moderators: open

Message Too Old, No Replies

The Ask Jeeves/Teoma Spider is out of control!

     
7:30 pm on Feb 22, 2004 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Jan 16, 2003
posts:746
votes: 0


Anyone else seeing this relentless spider?
I'm over 200,000 requests so far this month.
I was never one to consider blocking a spider before but it's about time to tame this beast.
8:31 pm on Feb 22, 2004 (gmt 0)

Junior Member

10+ Year Member

joined:Dec 1, 2003
posts:120
votes: 0


Yes, same thing here. Being pounding for about two months now. Still does not return any useful results for spidered pages.

But I'm not disabling it. It's a pity there is no way in robots.txt to limit query frequency without total blocking.

11:07 pm on Feb 22, 2004 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Nov 3, 2003
posts:1092
votes: 0


Same story - Teoma hits new sites (right behind G-bot) - but NO traffic . ..
5:23 pm on Feb 23, 2004 (gmt 0)

New User

10+ Year Member

joined:Nov 16, 2003
posts:31
votes: 0


I had GIGS of transfer from the AskJeeves/Teoma bot in the past month or two and I have yet to be included in the index! Ridiculous.
12:58 am on Mar 5, 2004 (gmt 0)

Junior Member

10+ Year Member

joined:Feb 15, 2004
posts:124
votes: 0


same problem here. 12,000 requests in a 3-4 hour period. My mod_throttle actually kicked in.
3:47 am on Mar 5, 2004 (gmt 0)

Preferred Member

10+ Year Member

joined:July 5, 2002
posts:602
votes: 0


133K hits in January and 435K hits last month, all on one site. Total number of pages in Teoma: 0

I could say the same about AltaVista though.