Anyone else seeing this relentless spider? I'm over 200,000 requests so far this month. I was never one to consider blocking a spider before but it's about time to tame this beast.
andrew_m
8:31 pm on Feb 22, 2004 (gmt 0)
Yes, same thing here. Being pounding for about two months now. Still does not return any useful results for spidered pages.
But I'm not disabling it. It's a pity there is no way in robots.txt to limit query frequency without total blocking.
howiejs
11:07 pm on Feb 22, 2004 (gmt 0)
Same story - Teoma hits new sites (right behind G-bot) - but NO traffic . ..
filo
5:23 pm on Feb 23, 2004 (gmt 0)
I had GIGS of transfer from the AskJeeves/Teoma bot in the past month or two and I have yet to be included in the index! Ridiculous.
pcgamez
12:58 am on Mar 5, 2004 (gmt 0)
same problem here. 12,000 requests in a 3-4 hour period. My mod_throttle actually kicked in.
Bobby_Davro
3:47 am on Mar 5, 2004 (gmt 0)
133K hits in January and 435K hits last month, all on one site. Total number of pages in Teoma: 0
