drbobhmi - 5:02 pm on Dec 13, 2003 (gmt 0)

I have maintained a 250+ page, 42+ MB church site for the last 4 years with low activity (20-30 hits per day). Since 12/7/03, slurp has been running non-stop, getting pages and gobbling up bandwidth. I have contacted my web host daily and they keep zeroing out my monthly totals so that browsers can get to my site. They also installed a robot.txt file to inhibit ALL robots, but slurp continues to run and I am over my monthly limit of 5 GB every day with 200,000 to 400,000+ page accesses!

Inktomi claims to adhere to the standards that recognize robot.txt directives. Anyone know why they would be ignoring the following robot.txt entries?

User-agent: *
Disallow: /

Anyone else having this problem?

My web host has suggested that I set up my pages with a new member name. But would that really help? If inktomi finds us again, we'll be in the same boat.

Is there anything I could have done by mistake to trigger this looping? To my knowledge, I haven't done anything different than what I've been doing for years.

BTW: Inktomi has yet to respond to my queries.