Welcome to WebmasterWorld Guest from 22.214.171.124
Forum Moderators: open
Inktomi claims to adhere to the standards that recognize robot.txt directives. Anyone know why they would be ignoring the following robot.txt entries?
Anyone else having this problem?
My web host has suggested that I set up my pages with a new member name. But would that really help? If inktomi finds us again, we'll be in the same boat.
Is there anything I could have done by mistake to trigger this looping? To my knowledge, I haven't done anything different than what I've been doing for years.
BTW: Inktomi has yet to respond to my queries.
Our FAQ is at: [support.inktomi.com...]
How do I stop your robot / spider (Slurp) from crawling my site?
I read the thread over the weekend and this morning discussed the problem you are referring to with a few people but really need some examples to diagnose further if and where this is happening. Please sticky some examples to me.