Slurp/si bot hitting only robots.txt every day, dropped from index.

Can i hope for reindex?

[GreenLeaf]

Hi All!
My site has been indexed by Inktomi for free about a half a year ago. Then, about 3 months ago Ink dropped all my pages and i lost all MSN traffic.
Now i have Slurp/si bot visiting my site every day but hitting only robots.txt.
Does it mean that i got some penalty and it's impossible for my site to be indexed for free now?
Thanks in advance.

[sidyadav]

There could possiblly be an error in your robots.txt, which triggers it to leave your site and comeback another day.

Get your robots.txt file checked at [searchengineworld.com...]

And I know this seams almost impossible, but is your robots.txt or .htaccess banning any of Inktomi's UAs or IPs?
Check them just incase.

Sid

[GreenLeaf]

Thank you, Sid, for your advice.
I checked my robots.txt - no errors detected.
It's simple:

User-agent: *
Disallow: /admin/
Disallow: /webstat/
Disallow: /test/

My .htaccess is unchanged since site's beginning.
I still can't understand the situation.
?

[budterm]

If your site is dropped because it is banned, it won't help to pay for inclusion. I know from experience on this one. If anybody has had other experience, I would be very interested to hear.

[tennismaster]

Greenleaf,

Our experience is exactly the same as yours,
and we have been told that our site has picked up
a penalty. The slurp bot stills comes every day,
but only takes robots.txt now. PFI will not help
your pages will be ranked last on any search.
All of our free pages are gone fron the index.

TM

[MarkHutch]

This happened to us too on several domains. All of them had pages listed via PFI until about six months ago. When we didn't renew the PFI, all pages including the non paid ones disappeared within a couple of months and the crawling stopped. Could these penalties have anything to do with dropping the pay program? I'm beginning to think so because we have several other domains that were never included in the PFI and they are crawled and updated on a regular basis even today.

[GreenLeaf]

Bad.
Does anybody know what could be the reason for a ban in Ink?
Does it mean that it's a manual ban? Is it for ever?

[Tim]

If you are banned, you know what the problem was and have fixed it. Please send an email to [e-mail] and ask for them to let you back in the index. I would explain what you did wrong and how you have resolved the problem. There is no guarantee that this will get you back in the index but this is the channel we have set up for these sorts of issues.

[GreenLeaf]

Thank you very much, Tim, for this address.
The problem is that i don't know what could trigger a ban.
What are the possible reasons? What Ink doesn't like?
Could you be so kind and share your experiences/guesses on that?
Thanks.

[Tim]

There is a content policy listed on Inktomi.com It does not cover every technique as new techniques tend to pop up all the time. Also spam tends to be the extent to which people use seo techniques not just using the technique itself.

Looking through the archives of WebmasterWorld is always a helpful resource too.
Tim

[GreenLeaf]

Thanks, Tim and all!

[helenp]

sorry to reopen this post,
but looking for answers for my problem Inktomi only spiders robots.txt since 13/1, I found this post, and read the anser from Tim saying: If you are banned, you know what the problem was and have fixed it. Please send an email to [e-mail] and ask for them to let you back in the index.

Well I donīt know the problem, but anyway just wrote to that e-mail address, but it isnīt valid, keeps coming back. What can I do please.......

Pd, The new yahoo spider does the same.

[Tim]

Remove the "removethis" from the email address.

[helenp]

Hi Tim,
I wrote, suppose take long to get an answer.
Is there a way to know if you ever been indexed before?
I donīt know if been indexed and dropped, or ever indexed, though before didnīt have stats of site.
This really drives me mad, tried everything, and I donīt have and havnīt ever had any spam on my site, only thing I can imagine is: for years ago I was on an linkfarm, but took away quickly when understood that wasnīt to good, to start with I didnīt know anything so just signed up where I could, and maybe Inktomi ever goes back to check if situation changed.
Just checked the linkfarms and can see sites on there listings that are indexed by Inktomi, so that shouldnīt be the reason either.
Thanks in advance