Welcome to WebmasterWorld Guest from

Forum Moderators: goodroi

Message Too Old, No Replies

How to ban inktomisearch.com's bots?

I have been hit hard with so many requests by this.



5:21 pm on Apr 11, 2004 (gmt 0)

10+ Year Member

I think this(inktomisearch.com slurp) is from pure inktomi, right? Isn't the Yahoo Slurp use different names?

I use AWStats and inktomisearch.com slurp has been hitting my sites so hard that it knocked down the site a few times already in the past few weeks.

The site is ranked well on Pure Inktomi and Yahoo as well as Google.

I don't want to lose the traffic from Yahoo and Google but it is OK to lose the traffic from MSN or others that use pure Inktomi feed, since it is less than 10% of total SE traffic.

I don't like it crashing my site with more than 10,000 hits a week to my site.

So, my question is "If I ban
inktomisearch.com's slurp, am I banning Yahoo slurp, too or just pure inktomi slup?"


5:22 pm on Apr 11, 2004 (gmt 0)

10+ Year Member

And what user agent name do I have to use to ban the pure inktomi slup in the robots.txt?


5:24 pm on Apr 11, 2004 (gmt 0)

10+ Year Member

The site is a forum site with tens of thousand pages with one file name "forum.asp" with all the parameters after question mark for the category ID and post ID, etc. So, If I ban the slurp for the "forum.asp" it will ban the slurp from all my forum pages.

Featured Threads

Hot Threads This Week

Hot Threads This Month