homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Advertising / Paid Inclusion Engines and Topics
Forum Library, Charter, Moderator: open

Paid Inclusion Engines and Topics Forum

  posting off  
Inktomi Bot: busy at the moment
Its going mental!

 9:51 pm on Mar 3, 2003 (gmt 0)

I have only recently implemented some bot tracking on my server and Inktomi is constantly spidering my site. My tracking has been in place for a week and slurp is showing up pretty much all day every day. Is this usual behavior? Im quite familiar with Googlebots idiosyncrasies but slurps activities seem a little excessive. Any comments or advice would be greatly appreciated.



 10:32 pm on Mar 3, 2003 (gmt 0)

Are you doing PFI?

You are paying to be re-visited every 48 hours, right?


 10:40 pm on Mar 3, 2003 (gmt 0)

I dont pay ink but slurp still shows up about twice a week and takes quite a lot of pages.

could be viral marketing :)

you see it so you try it.


 11:08 pm on Mar 3, 2003 (gmt 0)

Slurp has visited my site almost every day last month. It grabs the index page up to 3 times a day and one or two other pages, grabbed most of our pages twice during the month. Mozilla/5.0+(Slurp/cat;+slurp@inktomi.com;+http://www.inktomi.com/slurp.html)

Site is not PFI. Wonder what Yahoo is planning.


 11:16 pm on Mar 17, 2003 (gmt 0)

I tend to agree that Yahoo/Ink must have something up their sleeve. On one of my sites, I only paid for the index page, and the whole site is now being slurped, but none of the crawled pages are showing up in the SERPs, only the submitted index.

On another newer site, I submitted 10 pages, and only those pages are getting slurped for now.

I think they're warehousing & testing for an eventual go at Google, but I guess that's pretty obvious.


 11:21 pm on Mar 17, 2003 (gmt 0)

None of my sites on a particular server are paid inclusion in anyway and slurpy is all over them. Shows with enough links you can be included without paying :P


 11:25 pm on Mar 17, 2003 (gmt 0)

There's nothing unusual with Slurp constantly visiting the robots and index page.
What is more unusual is the deep spidering. A good thing too, their db will eventiually be more up-to-date and thereby more attractive to use.


 11:30 am on Mar 18, 2003 (gmt 0)

I have not paid anything, have only had my site up for a week, and have been spidered already several times by inktomi, and I assume as a result already show up in msn and hotbot/inktomi searches. (my url, not with keywords)

It has only grabbed the index page so far, but maybe it will come back for the other ones later since they have absolute URLS.

How to tell how far down I am in the rankings, I don't know however. Anyone know what kind of algorithm they use at msn or inktomi?

Alternative Future

 11:33 am on Mar 24, 2003 (gmt 0)

Hi all,

Same thing here Ink is going mental in my site hitting anywhere between 400 & 500 pages a day, when it leaves it does not come back for at least a week or so later...

Doe's anyone have any idea as to when we might start seeing the results?



 4:47 pm on Mar 25, 2003 (gmt 0)

I have a PFI and Slurp hasn't been by for a week now.


 5:16 pm on Mar 25, 2003 (gmt 0)

Inktomin only hits the specific pages I paid for inclution every day at the same time. I sure wish it would go mental on my site and spider more pages.

Through my Pay for Inclution provider, I have edited the URL of the pages I want spidered and have noticed the original page has been sticking around in the MSN SERPs. I wonder how long before it drops off? If it takes a while, I'll rotate my 5 paid for inclution URLs to every page on my site over the course of a month.

Global Options:
 top home search open messages active posts  

Home / Forums Index / Advertising / Paid Inclusion Engines and Topics
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved