Welcome to WebmasterWorld Guest from

Forum Moderators: open

Message Too Old, No Replies

Inktomi Questions

How to encourage the crawl?

9:03 am on Feb 4, 2003 (gmt 0)

Preferred Member

10+ Year Member

joined:Sept 22, 2002
votes: 0


I have been paying more attention to Inktomi, now that Yahoo has bought it. I used to submit sites to it for free by using Anzwers or SwissSearch, but these are gone. Doe anyone know how to get Inktomi to crawl a new site that has no links. I noticed that all my older sites are listed and really well ranked. However, I can't find the Inktomi spider in my web logs. How do I check for it?
I mainly target Google's algo, but have found them to work on Inktomi as well. Does anyone know of any additional tactics which improve Inktomi rankings?
Sorry for the 20 questions, but I figure I might as well get them all asked at once.
Thanks for any help :)

3:14 pm on Feb 4, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Sept 16, 2001
votes: 0

Inbound links is by far the easiest way to get listed.

You can do a search in your logfiles for something like:
Mozilla/3.0 (Slurp/si; slurp@inktomi.com; [inktomi.com...]

"Slurp" is always in the User Agent for the Inktomi.

3:18 pm on Feb 4, 2003 (gmt 0)

Junior Member

10+ Year Member

joined:Aug 20, 2002
votes: 0

As Laz said, inbound links are the key.

Inktomi has two types of spidering...

1.) PFI - the paid inclusion spider will visit any page that you pay the spidering fee for. It will index (and refresh) all the content on those paid pages several times a week. But...it won't follow links to other pages of your site.

2.) Natural Inclusion - this spider works more like Googlebot...it fully spiders the content, links, etc.. on a web page as it works its way though the net. So, the key is to get an incoming link to your site from a web site that is already included in the "natural inclusion" list. There's not really any way to tell by looking at a site if it is naturally included, so you need to work on building up incoming links and trust that eventually, you'll gain a link from a site that IS included, and that Inktomi will find, and fully spider your site that way.

4:26 pm on Feb 4, 2003 (gmt 0)

Preferred Member

10+ Year Member

joined:Sept 22, 2002
votes: 0

The reason I asked, if there was way to get a new site that was without links, was I wanted to know if there was a way to speed up the process. In the past by submitting to these other search engines it was a quick and simple solution. This new site I not my own. I just want to get it listed but the owners haven't attempted to get links.

To my other question are there any special SEO tricks?