Inktomi's Slurp vs Google's googlebot

How the two differ in indexing

[esquire]

I've found astounding differences in how Inktomi's spider (slurp) differs from Google's (googlebot). To get a top spot in the Inktomi search, all you need is a good title, description, keywords, and page copy (text). As a Web Developer, I have launched sites that had zero links pointing toward them and ended up with the #1 spot in the Inktomi search. This database is used by Overture, MSN, AskJeeves, and now Yahoo. In other words if you have a #1 position on MSN, you'll have the same for Overture, and AskJeeves, Excite, and HotBot. You don't need link popularity for "slurp." Google is different though, you need to be popular, but if you have a #1 spot in Google, it'll be the same as Netscape, FAST(AllTheWeb) and AOL.

Now here is the trick. Alta Vista's robot (infospider) has been hanging out with googebot lately, and is now adapting its mannerisms. Also if you have 200 pages and you don't want to pay Alta Vista for Express inclusion, all you have to do is submit one page with a simple code that will cause infospider to include all the pages. The same applies with googlebot. It remains to be seen how the acquisition of Inktomi by Yahoo! will affect the Yahoo results. The smaller search engines really don't matter as long as you're with the heavyweights (dmoz and inktomi database) because that's where they generally get their results. So you really you need not submit to 2,000 search engines, it will save you a lot of time, and you won't be bombarded with span.

As for link popularity, "One Good Link can do more for you than the top spot on Yahoo." Believe me, it's better to be in the #2 or #3 spot, than it is to be in the number one spot. You'll get a lot more visits that way. In the meantime, keep building your link directory.

[scottpes]

Hi Esquire,

I'm kind of new to Webmasterworld myself. What is the code you can use to get the spiders to fully crawl your site when they drop by?

Thanks,

Scott

[esquire]

Hey Playa,

You wanna hang with the big dogs don't you? There are actually two codes that you will find helpful
1. <meta name="robots" content="index, follow">
2. <meta name="robots" content="All">

*Don't use both codes at the same time. Pick the one you want (Place it within the head of the page)

Let's talk about code#1. That instructs the robot to index your page, and follow all the links on it. Code#2 instructs the robot to index everything! If you have fifty pages, all you have to do is submit just one page with that code, and your entire Web site will be indexed. (Saves you money.) Now if there are some pages you don't want indexed, use this code right here.
<meta name="robots" content="no index, follow">

This instructs the robot not to index the page, but follow the hyperlinks. It's something you have to try out yourself to guarantee that it works. I'm not giving you a 100% guarantee that it will work, unless you yourself take that next step. For me it has done wonders as far as my Website is concerned, so give it a try, you have nothing to lose. Remember, content is King. (The text in the page.)

[scottpes]

Thanks Esq.

I put code#2 in my header. Let the crawling begin.
And yes I DO want to hang with the big dogs!

Scott

[martinibuster]

Forget it. I'm not going to touch this one.

I was going to say something about using spanish fly and stuff, but forget it.

I'm not touching this.

[tennismaster]

Esquire,

I thought that Askjeeves uses the Teoma Database - not
Inktomi.

Also Fast(Alltheweb) has nothing to do with Google does it?

TM

[Conard]

To late martinibuster, you touched it:)
Hmmm...Big Dogs

[seth_wilde]

if you have a #1 position on MSN, you'll have the same for Overture, and AskJeeves, Excite, and HotBot

Ask Jeeves does not use Inktomi and the SE's that do, use there own algo, so you will not rank the same across the board..

it's better to be in the #2 or #3 spot, than it is to be in the number one spot

That's goes against every traffic study ever published... But I guess if your content being 2 or 3 I'm more than happy to take the top spot

1. <meta name="robots" content="index, follow" 2. <meta name="robots" content="All">

These codes will have no effect on paid spiders.

[esquire]

Hey TennisMaster,

AskJeeves and Teoma use the same database for their search results. That database is none other than INKTOMI. That's right! So making a search on either Ask.com or Teoma will almost guarantee the same results. Have fun!

[edited by: heini at 10:19 pm (utc) on Jan. 30, 2003]
[edit reason] No urls or specific search terms allowed ¦ thanks [/edit]

[seth_wilde]

AskJeeves and Teoma use the same database for their search results.

This is TRUE

This database is none other than INKTOMI

This is 100% FALSE

[martinibuster]

Sorry esquire,
You can read the history of the Teoma search engine here [sp.teoma.com], at the Teoma web site.

And Inktomi here [inktomi.com], at the Inktomi web site.

Teoma is a Rutgers University Project. Inktomi is a Berkeley University project.

The best place to find out about the search engines is at the search engine's own web site.

[esquire]

Hey Seth and company,

I'm gonna have to disagree with both of you on this one. AskJeeves and Teoma actually use Inktomi. Do this go to [ask.com...] when you get there, look at the very bottom on your right hand side at the AskJeeves Web site. There is a link that says 'Submit a Site.' Click on that link and it will take you to a registration area. In that area, there is a logo that says "Managed by ineedhit.com" Now get out of that area and click on this link [ink.ineedhits.com...] (Notice it begins with ink. meaning INKTOMI) Now remember AskJeeves is managed by ineedhits.com. So when you've click the link it takes you in the INKTOMI Database. Ironic isn't it. They then go ahead and explain to you that ineedhits.com has partnered with INKTOMI to supply search results. Oh, by the way, AltaVista too signed with ineedhits.com and guess where ineedhits is going to get its search results?

As for the results differing on MSN, Overture, and AskJeeves, I disagree. I'll try you, playa. Go to MSN.com and type in the keywords 'best flash music sites' my Web site comes out #1. go to Overture and type the exact keywords. Again I come out #1. Then go to your friend "Jeeves" with the same keywords and ask him. I'm still on top. This is because they use INKTOMI, and the results are almost identical. (Ignore the Sponsored Listings, those are pay-per-click listings) I've held the #1 spot for 12 months in a row.

No, I wouldn't settle for #2 or #3, it's just I get a better response if I'm in the second or third slot, than I do in the #1 slot. There is a simple explanation - That's where the 20/20 vision focuses when you receive search results, and that's where surfers tend to click. Other than that, keep on "Power-Linking."

[edited by: Marcia at 11:21 pm (utc) on Jan. 30, 2003]
[edit reason] no sigs, URLs, or specifics please. [/edit]

[tennismaster]

Esquire,

I subscribe to ineedhits and you pay one fee for Inktomi
and another for Teoma.

I am well ranked on Inktomi, but nowhere on Ask/Teoma!

TM

[martinibuster]

Go to the ineedhits website. Top right hand corner. A link for adding your site to askjeeves/teoma. Directly beneath it: A link for adding to Ink. Different databases. Different links. Otherwise it would be one link for all three.

Go to the links I indicated above. You're incorrect.

[jeremy goodrich]

Mr. esquire -> you might want to reread whatever you were on when you got into 'they're the same db...'.

And - please do STOP trying to spread mis-inforamation - it makes it that much hard for people that know SEO to go and help people.

BTW, since no body else touched it -> altavista's spider is 'scooter' not 'infospider'...as you mentioned.

Cheers.