Inktomi check

Am I being indexed?

[CubsRule4]

My site is showing up for Inktomi when I go to positiontech.com, but it only shows two listings (one is the home page with an outdated description, and the other is a dead page). We have about 200 pages to our site, so I have a couple of questions for the more experienced out there... Does the fact that our site shows up at all indicate that we are on the radar and don't need to resubmit? Also, is Mozilla the Inktomi spider that is showing in our web logs? If so, at some point should we see a lot more of our pages being listed, or does it generally just show the primary/home page?

Thanks for the help!

[jdMorgan]

CubsRule4,

Here is a request from Inktomi's spider "Slurp" from my logs:

66.196.65.14 - - [27/Feb/2003:01:54:19 -0500] "GET /robots.txt HTTP/1.0" 200 2017 "-" "Mozilla/5.0 (Slurp/si; [e-mail]; http://www.inktomi.com/slurp.html)"

If you see this user-agent in your logs, then you are indeed "on the radar." If you wish to submit to PT's paid inclusion plan, that's up to you. The factors that control how deeply Ink will index your site are pretty much the same as those for other search engines: Easy-to-navigate link structure not dependent on JavaScript or any other non-HTML methods, incoming links to your pages from other sites which in turn are well-linked, a listing in one or more Web directories, such as the OPen Directory Project at dmoz.org, etc.

I have several sites which are fully-indexed, and have been since before I was aware of Ink's pay-for-inclusion.

HTH,
Jim

[CubsRule4]

Thanks for the info. How do you check to know that your sites are fully indexed? Do you go to PositionTech, and if so, does it list every page when you type in your domain name?

[martekbiz]

Hi,

I have the same issue here in that positiontech will list all the page indexed for particular domains yet while I check in MSN using the same domain:www.url method I typically only have 2 results showing.

If positiontech is showing several pages indexed using the domain:www.url method then why wouldn't the same method produce the same results in MSN instead of just showing two pages?

If someone can answer that I'd be a happy camper.

All these sites are spidered frequently by Ink.

Thanks.

Aaron

[Jim_Stob]

When you use the domain prefix at positiontech we override our default two domain limit and display all URLs from the domain. This has been specifically programmed into our search interface and is most likely not being used elsewhere.

[Alternative Future]

I wish someone from PT would reply back to my continual emails regarding my domain, quick enough to take my money and index the page. But as soon as there is any sign of a problem incommunicado is the best description I have for their services... :(

KR,

-gs

[martekbiz]

>>When you use the domain prefix at positiontech we override our default two domain limit and display all URLs from the domain. This has been specifically programmed into our search interface and is most likely not being used elsewhere.

Alright.. so then how do we effectively search for ALL our pages indexed in MSN then?

I would think a lot of people are paying for inclusion based on the fact they only see 2 pages indexed when they go hunting on their domain name yet the results have been limited to ONLY show 2 occurances matching the domain name in question.

Seems liek trickery to me, no?

Aaron

[Alternative Future]

I wish someone from PT would reply back to my continual emails regarding my domain, quick enough to take my money and index the page. But as soon as there is any sign of a problem incommunicado is the best description I have for their services... :(

I apologise for the above mail directing my problem at PT, when in fact I have been mailing the wrong company :(

MOD could you please remove my posts further up regarding this and also accept my apologies.

Yours apologetically,

-gs

[skibum]

Use this:

[search.msn.com...]

enter your domain name w/o the .com or whatever in the search box. Enter your domain name - my company.com in the domain box. As fas as I can tell that will show you all pages listed in Ink.

[martekbiz]

>>Use this:
[search.msn.com...]

enter your domain name w/o the .com or whatever in the search box. Enter your domain name - my company.com in the domain box. As fas as I can tell that will show you all pages listed in Ink.
<<

Works great! thanks a ton!

Aaron

[skibum]

glad it helps!

[GeorgeGG]

jdMorgan, from your logs:

Slurp/si

all I been seeing is

Slurp/cat

Have any idea what the difference might be?

GeorgeGG

[McMohan]

Interesting Skybum!

Now we have two ways to check pages indexed in Ink -

1. [search.msn.com...]
2. domain:www.mycompany.com in Positiontech.

What can one derive if the results show significant differance in no. of pages indexed? When I tried out in MSN adv. search more than 1000 pages of my site are indexed and in PositionTech pure search, only 70 pages are indexed. Infact for all the Keywords that I am ranked anywhere within 20 in MSN and PT, all are from these 70 pages. Though I am shown large no.of pages indexed in MSN Advanced Search, for rest of the pages (apart from those 70) I am not ranked at all even when I search for a complete sentence from my pages. Infact in MSN those pages are ranked at the end! In PT no trace of them?

A basic question? Who else has supplied those pages to MSN if not Inktomi? Definitely not Overture, since Overture too has only those pages as in PT.

[mayor]

GeorgeGG, here's what I see as the difference:

Slurp/si checks the robots.txt file
Slurp/cat comes along later and gets the site's pages

[GeorgeGG]

Slurp/si checks the robots.txt file
Slurp/cat comes along later and gets the site's pages

Ooook, I only have a personal site, so guess /si
is getting the server's robot.txt.

Thanks
GeorgeGG

[skibum]

Dunno the answer to that one, McMohan.

Maybe Jim_Stob can shed some light on it?