homepage Welcome to WebmasterWorld Guest from 54.167.185.110
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Advertising / Paid Inclusion Engines and Topics
Forum Library, Charter, Moderator: open

Paid Inclusion Engines and Topics Forum

  posting off  
Inktomi check
Am I being indexed?
CubsRule4




msg:29339
 1:33 am on Mar 4, 2003 (gmt 0)

My site is showing up for Inktomi when I go to positiontech.com, but it only shows two listings (one is the home page with an outdated description, and the other is a dead page). We have about 200 pages to our site, so I have a couple of questions for the more experienced out there... Does the fact that our site shows up at all indicate that we are on the radar and don't need to resubmit? Also, is Mozilla the Inktomi spider that is showing in our web logs? If so, at some point should we see a lot more of our pages being listed, or does it generally just show the primary/home page?

Thanks for the help!

 

jdMorgan




msg:29340
 1:46 am on Mar 4, 2003 (gmt 0)
CubsRule4,

Here is a request from Inktomi's spider "Slurp" from my logs:

66.196.65.14 - - [27/Feb/2003:01:54:19 -0500] "GET /robots.txt HTTP/1.0" 200 2017 "-" "Mozilla/5.0 (Slurp/si; slurp@inktomi.com; http://www.inktomi.com/slurp.html)"

If you see this user-agent in your logs, then you are indeed "on the radar." If you wish to submit to PT's paid inclusion plan, that's up to you. The factors that control how deeply Ink will index your site are pretty much the same as those for other search engines: Easy-to-navigate link structure not dependent on JavaScript or any other non-HTML methods, incoming links to your pages from other sites which in turn are well-linked, a listing in one or more Web directories, such as the OPen Directory Project at dmoz.org, etc.

I have several sites which are fully-indexed, and have been since before I was aware of Ink's pay-for-inclusion.

HTH,
Jim

CubsRule4




msg:29341
 6:49 pm on Mar 4, 2003 (gmt 0)

Thanks for the info. How do you check to know that your sites are fully indexed? Do you go to PositionTech, and if so, does it list every page when you type in your domain name?

martekbiz




msg:29342
 1:39 pm on Mar 28, 2003 (gmt 0)

Hi,

I have the same issue here in that positiontech will list all the page indexed for particular domains yet while I check in MSN using the same domain:www.url method I typically only have 2 results showing.

If positiontech is showing several pages indexed using the domain:www.url method then why wouldn't the same method produce the same results in MSN instead of just showing two pages?

If someone can answer that I'd be a happy camper.

All these sites are spidered frequently by Ink.

Thanks.

Aaron

Jim_Stob




msg:29343
 3:19 pm on Mar 28, 2003 (gmt 0)

When you use the domain prefix at positiontech we override our default two domain limit and display all URLs from the domain. This has been specifically programmed into our search interface and is most likely not being used elsewhere.

Alternative Future




msg:29344
 3:30 pm on Mar 28, 2003 (gmt 0)

I wish someone from PT would reply back to my continual emails regarding my domain, quick enough to take my money and index the page. But as soon as there is any sign of a problem incommunicado is the best description I have for their services... :(

KR,

-gs

martekbiz




msg:29345
 3:47 pm on Mar 28, 2003 (gmt 0)

>>When you use the domain prefix at positiontech we override our default two domain limit and display all URLs from the domain. This has been specifically programmed into our search interface and is most likely not being used elsewhere.

Alright.. so then how do we effectively search for ALL our pages indexed in MSN then?

I would think a lot of people are paying for inclusion based on the fact they only see 2 pages indexed when they go hunting on their domain name yet the results have been limited to ONLY show 2 occurances matching the domain name in question.

Seems liek trickery to me, no?

Aaron

Alternative Future




msg:29346
 7:23 pm on Mar 29, 2003 (gmt 0)

I wish someone from PT would reply back to my continual emails regarding my domain, quick enough to take my money and index the page. But as soon as there is any sign of a problem incommunicado is the best description I have for their services... :(

I apologise for the above mail directing my problem at PT, when in fact I have been mailing the wrong company :(

MOD could you please remove my posts further up regarding this and also accept my apologies.

Yours apologetically,

-gs

skibum




msg:29347
 7:57 pm on Mar 29, 2003 (gmt 0)

Use this:

[search.msn.com...]

enter your domain name w/o the .com or whatever in the search box. Enter your domain name - my company.com in the domain box. As fas as I can tell that will show you all pages listed in Ink.

martekbiz




msg:29348
 8:36 pm on Mar 29, 2003 (gmt 0)

>>Use this:
[search.msn.com...]

enter your domain name w/o the .com or whatever in the search box. Enter your domain name - my company.com in the domain box. As fas as I can tell that will show you all pages listed in Ink.
<<

Works great! thanks a ton!

Aaron

skibum




msg:29349
 1:51 am on Mar 31, 2003 (gmt 0)

glad it helps!

GeorgeGG




msg:29350
 5:05 am on Mar 31, 2003 (gmt 0)

jdMorgan, from your logs:
Slurp/si

all I been seeing is
Slurp/cat

Have any idea what the difference might be?

GeorgeGG

McMohan




msg:29351
 5:35 am on Mar 31, 2003 (gmt 0)

Interesting Skybum!

Now we have two ways to check pages indexed in Ink -

1. [search.msn.com...]
2. domain:www.mycompany.com in Positiontech.

What can one derive if the results show significant differance in no. of pages indexed? When I tried out in MSN adv. search more than 1000 pages of my site are indexed and in PositionTech pure search, only 70 pages are indexed. Infact for all the Keywords that I am ranked anywhere within 20 in MSN and PT, all are from these 70 pages. Though I am shown large no.of pages indexed in MSN Advanced Search, for rest of the pages (apart from those 70) I am not ranked at all even when I search for a complete sentence from my pages. Infact in MSN those pages are ranked at the end! In PT no trace of them?

A basic question? Who else has supplied those pages to MSN if not Inktomi? Definitely not Overture, since Overture too has only those pages as in PT.

mayor




msg:29352
 3:28 pm on Mar 31, 2003 (gmt 0)

GeorgeGG, here's what I see as the difference:

Slurp/si checks the robots.txt file
Slurp/cat comes along later and gets the site's pages

GeorgeGG




msg:29353
 11:20 pm on Mar 31, 2003 (gmt 0)

Slurp/si checks the robots.txt file
Slurp/cat comes along later and gets the site's pages

Ooook, I only have a personal site, so guess /si
is getting the server's robot.txt.

Thanks
GeorgeGG

skibum




msg:29354
 7:01 am on Apr 1, 2003 (gmt 0)

Dunno the answer to that one, McMohan.

Maybe Jim_Stob can shed some light on it?

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Advertising / Paid Inclusion Engines and Topics
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved