Google only indexing 10 pages of 380.

Inktomi/Hotbot shows 380 pages, Google shows 10

[daveo]

I have a 380 page website that has been around a while. I know it has recently been indexed on Google, but it only displays 10 of 380 pages if I do site:url search in Google.

Have done a saturation report in marketleap.com which indicates that Google has only 10 pages indexed, while Inktomi has all 380 pages.

The site is built as .asp, since the design of the site was ever-changing, and I made extensive use of include files for all of the navigation. All of the links are plain <a href> text links, no graphics or JavaScript.

The navigation includes are all parsed when the page is displayed, but I am concerned that using them has stopped Google from drilling down into the rest of the site.

Any recommendations greatfully received.

[Gorilla]

Currently it can take weeks from when new pages first are read by Googlebot until they appear in search result pages. How long it takes, depend on the PageRank of pages linking to the pages you want to have indexed. I recently saw it take around a week before new pages referenced from a PR7 page appeard.

You can get a list of all pages Google has in its index from your site by executing a query on this format: "site:www.mydomain.com".

As long as Googlebot has read your pages, you can expect them to appear, but it may take some time.

Have patience - don't panic :)

[daveo]

Thanks for the reply. However, the site has been around for about 8 months now. The 10 pages that are displayed in a site:www.example.com have been there since about 1 month after the site went live. Since then they have been revisited (changes to the pages have been reflected in the description info in Google), but G just wonty drill down into the rest of the site.

[takagi]

>> The site is built as .asp

Do you use session-id's or parameters that look like them as in:

www.domain.com/forum.asp?id=12345

If so, then it is easy to understand why Google doesn't want to spider all these 380 pages.

[bastek]

I've got similar problem but I don't really uderstand the explanation above.
Do you say that Google stopped following links with parameters in it?

[daveo]

I have no www.example.com?xyz=12345 type tags at all. All of the menu items are standard <a href="example.asp">abcdefg</a> text links. However, they are included in functions called from other include file (structure described below).

Page Format Top Include
Menu Include

<%
function getMenus() {
%>

<a href="/services/example.asp">Example</a>
<%
}
%>

End Menu Include
End Page Format Top Include
Page Content
Page Format Bottom Include
End Page Format Bottom Include

[nuevojefe]

What do the 380 URLs look like in the other SE's that have indexed them?

What is the PR of the pages linking to these 370 or whatever amount of pages?

[daveo]

It looks like the 370 or so pages not displayed have a PR < 4, so maybe that's why they are not listed...... Although some of the pages that are listed also have a PR < 4.

[takagi]

I've got similar problem but I don't really uderstand the explanation above.
Do you say that Google stopped following links with parameters in it?

One or two parameters is no problem, but avoid parameter names like 'id' or 'SESSIONID'.

As for daveo's original question, you could try adding one or more site maps if you think the function causes this problem.

[rogerd]

Daveo, it seems like your links are spiderable, but I'd be sure to include a site map and, perhaps, some straight HTML links (e.g., at the bottom of the page).

I'd also go through the usual routine of spidering the site yourself (using Xenu or similar) and checking some pages with tools like SimSpider and Server Header Checker). Chances are all is OK, but it's worth checking before you launch some other major effort.

If everything looks good (and you don't have query strings or SessionIDs on your asp pages), working on linkage to key internal pages (or boosting overall site linkage) may be your best bet. Just because a page is below PR4 doesn't mean it won't get indexed, but higher PR is likely to result in more frequent and aggressive spidering.

[onlineleben]

>>but I'd be sure to include a site map<<
... and maybe even link from an external site (that is well placed in G) to the sitemap. Helped me to gain some authority for a site that wasn't spidered well.

[DaveN]

also daveo try and get some deeplinks and rewrite the urls if the server allows

DaveN

[Leosghost]

Hope this won't be "off topiced" ...because I'd really love it if someone can tell me how to find out "pagerank" in google on my or anyones pages without installing the "googlebar" ....
I did install it breifly ..and watching it's data transfer when it was in use I took it right back out as "scumware"....( fits all the parameters )..same for Alexa ...

[daveo]

Many thanks for everybody's help. I have been overwhelmed by the amount of support. I have got lots to be going on with. I'll let you know how I get on.

Thanks again,

Dave

[takagi]

Hello Leosghost, Welcome to WebmasterWorld,

The only official way to read the PR is using the toolbar with the 'Advanced features' on. Having a discussion about the Toolbar being scumware or not is too much of topic for this thread. If you want to have such a topic, please start a new thread in the Google Toolbar forum [webmasterworld.com].

There are some ways to circumvent the toolbar, but it is against the Google's TOS and after some time Google could block PageRank information for your IP address or send bogus info.