homepage Welcome to WebmasterWorld Guest from 54.211.68.132
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Visit PubCon.com
Home / Forums Index / Google / Google SEO News and Discussion
Forum Library, Charter, Moderators: Robert Charlton & aakk9999 & brotherhood of lan & goodroi

Google SEO News and Discussion Forum

    
Google indexes under 5% of our pages
hellboy




msg:3797487
 2:27 pm on Dec 1, 2008 (gmt 0)

HI there,

I'm pretty sure that this topic was discussed here many times, but anyway I couldn't find the right answer.

Problem:

We have a products catalogue which has about 450 000 products (pages) and still groving, but what happend is that we struggle with indexing pages into google, it's been about 6 months we have only about 20 000 pages in google index and this is not grooving at all. The number has just stopped on 20 000. It's very flustering for us.

We have all pages in google sitemaps (webmaster tool ) and all this stuff, very little pages with duplicated meta descriptions, but we do not think this may have such a big impact on the indexing, or am I wrong?

Thank you!

 

tedster




msg:3797673
 7:11 pm on Dec 1, 2008 (gmt 0)

Google does have automated rules about which urls they will keep in the index. They apparently look at PageRank as one of the factors and low PR for a url often means a low chance that they will keep that url in the index. Otherwise their index could easily be flooded with what some have called "database spam". So PR and backlinks are critical.

In addition to the help that unique titles and meta descriptions give you, there needs to be enough information available in the content, so that the page does not look like a nearly empty "stub page" - a page that's almost all template.

Another factor is making sure you have minimized canonical problems [webmasterworld.com]. Those can flood Google with confusing duplicate urls and even cause further indexing to pretty much stop.

bumpaw




msg:3798052
 6:17 am on Dec 2, 2008 (gmt 0)

The number has just stopped on 20 000.
Is it certain that Google shows all the pages in it's index when you search for number indexed?
tedster




msg:3798056
 6:26 am on Dec 2, 2008 (gmt 0)

Good question. In fact, it's frequently reported that the number of results you see for a site: operator query is inaccurate. As an example, if you take a site with a clear directory structure and do a set of searches:

site:example.com/directoryA/
site:example.com/directoryB/
site:example.com/directoryC/

...it is common to see urls show up in those searches that were NOT reported in a site:example.com search.

However, those discrepancies are not likely to be enough to change the 5% figure for this site into something worth smiling about.

hellboy




msg:3798101
 8:57 am on Dec 2, 2008 (gmt 0)

bumpaw & tedster thank's for your replays:

I just type site:www.example.com and it shows only about 20 000 pages from 450 000 we already have created. (6 monhts ago)

As a example I was checking another catalogue like we have, and they have rised number of pages in google index from 300 000 to 450 000 in about 2 weeks!

I know that the site: operator might be inaccurate, but having 450 000 pages and google shows only 5% of them this must be something wrong going on. I would agree if we have about 25 000 pages and google shows 20 000. But this is a huge disproportion.

Robert Charlton




msg:3798344
 5:24 pm on Dec 2, 2008 (gmt 0)

The problem could be due to PageRank distribution in relation to the distribution of your inbound links. The nav structure, eg, might be too wide or too deep.

The quality of the inbound links is also something you need to examine. Are you relying on sitewide links from other sites? If they're paid links, is there a possibility that Google is not crediting them?

Very roughly, what's your site structure like? How good are your inbound links?

wingslevel




msg:3798460
 7:46 pm on Dec 2, 2008 (gmt 0)

Last time I checked, less than 1% of amazon's pages were in the primary index

JS_Harris




msg:3798828
 4:49 am on Dec 3, 2008 (gmt 0)

Duplicate content issues.

If I look at your site will I find several paragraphs of 100% unique content for each of the 450,000 pages? Is all of that unique content filled with terms that people actually search for?

Don't you dare say yes, I can almost guarantee you that your product descriptions closely resemble other sites promoting the same products.

A blue widget is a blue widget is a blue widget to Google. You need to "offer more" to make Google happy with your site.

Word of warning - when you make some inroads into doing that you will draw attention from the SEO's backing the sites currently in front of you and they will react accordingly, be ready for the long battle inherent with most e-commerce sites.

hellboy




msg:3799081
 1:06 pm on Dec 3, 2008 (gmt 0)

HI All, thanks for your replay first!

Robert:
The structure of the page is very simple:
Category->Sub Category( max 4 levels) - > Product
But at the end the product URL is
www.example.com/product-123-product-name/

On each product page is navigation to the categories so the pages should be well linked

JS_Harris:

Right the content is not on 100% unique there some products descriptions which are the duplicated with other shops products about 250 chars.

Well that might be truth, there are products from other shops that have the same content, but on the product page are also related products, so it's not only content from one product

I can admit that there are some duplicated titles, meta descriptions according to webmaster tools reports, but there are very few of them about 0.1%

But, what is strange to me is that the others similar catalog pages doesn't seem to have these issues as we have, and they have the same product pages from others shops, navigation etc...

What about the XML feeds from many other pages (articles, news ) which you are using on your page, is this considered as a duplicated content as well?

I would understand if I had exactly the same copy on my page, but being honest this is just short product description and like I said, there is more other content on each product pages, which shouldn't be unig with any other pages on the inernet

incrediblehelp




msg:3799088
 1:21 pm on Dec 3, 2008 (gmt 0)

1. Get more links to internal pages. You cant just rely on internal navigation to help your massive site get indexed.

2. Yes PR sculpting and improved internal navigation can help.

Robert Charlton




msg:3801773
 11:16 pm on Dec 6, 2008 (gmt 0)

The structure of the page is very simple:
Category->Sub Category( max 4 levels) - > Product
But at the end the product URL is
www.example.com/product-123-product-name/

On each product page is navigation to the categories so the pages should be well linked

Taking the last part first, the "navigation to the categories" links probably wouldn't be helping the product pages to rank. Oversimplifying a bit... keep in mind that it's inbound links that help you rank, but also that you can't pull yourself up simply by recirculating PageRank from a page back to itself.

I'm a big fan of categories and subcategories, and four levels may well be the only good way to handle a very large number of pages on some sites. That said, interior pages have appeared to drop down faster in Google than they used to be, and four levels can be very deep if you have insufficient PageRank at the top. You need inbound links coming in to category and subcategory pages, and also to product pages... or perhaps have fewer pages until you build up your inbound PR.

Re duplicate content:
Well that might be truth, there are products from other shops that have the same content, but on the product page are also related products, so it's not only content from one product

Again, good thinking to a point, but it's likely that the related product content is also not unique... so the question is whether rearranging duplicate content on a page is enough for Google to see the page as unique.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Google / Google SEO News and Discussion
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved