Google Indexing Large Sites Too Slowly

[santip]

Hi all, I am new here.
I have site with more than 40.000 page, google index only 6980 pages only, every day I check google and found google add exactly 40 page each day, so it will takes more than three years to get all pages in the site indexed, how to make the google index my site faster?, I hear new google feature, google sitemap, is it help?

Regards,

[goodroi]

Get more links, then get some more links and after that get even more links.

Sitemaps is still in the experimentation stage. The final results are unknown for the sitemap program.

[discrete298]

"I have site with more than 40.000 page"

Spam?

[santip]

not spam, product and its variation, but thanks for your comment and suggestion

[BigDave]

Get deep links to some of the individual product pages, not just your home page.

[shri]

Santip, this is one of the places where good page rank and fresh pages help a lot.

First -- do you pass parameters in the URL. If so, drop them and move to a mod_rewrite.

Second -- Do you have a high enough visible toolbar? 5-7?

Third -- Do you have any pages in the daily crawl? If yes, start rotating deep links to uncrawled pages from them. If you don't have pages in the daily crawl, try to keep your front page as fresh as possible and aim for a PR4-5.

Fourth -- Do you have a good on-site nav structure? Good breadcrumbs? A good site map?

Have you memorised the crawling pattern? There are a few things to be learnt from how the bot hits your site. The frequency of homepage visits and the pages collected later...

[santip]

BigDave,
do you mean cross linking on product page?
the current site structure was: index > category > product, no cross linking between product page.
Shri,
1. no pass parameter in url, I read in google guidelines, that's why the site has so many pages
2. pr was 4, expected to be 5 in next update, when?
3. yes, daily crawl from google added exactly 40 pages each day
4. maybe that is the answer, we had to work on site navigation, do you had any sample site with good site sturcture?

[walkman]

>> product, no cross linking between product page

you should do a "Random Page" thing and each page links to 2-5 of your other pages. Not spammy at all and everything is guaranteed to be eventually seen by Google.

[shri]

>> do you had any sample site with good site sturcture?

This site.

Also, take a look at theme pyramid documents (google them)

[shor]

I agree with Shri.

Adding deep(er) links to daily crawl pages will definitely help. Rotating deep links in our sitemap/directory has been a surefire way of helping G find our deep pages. Naturally, having a site structure that is Googlebot-friendly is also important.

We recently streamlined the site structure of one of our portals and noticed significantly higher daily crawls. As a result, site indexing almost quadrupled from 120k -> 450k pages.

Also remember that being crawled doesn't necessitate inclusion in the index.