Forum Moderators: Robert Charlton & goodroi

Message Too Old, No Replies

How do I get more indexed pages in Google?

         

poweri

12:19 pm on Apr 20, 2009 (gmt 0)

10+ Year Member



Hello,

I have a website with 10.000 pages, but after 3 months I only see 300 indexed pages in Google.
What can I do to tell Google to index ALL 10.000?

Looking forward to an answer!

tedster

1:07 am on Apr 21, 2009 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Three months is a very young site, so part of the answer is patience. Another part is getting more backlinks. And yet another could be an xml sitemap.

Lame_Unit

2:05 am on Apr 21, 2009 (gmt 0)

10+ Year Member



I can second that having a sitemap is very helpful

You can try making one with [xml-sitemaps.com...]

tedster

3:58 am on Apr 21, 2009 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



No reason to play any fovorites here. Google maintains a list of all kinds of xml sitemap generators at [code.google.com...] - including online generators, downloadble tools for various operating systems, server-side programs and CMS plug-ins.

apauto

11:20 pm on Apr 24, 2009 (gmt 0)

10+ Year Member



I never submit sitemaps to Google, and I'll tell you why.

If your site is easy to navigate, there is no need. Google determines the importance and rank of a page by natural crawling. By simply giving it a sitemap, it loses the ability to determine where the link lies in the overall picture of your site.

g1smd

12:40 am on Apr 25, 2009 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Google has a "crawl budget" assigned to your site. That budget will increase over time as they trust it more. Get a few incoming links from other related sites, and triple check that your site is easy to spider, and doesn't have bot-stopping issues. Check it with Xenu LinkSleuth or similar.

tedster

1:35 am on Apr 25, 2009 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



There's a step I often forget to mention, because for many servers these days it is standard.

That step is - make sure you server is configured to respond properly to the "If-modified-since" http request. The server should give a 304 status if there have been no changes to the url, and the server clock should be set accurately. This economizes googlebot's crawl budget for your site and allows more pages to be requested within any given budget.

The idea is that instead of requiring googlebot to fetch another full copy of an unchanged url, the server just replies with a few bytes - "no change".

You can also follow all the disciplines that limit canonical URL problems [webmasterworld.com]. Those steps also offer Google another kind of crawling economy.