I set up a new site about three weeks ago, and, while Google picked it up on the second day that it was live, Google still has yet to go through the whole site.
Presently only the front page (an intro page with just text links and not much content) is indexed in Google. But I'd like to know when will Google start indexing the rest of the website? I've made sure that the pages validate as HTML 4.01 Transitional or XHTML 1.0 Transitional (asside from a flash object which I have at the bottom of the code).
How deep is your site. You should thrive to make every page accessible in max 3 clicks with the mouse. You should thrive to have more than one internal link to every page. A page with only one link is not important. If you can, you should get external links to pages inside your site.
Make it easy for the spider/crawler to get to all your pages and create many side doors for the spider/crawler to come in through.
Also your links are important. Spiders like common href links the best, if you use dynamic links created with special characters, you might make problems for the spider.
I launched one in Feb that has only just been crawled deeply. The index page had been grabbed almost immediately, but it's taken 9 weeks for the others to be crawled.
Patience is the key.
<added> Probably pure coincidence, but I had this new site linked from a few of my existing, well ranked, sites - but only to the index page. I then changed those links to point to the site map page. Next day, the deep crawl happened. </added>
Well, the site finally got deep-crawled yesterday. Now comes the waiting for keywords to work. But that's all sceptical stuff, so who knows if my keywords will do any good. I guess now is the time to wait for pagerank to improve since it's still at 0.