Encouraging Googlebot to Deep Crawl

How can get it to take all my pages?

[Smiley]

I have a site with 3000+ static .html pages, most of these pages are accessible in two clicks from the home page, however Googlebot is only taking about 700 of them. What I can’t work out is why googlebot takes some of these deep content pages but not others.

Is there a way to get googlebot to visit the rest of the site? Would a site map with links to all these 3000+ pages be of benefit?

Thanks for your help!
Smiley

[jackofalltrades]

A sitemap would be of benefit to your users!;)

It would be another way helping the Googlebot along as well!

JOAT :)

[Smiley]

But would a site map with 3000+ links really be of use to a user?

[jackofalltrades]

It would have to be split over multiple pages, but yes i think i would be.

If you split it down into directory style categories, then a user should easily be able to navigate their way through and find what they want.

JOAT

[Duke_of_Url]

smiley

I've read the ggl isn't too hot at following links buried within trick javascript dropdown menus (dont use 'em myself however so happy to be corrected), so stick to normal text links between pages is fave. Other than that its a case of sit tight and ggl usually gets all pages picked up over a series of updates, rarely all in one go if there are lots of new pages added in.

DoU

[Smiley]

Thanks all for your help.

All links are text. Will add a site map and sit tight and wait!
Smiley

[Grumpus]

Get your PR up sitewide. With my front page at PR4, it took about 6,000 pages. At PR5, it took about 17,000 pages. At PR5 (with a couple of other second level pages reaching PR5, too) it's about 27,000 pages. At PR5 with two internal pages reaching PR6 - yep, internal pages ranked higher than front page due to DMOZ's disability to bother getting around to adding my front page to their directory, but they've added a few deep pages...) it's about 40,000 pages in the index.

The more PR you have, the deeper the bot will crawl.

G.