deeper than home page

[RobBroekhuis]

Reading through the forums here, I find many references to websites that have been up for a couple weeks and have already been deep-crawled. So far, Google just comes by my website (up for several months) regularly, grabs robots.txt and the home page, and leaves. What am I doing wrong? My robots.txt doesn't disallow any of my regular pages, and the home page has
<META NAME="robots" CONTENT="INDEX,FOLLOW">
<META NAME="GOOGLEBOT" CONTENT="index,follow">
The website is still PR0, even though I have a good number of inbound links (not from high-PR pages though). I've no problem exerting patience, but I'm afraid something else about my site is causing this. Is this typical bot behavior?

[rogerd]

Are other bots spidering your content, and is Google the exception? If so, the situaion you describe may not be a problem at all - it could be that only time and better linkage are needed.

If your site isn't getting spidered by anyone, I'd check your robots.txt file for possible syntax errors. The robots meta tags shouldn't be necessary to encourage spidering; I never include them except to exclude pages (noindex).

A couple of other things you can do are run a spider simulator like the one at searchengineworld.com to make sure your server headers are OK, and try spidering the site yourself using Xenu or similar. If all this checks out fine, then keep working on linkage.

[bufferzone]

It’s probably a linking problem. A good way to overcome this. is to create a sitemap with standard href links to all the rest of the pages of the site. Link form your index page (again with a standard href link) to the site map. This way, the crawler will have good, easy to handle links that leads out to your entire site. It should result in a deep crawl

[bufferzone]

check out this post

[webmasterworld.com...]

[RobBroekhuis]

Thanks for pointing me in the direction of Xenu - I wasn't aware of that tool, and it will be helpful. However, using it did confirm that my site links together nicely. Every page has a navigation bar along the top and bottom, linking to main category pages.
Googlebot comes in most consistently. I've had some very strange visits by AskJeeves/Teoma (they didn't get much more than my ErrorDocument404 page). Wisenutbot and IAarchiver don't venture below homepage either.