vordmeister - 7:08 pm on Jun 2, 2006 (gmt 0)

frederico> looks to me like BD just raised the bar of what gets crawled: high page rank, backlinks, and most original content

Original content doesn't feature in my experience. Hand written HTML pages aren't getting spidered. I've written a few after the start of bigdaddy, and they are simple - original content and the common headers and footers are tiny in the code.

Not the PR either (maybe) these new pages are sitting on the same level as established pages with PR3. Surely that's enough for at least a quick peep?

For backlinks - it's true my new pages don't have external backlinks, but that's arguably because nobody knows the pages are there (not in google etc).

Someone posted in some other thread they thought this was to do with homepage PR (or equivalent) and the depth GG were willing to crawl as a result. That argument makes a lot of sense to me.

It seems there are "factors" that are currently stopping Google from crawling new pages that are more than one or two links from the index page for me.

I'm sure that someone, somewhere on the web will have written something half useful that deep on their site over the last couple of months. I'm sure Google have realised this information could be useful to their visitors and might get around to spidering it eventually.

I'm not going to suggest Google is broken, but I suspect things aren't going quite the way they want and that they might hopefully be doing some adjustments soon.