Great community - best I've ever seen. Snag. Site has PR 7 and yet only 38 pages indexed (out of 19,000 - all dynamic) - most of which are orphans that have come from other external links.
Index page ranks top for one of its keywords an has PR 7 with dmoz entries. I see additional pages come and go - usually with a new date on the right - then gone in 24hrs.
There are two domains abc.com and abc.co.uk pointed to the same IP (history is great eh?...) Is this causing us a problem? One white bar orphan page. The bot hits us regularly but never indexes any links. All content is real with no attempt at manipulation other than following the layout rules.
New at this and stuck. What should I be thinking about? Should we move to all static pages? Are we barred? Why if it finds the links does it not index them If so how do I tell.
How old is your site? If Google has only just found your site, your pages will pop in and out of the index until you've been spidered for a complete month-long Google cycle. This is a normal situation and nothing to worry about.
You should consider linking those orphans back to your home page.
I've seen some very interesting things with google lately and dynamic pages. We have site 'A' which has been around for 3 years and has had 100% of it's 3 and 2 variable dynamic pages indexed up until the may update. It then dropped the 3 and 2 variable pages. But it kept the 1 variable dynamic pages which were original content and the 1 variable pages which were the same as the 3 variable pages that had been indexed (same page, just accessed from a different path through site. And only 20% of our content is available via 1 variable URL's so 80% got dropped).
Last month we brought 2 new sites online using the same software and layout. We have several links from PR8 pages to the new sites and when googlebot visited I thought they'd get a big deep crawl because of the PR8 links. BUT NO... They get the same crawl pattern as Site 'A'. IMO googlebot is much smarter than most think. My guess is that googlebot has some pretty advanced features to keep it from grabing dupe content and will avoid patterns that bring it to the same content even if it finds original content on the same path.
Right now we are building some site maps with just 1 variable links to our content. My guess is googlebot will eat these links up and I'll get most of our content back in. When the next crawl comes around I'll let everyone know how this experiment turns out.
My suggestion is give google your content through 1 path. Don't have diffent possible URL's to the same content or parts of the same content. This might happen through static pages or search results... Hope this helps...
Chris: We had similar activity on some of our sites, only PR6 and PR7 and only something like 30 pages indexed, recorded inbound links dropped to a bare minimum. These sites have been through some big changes though and many pages have moved.
Googlebot crawled over a 10 day period starting late last month until the 7th August and with hits of over 3,000 pages on some sites, we are expecting (and hoping) that things will return to normal.
Essentially one possibility is that this behaviour could be transient, check your linking structure (we had a lot of orphans) and make sure it is neatly laid out & the dynamic content is spiderable...I guess we'll find out and compare notes on the next update!
Thx to all, I will link in the orphans etc. Following on and after thinking about it - does G follow form navigation esp if it uses onchange to fire the new page off?? or is this where I defeat the bot.
Please excuse daft question but can't see a definitive answer.