pageoneresults - 2:31 pm on Sep 21, 2007 (gmt 0)

The strange thing is that Google DID in fact update the index for the non-dynamic pages, the ones that just end in .asp and were changed to .htm.

My guess is that was the simplest change for Google to figure out immediately. Compare the previous version vs the new, no change other than file extension, update immediately.

Why would Google react in a different way when it comes to updating the dynamic-looking pages? Did this trigger some kind of filter?

You changed the URI paths. In theory, the page has a brand new address and therefore needs to go through the process of recalculation.

Once that process has fully propagated, you should see most, if not all pre-rewrite traffic return. Yes, there is a process and time delay based on my experience with this. And, if you changed any of the click paths, the delay seems to be a bit longer as that is a rather major change and will take a bit of time to sort things out. Yes, PR is a determining factor in how long this process takes. On average, I'd say anywhere from 90, 120 to upwards of 180 days. Again, that will all depend on what changes took place and other external factors.

The above is why these days I am recommending a slow release of URI restructuring if it can be done. If your livelihood depends on organic search, DO NOT make any major URI changes in bulk. Test first, test again, and then if you feel comfortable that things are okay, test some more! By the time you are done testing, you will have released each new section slowly over time by default. :)

Of course there is always the chance that things are not 100% correct with the rewrite. Improper server headers can wreak havoc on a sites indexing. In fact, I've seen new sites launch with new URI structures and where there were server header issues, the indexing of those sections was challenging to say the least.

If you release a site and there are technical problems from the first indexing, you will be behind the eight ball from that point forward until you get things in order. Expect 30-60-90 day delays each time you make a major change to correct problems that were discovered after the fact. A Proactive approach to these types of technical challenges is the way to go. You sure don't want to be Reactive. At that point, the problems are already in existence and it takes a while to undo them.

Patience is key in this entire process. It isn't going to happen quickly. Any major changes like this should be done well in advance of any naturally occurring major spikes in traffic to your site such as the holidays! From experience, I'd say if you have plans on a major URI Rewrite, wait until after this holiday season and then do it immediately thereafter. You'll have a good 6-8 months of recovery time before the next holiday. :)