joined:Aug 9, 2009
Update to previous post: [webmasterworld.com
One of my websites has a section that lists websites that are similar to other websites. In early 2009 I changed this section to have mod_rewrite rewritten URLs. The URLs were like www.example.com/view-sites/example.com. Previously they were www.example.com/view-sites.php?id=1000. All URLs were correctly 301 redirected. I updated my XML sitemap code, etc. Everything went fine for 2-3 months, then suddenly ALL of the rewritten URLs disappeared from G virtually on the same day. There were some (5%) non-rewritten URLs (that had slashes, etc, in them) that remained in G. Once I realized it was only the rewritten URLs that had disappeared I rolled the site back in panic, submitted a reinclusion request to G (just to be safe), and the website listing pages started to show back up with the querystring URLs version. It took MONTHS for the pages to show back up, I kept track of them in G sitemaps every day.
In February of 2010 I decided to slowly switch, again, to the rewritten URLs and hope for the best. I changed the code to only rewrite URLs that had been added to the database in 2010+. So it would show rewritten URLs for all websites added to the database then on, and the ones that had been added already in 2010, but keep the querystring URLs for anything prior to 2010. So far G has not index a SINGLE, rewritten, newly added website, and has kept the querystring version for websites between Jan 1 2010 and the day that I made the code change. So page links that it already knew about, that now have 301 redirects to rewritten URLs, have not been updated with their new location, in G's index.
So essentially, G will not index rewritten URLs for this ONE website that I own
. I have 2 other websites that rewrite URLs in the same fashion (except the format is www.example.com/page/variable/ - with the directory style slash) that G indexes just fine.