MelissaLB - 6:15 pm on Nov 15, 2012 (gmt 0)
phranque, thanks for weighing in over here :)
--do you have a good handle on any errors that are being generated by the server?
we are on a dedicated server and we have no server errors being reported in GWT ( and have only had a few due to a few minutes of outage at one point earlier in the year)
--are you internally linking to any URLs that return a non-200 status code?
good question. everything on our site is automated in a way that when we remove a product (for instance) any links to that product, such as in related products from other pages, are immediately removed. So our site is basically set up for that to be an impossibility. However, since we have implemented the 404's, I can now check in WMT to see what pages are linking to the 404 page, there are often a few external pages but there are also a lot of internal pages. I am currently on the assumption that those links don't actually exist as many of the 'linked from' pages are pages that are now 404 themselves.
--do you have any idea what percentage of your URLs have been recrawled since you made the changes?
Not exactly, i don't have easy access to my server logs unfortunately, so i pretty much gather all of my information through G Analytics and WMT. However, we have about 6690 pages indexed and the last time we were crawled heavily, On Nov 4th before the last Panda update, GWT shows that it did crawl abotu 7000 pages that day to an average of about 2000-3000 per day.
lewis1 >> 3 weeks sounds about the same time frame that tedster mentioned. You couldnt tell me how quickly new pages on your site are indexed when you add them? just wondering if there is any correlation between that and indexing a subdomain with a 301. As generally our new pages are indexed within 3 hours -24 hours at the longest.