indyank - 3:04 pm on Mar 16, 2011 (gmt 0)
aakk9999, The URLs on the todo list will drop off, if there are no internal or external links to them.The dynamic urls that accidentally got indexed is a good example.
For those pages that return 404 or 410, I don't see google crawling them as they don't exist anymore.So, they too wouldn't remain on the to-do list unless there are links pointing to them.
But noindex pages are a different case.Google need to crawl them to see the noindex meta tag.
You can instead block them via robots.txt. But the real issue in blocking them via robots.txt is whatever incoming links that those pages had accumulated would be lost.At-least that is my understanding.Not sure what others feel on this.Would love to know.
So I do prefer the noindex, follow meta tag as whatever juice that these pages accumulate can still be spread to urls followed on those pages.You can at the same time work on improving them and then eventually release them into google's index.