Msg#: 4193073 posted 2:32 pm on Aug 27, 2010 (gmt 0)
I understand that pages using the metatag noindex allow for crawling, following links and accrual of page rank but do not get indexed. However, I see my site growing in the index rapidly because of these pages -mostly worthless-. Does anyone know how it works? Googlebot crawls them and then drops them from the index? I was expecting these pages never to be counted in the "About xx results" that come up in a "site" search command. Right now, the headers have the "noindex,follow". Should I switch to "noindex,nofollow" to stop the crawling and prevent the indexing completely?
Msg#: 4193073 posted 4:55 am on Aug 28, 2010 (gmt 0)
It's probably too early to draw any conclusion. Site is fairly new and I made changes as it got crawled and indexed. The noindex pages that have been indexed do have content showing in cache but taking a closer look I also see that they were crawled before I switched the meta to noindex. That explains it. Planning ahead is essential!
I do not see the pages that have been crawled in the last few days, after I made the change. I imagine the ones indexed will get drop after they are crawled again. Sorry for wasting your time.