Welcome to WebmasterWorld Guest from

Forum Moderators: goodroi

Message Too Old, No Replies




2:32 pm on Aug 27, 2010 (gmt 0)

5+ Year Member

I understand that pages using the metatag noindex allow for crawling, following links and accrual of page rank but do not get indexed. However, I see my site growing in the index rapidly because of these pages -mostly worthless-. Does anyone know how it works? Googlebot crawls them and then drops them from the index? I was expecting these pages never to be counted in the "About xx results" that come up in a "site" search command. Right now, the headers have the "noindex,follow". Should I switch to "noindex,nofollow" to stop the crawling and prevent the indexing completely?


1:04 am on Aug 28, 2010 (gmt 0)

WebmasterWorld Administrator phranque is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

are you seeing just the noindexed urls in the index or actual content from those pages?


4:55 am on Aug 28, 2010 (gmt 0)

5+ Year Member

It's probably too early to draw any conclusion. Site is fairly new and I made changes as it got crawled and indexed. The noindex pages that have been indexed do have content showing in cache but taking a closer look I also see that they were crawled before I switched the meta to noindex. That explains it. Planning ahead is essential!

I do not see the pages that have been crawled in the last few days, after I made the change. I imagine the ones indexed will get drop after they are crawled again. Sorry for wasting your time.

Featured Threads

Hot Threads This Week

Hot Threads This Month