Welcome to WebmasterWorld Guest from

Forum Moderators: goodroi

Message Too Old, No Replies


2:32 pm on Aug 27, 2010 (gmt 0)

Junior Member

5+ Year Member

joined:Jan 3, 2008
posts: 42
votes: 0

I understand that pages using the metatag noindex allow for crawling, following links and accrual of page rank but do not get indexed. However, I see my site growing in the index rapidly because of these pages -mostly worthless-. Does anyone know how it works? Googlebot crawls them and then drops them from the index? I was expecting these pages never to be counted in the "About xx results" that come up in a "site" search command. Right now, the headers have the "noindex,follow". Should I switch to "noindex,nofollow" to stop the crawling and prevent the indexing completely?
1:04 am on Aug 28, 2010 (gmt 0)


WebmasterWorld Administrator phranque is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Aug 10, 2004
votes: 8

are you seeing just the noindexed urls in the index or actual content from those pages?
4:55 am on Aug 28, 2010 (gmt 0)

Junior Member

5+ Year Member

joined:Jan 3, 2008
votes: 0

It's probably too early to draw any conclusion. Site is fairly new and I made changes as it got crawled and indexed. The noindex pages that have been indexed do have content showing in cache but taking a closer look I also see that they were crawled before I switched the meta to noindex. That explains it. Planning ahead is essential!

I do not see the pages that have been crawled in the last few days, after I made the change. I imagine the ones indexed will get drop after they are crawled again. Sorry for wasting your time.

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members