Msg#: 4698873 posted 10:44 pm on Aug 29, 2014 (gmt 0)
The question I have is relatively straightforward. Would Google penalize a website for potential violations on pages that contain no-index and no-follow meta tags?
A client of mine runs a photograph site and was forced to de-index the overwhelming majority of their site in the wake of Panda 4.0. At this time the only pages indexed are the main page and the category pages. Originally they had a mobile redirect advertisement on the main page. Mobile users would redirected to the App Store or an alternative website. While Google certainly frowns upon redirects of this sort, would it be acceptable on pages these no-index / no-follow pages given that Google should not be crawling / indexing these pages in the first place?
Msg#: 4698873 posted 1:56 pm on Aug 30, 2014 (gmt 0)
If you have disallowed crawling before the bot has crawled the pages since the noindex tag was added, the bot will continue to consider the pages as "index, follow" because they can't see that you have added a noindex tag.
When you change a robots metatag to noindex, you need to wait until it has been crawled, then you can block it in robots.txt.
Msg#: 4698873 posted 7:37 am on Aug 31, 2014 (gmt 0)
@panoramic, i thought sbout it too and in regard of panda, geevil says it is a way to come out of panda by noindex low and shallow pages. I don't know if it realy works for redirects. I would say that this is a more severe violation against their guidelines (that certainly don't account for big brands and geevil itself)
Msg#: 4698873 posted 3:42 pm on Aug 31, 2014 (gmt 0)
I actually manually removed all the pages in question utilizing Google Webmaster Tools.
This removes those pages from the index immediately. You have added the noindex metatag to prevent indexing, good move. But you should not block those pages in robots.txt or they cannot see the noindex metatag. Manually removing pages is temporary and blocking in robots.txt should not be done until you are sure they have crawled those pages with the noindex tag.