Welcome to WebmasterWorld Guest from 18.204.48.40

Forum Moderators: Robert Charlton & goodroi

Message Too Old, No Replies

Google ignoring NOINDEX tag?

     
6:49 am on Sep 5, 2017 (gmt 0)

Full Member from GB 

5+ Year Member Top Contributors Of The Month

joined:Mar 26, 2013
posts:270
votes: 36


We have a client that just had a complete site redesign. Unfortunately, the guys who developed the site don't really understand the issues that simple mistakes can cause when it comes to indexing a new site with a new url structure. One of the issues caused is by the fact that the whole staging site was allowed to be crawled. Not sure how but it resulted in over 500 duplicate staging. pages being indexed. (for clarity those are NOT on the same domain as the live site) We have, last month requested a NOINDEX tag to be placed on those pages which they have don. However, this morning it seems that there are over 600 pages indexed with staging.

Now, I understand that it can go up a little before it starts to decline but the strange things is that the cached pages of the staging. domain actually shows the correct/real url. Many with very recent dates well after we put the NOINDEX tag. I found a few that were recrawled yesterday but still showing up in the index for a site:staging.stagingdomain.com command (but with the correct url when looking at cached version).

Google just being Google or is there anything else that I may have missed?

p.s checked robots.txt on the staging domain and it allows robots through.

Any suggestions?
9:05 am on Sept 5, 2017 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:12913
votes: 893


Doesn't sound like Google is doing anything wrong. The pages were not NOINDEXED when they were crawled. Now that they are, it may take a week or two for them to be removed from the index.

Crawl and index time spans can vary sometimes.The conflicting indexing happens with things like this. It will work itself out.

But this should be fundamental to anyone who works with web development. I hope the responsible parties have benn made accountable.
11:31 am on Sept 5, 2017 (gmt 0)

Full Member from GB 

5+ Year Member Top Contributors Of The Month

joined:Mar 26, 2013
posts:270
votes: 36


Thanks Keyplyr. In my experience, it has always been dropped on re-crawl but I'm also experienced enough to know that there are not many constants with Google so will give it more time.

With regards to accountability, I am increasingly finding that the lack of basic SEO knowledge within some of the large design companies is quite scary. You wont believe the drop in revenue this company is experiencing if I told you. All purely due to no understanding of SEO fundamentals (or a lack of care).