|Will 301 Redirect Fix Supplemental Pages?|
or will new page be supplemented too?
Currently over 50,000 pages of our site (all from one folder) have just been made supplemental by Google. This is because Google (despite robot.txt telling them not too!) infiltrated our testing folder which has the exact same information as the one we leave open for indexing. i.e.
obviously we use that folder to tweak and test any changes to pages before putting them live. But Google is listing both sets of pages, only thing we can think of is that someone went to one of the test page whilst still having the Google toolbar on.
Anyway, the only solution we've come up with is to move the pages to a new folder e.g.
and then 301 both:
to the new page. Obviously Google will find the new pages through the internal linking so should see them all as new pages. The 301 would make the indexing faster and stop 50,000 404 header returns which could really cause some problems with Google.
Anyone know if you 301 from a supplemented page to a new page if that page is automatically supplemented or not?
Maybe I misunderstand -- but why not use robots.txt to excude /foldertest?
To address the original question, URIs do not necessarily stay supplemental, and there is a googlebot that spiders suppemental urls. That bot is spidering more frequently, and I recently (this past weekend) saw over 1,000 suppemental listings return to regular status for one domain, so "supplemental" is not necessariy a graveyard. There's no reason to assume a 301 would make the new content inherit supplemental status.
But as I said, why not just exclude /foldertest through robots.txt and be done with it?
|Maybe I misunderstand -- but why not use robots.txt to excude /foldertest? |
No, just mis-read, I already said:
|(despite robot.txt telling them not too!) infiltrated our testing folder |
... and yes, my robots.txt file is valid. Last time they did this they did it to my admin area which wasn't very well written and the Googlebot followed all the "delete" links .... that was a very, very bad day ...
Robots.txt doesn't keep Google out of a folder if there are links pointing to those urls. Use NOINDEX meta tag instead or password protect it with .htaccess. If you must link to those urls, flag those A HREFS with REL=NOFOLLOW.
As for the 301s, I woud not create a new folder to 301 the original. Just 301 the test pages to the originals, create a new test folder and conduct your experiments there.
[edited by: Halfdeck at 3:27 pm (utc) on Aug. 8, 2006]
|Just 301 the test pages to the originals, create a new test folder and conduct your experiments there. |
Really? Even though both the test pages and originals are both supplemented right now you would say to 301 the supplemented test pages to the supplemented original pages and hope for the best? Why is that? And how long do you think it takes to get unsupplemented?
Look at the answer steveb posted:
robots.txt has zero effect on supps
I'm curious to know how accurate steveb's answer is to 301's and supplementals, i.e. Google not obeying 301 on those pages.
If this is true, it goes a long way to explaining why we can't get our supplemental pages back into the main index.
|Even though both the test pages and originals are both supplemented right now |
If both are supplemental, then you may have other problems besides duplicate content (i.e. similar description snippets, thin pages, etc).
Like steveb said, I haven't seen 301s on my supplmenetals followed either, but one can only hope. Anyway, I would figure out why the original went supplemental before creating a new folder, else those new pages may end up supplemental as well.
|Anyway, I would figure out why the original went supplemental before creating a new folder, else those new pages may end up supplemental as well. |
You mean you don't think that they went supplemental because there was an exact mirror in the test folder?
What other causes are there?