Forum Moderators: Robert Charlton & goodroi
Way back in April we have done a url rewrite for our entire site. What we have not done was a 301 redirect from our original url's to our new url's.
For that reason Google has dropped both url's (old and rewritten) in supplemental results. The way our site is built, we can not do a 301 redirect. What we are trying to do at this time is put our old url's into robots.txt.
Being that our rewritten url's are in supplemental results, Google doesn't cache our pages, because the bots hardly ever come back to the supplemental pages.
How do we get Google to start caching our supplemental pages and get us out of this big mess?
Is thier any way I can e-mail them, notify them of what's going on?
Our site is a cold fusion site. Part of the site is dynamic and part of it is static.
We have rewritten all of our urls from the following format:
www.example.com/view.cfm?type=Exmaple%20Of%20Url
To:
www.example.com/emaple-of-url.html
The rewritten url pulls the old url and converts it behind the scene to the new url.
If we do a 301 redirect from the old url to the new it will create a loop, being that again it works as follows:
A. New url
B. Pulled from old url
With redirect:
A. New url
B. back to old url
C. 301 redirect-redirect to new url
D. New url back to old url
E. This process creates a loop
We where suggested to put the old url's in our robots.txt file so Google will not cache our old url's. (is that a good enough option?)
Being that our rewritten url's are in supplemental results, which hardly ever gets cached, will we ever see these pages out of supplemental?
Anything we can do to have Google expedite the process?
Why should we want to add the the view2?
The reason we have done the rewrite is to make our url's more user and search engine freindly..
repointing all links that went to example.com/view.cfm?type=Name%20Of%20url to example.com/name-of-url.html
301 redirecting all view.cfm?type=Name%20Of%20url to example.com/name-of-url.html
rewriting example.com/name-of-url.html to example.com/view2.cfm?type=name%20of%20url
Added advantage was after the 301's our supplemental page is gone and we regained rank recently lost. (came back to top5 from 70+)
We where thinking about this..Wasn't sure if it's going to work or not..
Where your pages in supplemental as well?
If so, how long did it take to get out of supplemental?
[edited by: F_Rose at 5:18 pm (utc) on Sep. 27, 2006]
So what you are saying is, the ones you have not tested yet have gone out of supplemental as well?
[edited by: F_Rose at 5:33 pm (utc) on Sep. 27, 2006]
Please clarify..
Your pages went supplemental..
So what you have done was changing the true url to view 2, and then you have done a 301 redirect from the true original url to the rewritten url?
Have you done this test with one page only?And everything went out of supplemental results?
If that's the case how could you be definite that this technique is actually working for you? Maybe Google got you out of supplemental for a different reason?
Sorry if I am being too harsh..I just want to get this straight,if it has worked for you, I definitely want to test it on our site..
Thank you for your quick response. I truly appreciate it..
So the one you ran a test for, was in supplemental or just had dropped in ranking?
Is it possible for you to pm me your web address?
What would be your recommendation on the above topic?
Anyone with similar experience and would like to share it?
Anyone with similar experience and would like to share it?
F_Rose, I do not have the experience of getting this issue fixed, but like you, my site uses ColdFusion via my eCommerce Hosting Provider. On September 15, my site dropped from number one for many keywords to very low rankings. It appears that about 80% plus of my pages went to supplemental. So, my traffic is WAY down and thus my earnings are way down as well.
Ok, so, what I have also been asking is, Is there a way to get your pages OUT of supplemental?
Would the best approach be to copy my pages that are supplemental, such as:
CURRENTLY www.example.com/pd_widget.cfm
CHANGE TO www.example.com/pd_widget2.cfm
Then do a 301 REDIRECT FROM the CURRENT page that is supplemental to the NEW CHANGED page?
Is this what might work?
I'm afraid of doing this to find out that with the next update or data push that my pages would have come back anyway, because I now have my 301 redirect in place from non-www to www.
Anyone else know or have experience with this?
This post was made in October of 2005. Whether this is still timely or whether things have changed since this post, I have no idea.
I do wonder if this works. It states that it is a good thing to try if you have like 20 pages or so, however, I have like 800 plus pages that went supplemental out of about a thousand.
However, if this technique works, it is worth the work, however, if it does not work, then it will be a lot of work for nothing.
I frankly am at a loss here. I have no problem with doing the weeks worth of work to get my most important pages out of supplemental but my question is this:
If I build my own site ( a new version of this same site ) and move it to a host where I can control the redirects, etc. would I just be better off building the new pages for the product pages that have gone supplemental, changing them so they are not duplicates of the supplemental pages?
Also, if your supplemental pages are not trusted in Googles eyes, (even though my pages were not something to trust, but just an ecommerce site selling many items that are similar), does this mean even when I build new pages that they will never be trusted very much, thus not receive any high rankings due to having so many pages in supplemental?
As far as getting pages out of supplemental that do not suffer from a duplicate content penalty, I bellieve the only way is to get outside inbound links from a quality source to those particular pages. I am running an experiment on this now but will probably be a while before I know for sure.
You should be able to add an external redirect from "A" to "B" and send a 301 response with it back to the browser so that it then knows to request the correct URL.
The stuff from "B" to "A" is an internal rewrite (not a redirect) and works entirely internally in the server.
There should not be interaction between the two. They should not cause a loop. I am sure that I have seen prior posts about this working.
Two redirects would cause a loop. You don't have two redirects, you have a redirect and a rewrite.
Our site is built part dynamic and part static. Doing a 301 redirect from A to B would be a major undertaking, for that reason I would rather prefer adding a new version (view2.cfm) instead of A (true url) and get rid of A by doing a 301 redirect to B.
The way it would work A redirected to B. A new structure named C (which Google should never get to see) for our true urls.
Thank you.
Before you start with the Google won't like removing the Metas...yea maybe, maybe not, speculation only, but nothing else has ever worked or appears ever will, so why not.
What about the people here that have already done this, and already do know the real answer? Why ignore them?
Take the site that I PM'd you about earlier as an example. That one did have missing meta descriptions on many pages a year ago. Those pages showed exactly the same symptoms as the pages with duplicate meta descriptions did.
You need a meta description and you need a unique one on every page of the site. Please go back and again examine the SERP that I sent you.
Any answers on the 301 redirects for our site?
Should we bother with the 301 riderects, or should we just try to get inbound links on our rewritten urls, which will show Google that our rewritten urls are much stronger than the true url's (which has nothing linked to it)?
And perhaps Google will start to recognize our url's and get it out of supplemental?
There is no other way for you.
I still don't know why you can't have a redirect from B to A, and a rewrite from A to B. I am sure that there was an example thread with just such a thing only a few months ago.
That is said even if we have no links to our true urls anywhere on the web? (Google does have these url's in supplemental)
Once Google has that URL in its database it comes back to your site and directly asks for that URL periodically.
The trick to get the URL out of the index is to make it serve 301 or 404 instead of 200.
They will then file that URL in Supplemental for a year (or leave it there for a year if it was already there), and if it still continues to send a 301 or 404 response it then gets dropped out of the visible index completely and permanently.
It is great information. I still have the following questions if you don't mind.
"Once Google has that URL in its database it comes back to your site and directly asks for that URL periodically."
1. Being that the url's are in supplemental results already do they still come back to check?
2. We don't have full access to our site..The cheapest and easiest way is to do the three way 301 redirect. Being that nothing will be ever linked to the new true url's we will be creating, will Google ever get to these url's?
3. Once fixes are made, what do I do to get our rewritten url's out of the supplemental results, just sit and wait?
If they continue to return "200 OK" and are still duplicates they are left as supplemental results forever.
If the URL is one day found to not be a duplicate (the other URLs are gone, or are 301, or are 404) then it can make it back into the normal index. This can take a few months, but is often weeks.
If the URL starts to return 301 or 404 then it is checked again periodically. After a full year of returning 301 or 404 it gets dropped from all the indexes.
2. Google is unlikely to find those URLs, but you can make sure that it is never a problem by either disallowing them in robots.txt or by dynamically adding a noindex tag if those URLs are directly accessed (need to modify the script to do that).
3. Yep. Once the URLs return a 301 or 404, you have to wait a full year for them to disappear from view. The good news is that within weeks of making them no longer return a "200 OK" status, they are no longer classed as being duplicate content anyway.
.
So, "Duplicate Content" is simply multiple URLs that return the same content, and have a "200 OK" status, and the URLs are not disallowed from being indexed.
Change the status code for all the alternative URLs, or disallow the bot from indexing all the alternative URLs, such that each "page" of content now has only one indexable URL for it, and you no longer have "Duplicate Content".
That's all a slight simplification, because URLs that return a "302 Moved" response can cause duplicate content too. So, avoid 302 redirects completely and you will be safe.
[edited by: g1smd at 5:57 pm (utc) on Sep. 28, 2006]
"The good news is that within weeks of making them no longer return a "200 OK" status, they are no longer classed as being duplicate content anyway."
Being that it will no longer be duplicated contents our actuall url's that are in supplemental should start coming up in Google's regular index? Right?
If so, should that take couple of weeks or rather up to a year, for our regular url's to start reappearing in Google's regular database?
