First off, do your old urls 404 now? If they are essentially the same page then you are going to suffer from duplicate content problems.
Our site is a cold fusion site. Part of the site is dynamic and part of it is static.
We have rewritten all of our urls from the following format:
The rewritten url pulls the old url and converts it behind the scene to the new url.
If we do a 301 redirect from the old url to the new it will create a loop, being that again it works as follows:
A. New url
B. Pulled from old url
A. New url
B. back to old url
C. 301 redirect-redirect to new url
D. New url back to old url
E. This process creates a loop
We where suggested to put the old url's in our robots.txt file so Google will not cache our old url's. (is that a good enough option?)
Being that our rewritten url's are in supplemental results, which hardly ever gets cached, will we ever see these pages out of supplemental?
Anything we can do to have Google expedite the process?
Have you considered making a copy of view.cfm as view2.cfm and doing the rewrite to there? that way you could 301 redirect the view.cfm pages and avoid the loop
Don't get it..
Why should we want to add the the view2?
The reason we have done the rewrite is to make our url's more user and search engine freindly..
view2 shouldn't get picked up by Google if you don't link to it. I did something similar for our site recently by:
repointing all links that went to example.com/view.cfm?type=Name%20Of%20url to example.com/name-of-url.html
301 redirecting all view.cfm?type=Name%20Of%20url to example.com/name-of-url.html
rewriting example.com/name-of-url.html to example.com/view2.cfm?type=name%20of%20url
Added advantage was after the 301's our supplemental page is gone and we regained rank recently lost. (came back to top5 from 70+)
That is great information...
We where thinking about this..Wasn't sure if it's going to work or not..
Where your pages in supplemental as well?
If so, how long did it take to get out of supplemental?
[edited by: F_Rose at 5:18 pm (utc) on Sep. 27, 2006]
The one page we tested this one (just recently made it live) went out after about a week. I was amazed since I was expecting it to take a year!
*Now they are showing that the pages are not supplemental... I wish I could make heads or tails out of what was going on!
"Now they are showing that the pages are not supplemental... I wish I could make heads or tails out of what was going on! "
So what you are saying is, the ones you have not tested yet have gone out of supplemental as well?
[edited by: F_Rose at 5:33 pm (utc) on Sep. 27, 2006]
Apparently, but I'm still not 100% sure why they were showing supplemental in the first place.
Sorry but I am not clear..
Your pages went supplemental..
So what you have done was changing the true url to view 2, and then you have done a 301 redirect from the true original url to the rewritten url?
Have you done this test with one page only?And everything went out of supplemental results?
If that's the case how could you be definite that this technique is actually working for you? Maybe Google got you out of supplemental for a different reason?
Sorry if I am being too harsh..I just want to get this straight,if it has worked for you, I definitely want to test it on our site..
Thank you for your quick response. I truly appreciate it..
I definitely can't say this helped *with getting my pages out of supplemental*. I was under the impression it did till I thought, "With everything else going on, I should make sure those pages are still supplemental before saying for sure that they are." I can say that even though the pages are no longer supplemental they are still ranking very low and we ranked top 5 for most specific searches before last week. Now, the page I have tested using my suggestions has moved back up and I have made the change for all other pages though I'm waiting for the next crawl for them. Sorry if I gave some false hope but I definitely feel like it can't hurt to rewrite the way you are talking about and I would think a 301 redirect would be more effective in getting the new pages indexed and old pages removed compared to simply blocking within robots
You are totally right.. I should give it a shot..
So the one you ran a test for, was in supplemental or just had dropped in ranking?
Is it possible for you to pm me your web address?
I would appreciate feedback on this from other users as well..
What would be your recommendation on the above topic?
Anyone with similar experience and would like to share it?
|Anyone with similar experience and would like to share it? |
F_Rose, I do not have the experience of getting this issue fixed, but like you, my site uses ColdFusion via my eCommerce Hosting Provider. On September 15, my site dropped from number one for many keywords to very low rankings. It appears that about 80% plus of my pages went to supplemental. So, my traffic is WAY down and thus my earnings are way down as well.
Ok, so, what I have also been asking is, Is there a way to get your pages OUT of supplemental?
Would the best approach be to copy my pages that are supplemental, such as:
CHANGE TO www.example.com/pd_widget2.cfm
Then do a 301 REDIRECT FROM the CURRENT page that is supplemental to the NEW CHANGED page?
Is this what might work?
I'm afraid of doing this to find out that with the next update or data push that my pages would have come back anyway, because I now have my 301 redirect in place from non-www to www.
Anyone else know or have experience with this?
Ok, upon doing a search on Google for how to get your pages out of supplemental, I stumbled upon a Webmaster World forum post.
This post was made in October of 2005. Whether this is still timely or whether things have changed since this post, I have no idea.
I do wonder if this works. It states that it is a good thing to try if you have like 20 pages or so, however, I have like 800 plus pages that went supplemental out of about a thousand.
However, if this technique works, it is worth the work, however, if it does not work, then it will be a lot of work for nothing.
I frankly am at a loss here. I have no problem with doing the weeks worth of work to get my most important pages out of supplemental but my question is this:
If I build my own site ( a new version of this same site ) and move it to a host where I can control the redirects, etc. would I just be better off building the new pages for the product pages that have gone supplemental, changing them so they are not duplicates of the supplemental pages?
Also, if your supplemental pages are not trusted in Googles eyes, (even though my pages were not something to trust, but just an ecommerce site selling many items that are similar), does this mean even when I build new pages that they will never be trusted very much, thus not receive any high rankings due to having so many pages in supplemental?
I think that the answer to most of your questions can be found in this thread:
As far as getting pages out of supplemental that do not suffer from a duplicate content penalty, I bellieve the only way is to get outside inbound links from a quality source to those particular pages. I am running an experiment on this now but will probably be a while before I know for sure.
Let me get this right. You have pages indexed as "A". You have an internal rewrite to transform "B" to "A", and you now link to "B" within the site. However, you can still access the URLs as "A" and that is seen as duplicate content.
You should be able to add an external redirect from "A" to "B" and send a 301 response with it back to the browser so that it then knows to request the correct URL.
The stuff from "B" to "A" is an internal rewrite (not a redirect) and works entirely internally in the server.
There should not be interaction between the two. They should not cause a loop. I am sure that I have seen prior posts about this working.
Two redirects would cause a loop. You don't have two redirects, you have a redirect and a rewrite.
Good point g1smd. I tried that with my rewrites and they do work so now I don't need view2.cfm, just view.cfm (well, mine is actually asp but you get the point)
Our site is built part dynamic and part static. Doing a 301 redirect from A to B would be a major undertaking, for that reason I would rather prefer adding a new version (view2.cfm) instead of A (true url) and get rid of A by doing a 301 redirect to B.
The way it would work A redirected to B. A new structure named C (which Google should never get to see) for our true urls.
Any more comment on this topic would be greatly appreciated..
Having a travel based website wherein many descrip meta tags are similar, no way around this, we have chosen to remove all meta descrip tags and allow Google to "look" at the pages content for a descrip. We will let you know how this works out. Nothing else has worked in the past.
Before you start with the Google won't like removing the Metas...yea maybe, maybe not, speculation only, but nothing else has ever worked or appears ever will, so why not.
>> Before you start with the Google won't like removing the Metas...yea maybe, maybe not, speculation only <<
What about the people here that have already done this, and already do know the real answer? Why ignore them?
Take the site that I PM'd you about earlier as an example. That one did have missing meta descriptions on many pages a year ago. Those pages showed exactly the same symptoms as the pages with duplicate meta descriptions did.
You need a meta description and you need a unique one on every page of the site. Please go back and again examine the SERP that I sent you.
Any answers on the 301 redirects for our site?
Should we bother with the 301 riderects, or should we just try to get inbound links on our rewritten urls, which will show Google that our rewritten urls are much stronger than the true url's (which has nothing linked to it)?
And perhaps Google will start to recognize our url's and get it out of supplemental?
If two URLs serve the same content with a "200 OK" now, then one of them needs to change to serve "301" instead, OR one of them needs to dynmically add a noindex tag to the page header.
There is no other way for you.
I still don't know why you can't have a redirect from B to A, and a rewrite from A to B. I am sure that there was an example thread with just such a thing only a few months ago.
"If two URLs serve the same content with a "200 OK" now"
That is said even if we have no links to our true urls anywhere on the web? (Google does have these url's in supplemental)
Yes, it is irrelevant whether anyone still links to any of them now.
Once Google has that URL in its database it comes back to your site and directly asks for that URL periodically.
The trick to get the URL out of the index is to make it serve 301 or 404 instead of 200.
They will then file that URL in Supplemental for a year (or leave it there for a year if it was already there), and if it still continues to send a 301 or 404 response it then gets dropped out of the visible index completely and permanently.
Firstly, thank you so much for sticky mail.
It is great information. I still have the following questions if you don't mind.
"Once Google has that URL in its database it comes back to your site and directly asks for that URL periodically."
1. Being that the url's are in supplemental results already do they still come back to check?
2. We don't have full access to our site..The cheapest and easiest way is to do the three way 301 redirect. Being that nothing will be ever linked to the new true url's we will be creating, will Google ever get to these url's?
3. Once fixes are made, what do I do to get our rewritten url's out of the supplemental results, just sit and wait?
1. Supplementals are checked several times per year.
If they continue to return "200 OK" and are still duplicates they are left as supplemental results forever.
If the URL is one day found to not be a duplicate (the other URLs are gone, or are 301, or are 404) then it can make it back into the normal index. This can take a few months, but is often weeks.
If the URL starts to return 301 or 404 then it is checked again periodically. After a full year of returning 301 or 404 it gets dropped from all the indexes.
2. Google is unlikely to find those URLs, but you can make sure that it is never a problem by either disallowing them in robots.txt or by dynamically adding a noindex tag if those URLs are directly accessed (need to modify the script to do that).
3. Yep. Once the URLs return a 301 or 404, you have to wait a full year for them to disappear from view. The good news is that within weeks of making them no longer return a "200 OK" status, they are no longer classed as being duplicate content anyway.
So, "Duplicate Content" is simply multiple URLs that return the same content, and have a "200 OK" status, and the URLs are not disallowed from being indexed.
Change the status code for all the alternative URLs, or disallow the bot from indexing all the alternative URLs, such that each "page" of content now has only one indexable URL for it, and you no longer have "Duplicate Content".
That's all a slight simplification, because URLs that return a "302 Moved" response can cause duplicate content too. So, avoid 302 redirects completely and you will be safe.
[edited by: g1smd at 5:57 pm (utc) on Sep. 28, 2006]
"The good news is that within weeks of making them no longer return a "200 OK" status, they are no longer classed as being duplicate content anyway."
Being that it will no longer be duplicated contents our actuall url's that are in supplemental should start coming up in Google's regular index? Right?
If so, should that take couple of weeks or rather up to a year, for our regular url's to start reappearing in Google's regular database?
| This 46 message thread spans 2 pages: 46 (  2 ) > > |