I am now using no redirects. All other search engines seem to figure out www vs non-www and I have just given up on Google. What will be, will be.
Dayo_UK et all,
The good news to come out of this fiasco for me apart from my recent recuperation of some traffic, is that the whole process of;
301 -> pages going PR0 -> then losing pages,
did at least lead to a complete flushing of the suplimental pages for the site.
Will the suplimentals come back? I really hope not
anyway, I will post a further chapter if that happens.
Thanks for bringing this to our attention cleanup.
Yes, Google is still handling the whole 301 situation very poorly. They alluded to the fact that a fix was coming with the BigDaddy rollout. Nothing has been fixed and an annoucement was never made that the problem was fixed. The lack of an annoucment is just evidence that non-www versus www are still a problem. How poor for such a large company.
Am going to give this a try (removing the 301 redirect) and hopefully I will see some traffic return to the affected sites.
Since it looks like we are pretty much in the same situation what do you think the best thing to do is?
Do you pull the redirect or continue to ride it out? (it has been over a year though)
Sure would like to see some indepth comments from google on this.
I remember when we slipped and had issues in google last year. It seemed like google was trying to index us under the non www version even though redirects and on-site urls pointed to the www version. A bunch of havoc resulted from this. Over the course of the year they finally started to get things a bit more right. That didn't occur until the home page went to our correct www version in the site: command but still the home page was buried. Eventually that did correct and we seen some improvements for the home page but the rest of the site never bounced back.
3/4 of the site is now in the index with the correct www versions but PR just does not seem to want to take hold. Since the redirect was in place before all the havoc non www version would have no PR. Google pushing the non www versions droping the www versions meant that over a years time PR was lost for those www pages as well as the home page www version.
This is how I seen those events and not sure how much has been corrected. But from some of the symptoms I have seen lately it mimics to a lesser extent of what we went through last year.
Well it seems that the indexing mechanism of yahoo and msn is far better than that of Google. I really wonder how my site got indexed in Google without a www whereas the same site had been indexed with www in yahoo and msn. Infact i really can't figure out that how the home page of this site is indexed in Google with www and the internal pages are indexed without www. It will be very helpful if you all post your opinions on this in this thread.
We replaced the 302 redirect with 301 to give clear instructions to google that it should index the www version and transfer all the credit (PR and backlinks) to the www site. It seems that google has somewhat messed it up coz the PR is now only concentrated to the non www pages and the with www pages now show a PR of 0.
Now i need some help from all of you regarding my action plan to do some damage control. Following are some cases for which i need feedback from u all :
Case 1 : I remove the 301 redirect. Have no redirect at all. Response code 200 ok. The possible result is that Google improves and may be the toolbar PR is visible again. But what about Yahoo and Msn. The site has its pages listed in Yahoo as [domain.com...] unlike that of google where it is listed as [domain.com...] SERPs in Yahoo and Msn point to [domain.com...] and so possibly a click on that url in the SERPs will lead to "Page not found".
Case 2 : Earlier the internal sections had a 302 redirect from non www to www. This was changed to 301 and then the abovementioned problems surfaced. So i agsin change it to 302 and hope thigs to get back to normal. The problem is that Google and various SEO experts have always mentioned that 302 should not be the preferable method of redirect.A change like 302->301->302 may initiate new poblems altogether.
Case 3 : Replacing the 301 redirect with a meta refresh zero second redirect. Well i really doubt this coz meta refresh is a well known method to implement door way pages and i am pretty sure that Google will not like it. Well still if anybody among you have some views on this then please post it in this thread.
Pls correct me if i am wrong with my analysis of the above mentioned cases.
Looking forward to your valued opinions and suggestions.
Thanks and regards
"Case 1 : I remove the 301 redirect. Have no redirect at all. "
All I can say here is that throughout all this MSN and Yahoo have indexed and maintained the pages faithfully as www. So I do not think you should worry about that.
as to your other questions,..well you know from my posts that I have removed the 301's and had the site revived.
I honestly don't think that anyone can tell you what is the best way is(not Google it seems thats for sure).
"Correct" and "best" are not always the same thing when it comes to Google.
I will say however that if you do not see positive resuls fairly quicky for the 301 redirect etc (eq 2 months max) then I would asume you have fallen through one of the many holes in Googles indexing process.
Can anyone else share their experiences?
Well I am looking for alternative employment at the moment.
On a site management issue in the future - any new site I may launch in the future I will probably redirect www to non-www.
The sites I already have a redirect from non-www to www I will leave the redirect in place.
From what I can remember from Cleanup site the redirect was more to do with cleaning up the supplemental rankings rather than a serp ranking issue - the reason for my redirect was to cure the ranking problem that went with the canonical error.
Arubicus is 100% correct PR seems to have been correctly recalculated for the www homepages in a lot of cases but the rankings have not returned - I was hoping that this might happen at a "ranking refresh" that MC likes to coin nowadays - but we have had yet another one (perhaps the 1st since Big Daddy) and still no improvements.
From what I can tell:
1. Some sites with 301 redirects from www to non-www seem to work properly.
2. Some sites with 301 redirects from non-www to www (such as webmasterworld) seem to be doing just fine.
3. Some sites with no redirects (Matt Cutts' blog) seem to also be doing just fine.
(Much of this may depend on certain factors; such as having the 301 in place when the site was first active)
Google may be wanting a different version that the one some of us are trying to "force" causing hiccups.
Our site had the redirect in place around the 3rd quarter of 2004. Things were sort of acting funny but nothing really serious. We got nailed early 2005 where our site home page and deeper pages were just trying to go to the non-www version.
I would be weary that msn may not like a switch to an alternate version as I have seen them index correct versions of pages after a 301 redirect but positions were lost (links not passing?). This is what is keeping me from switching as a test because we could very well kill much needed revenue if we make such a site wide change and plop ourselves out of the ranking for a while. Until I confirm certain pages returning and everything is okay then I may attempt such a change with an expected timeframe to "fix" in the results. Although...I really am getting to the point of despiration and may do it for the hell of it.
The only way I would go would just yank the 301/302 both and serve up both versions. Yes this may split PR but we did just fine for years without the redirect and Google should be smart enough to know by now that both versions are the same exact page and not penalize a whole site for any sort of duplicate content (Again Matt's blog is doing fine without redirects and many others).
[edited by: arubicus at 8:00 am (utc) on July 8, 2006]
"Well I am looking for alternative employment at the moment."
Actually I will have to go back to waiting on tables in a week or two. The money is drying up after a year and a half of patience with google. Glad I saved a ton just in case of times like these! To top it off my son just had brain surgery so that is going to hit the pocket book. Again I have the money to cover it but it will run us dry. My job will just be part-time to replenish what we will spend on medical bills then to add to the current site income just to keep ahead. I will avoid any debt like a plague and I will do what I must. I am still thankful I won't have to work full time.
I would be glad to fix our site if I knew what the darned problem is. I mean if I were doing something real shady I would probably know that I was breaking the rules so I could fix what I did wrong. But I try and play by the rules and those rules are so vague I don't have any clue as where to look nor any info as to where to begin.
>> Again Matt's blog is doing fine without redirects and many others <<
Matt's blog does now have a redirect from non-www to www. It has been in effect for several months I think.
No redirect to www when I check Matts site..
I felt sure there was a redirect there last week.
There certainly isn't now... musta dreamt it.
Since I believe this is an important thread and needs to be fully explored here is a question.
Has anyone else tested removing www 301 redirects with any positive results especially those who have put this redirect in place LONG after a site was established (not when the site was first developed)?
arubicus, sorry to hear of your tough times right now, hope everything works out for you.
|Has anyone else tested removing www 301 redirects with any positive results especially those who have put this redirect in place LONG after a site was established (not when the site was first developed)? |
Yes, I am in this category except that I am still waiting to see the effect of the removal of th 301. Will detail the situation for clarification.
I have a site that was started approximately September 2004. Site never got out of the sandbox but picked up long tail traffic. Put in a 301 from non-www to www approximately April 2005. Site pages were split between supplemental and non-supplemental since then and has lost virtually all traffic (long tail) since. Site currently shows cache dates of Dec. 2004 - can you believe that despite the site having many links! Just removed the 301 last week. No changes yet (in traffic, cache dates or spidering) but will update if anything positive results.
Wanted to mention this in my last post.
My belief is that if you have a 301 redirect from www to non-www (or the other way around) from day 1, this is okay. You just seem to get killed if you do it at somepoint after the site was launched. If you didn't put in in day one, don't put one in at all is my advice.
I have seen a lot of sites that had both www and non-www showing in the SERPs and which were doing badly in Google about a year or 18 months ago. They had many Supplemental pages, many URL-only listings, and many pages showing as either www or non-www. Some pages were listed twice, while a few others were completely missing - not listed as either version.
In all cases, a 301 redirect pointing to the www version cleaned up the www listings within a few weeks (they all gained a title and snippet), while some of the non-www listings went Supplemental and took more than a year to disappear. For some sites some of the non-www results are still showing up. The www pages are all doing well.
Factors to consider when deciding which way to redirect, from www to non-www or from non-www to www:
- Which set of pages www or non-www, has the most indexed?
- Is the root index page shown as the non-www or as the www version?
- If both show up, which has the highest PR?
If you redirect TO the version that has the smallest number of pages you give Google a lot more work to do. I would redirect to the one with the most pages listed.
I would redirect to the version that has the strongest root index page, the one that isn't URL-only or has low PR showing.
What happens when www has the most number of pages showing, but the stronger of the root index pages is the non-www version? (and vice versa) .
Which is more important - the number of indexed pages, or which version the root index page is shown as?
Now that really would be a useful hint from Matt or Adam.
I have had my site up since 1999. All was fine until July/August of 2004. I have made a ton of changes since then including the 301 from non www to www.
Recently I have tried removing the redirect a couple of tmes and each time within 24 hours my index page disapears from the site: listings whereas with the redirect it is 3rd down. All my 800 pages are supplemental except for my home page and a few others either way. So I guess I'll leave it.
[edited by: Tami at 1:38 am (utc) on July 13, 2006]
This site is htm
I had an Issue with this as well notice I said had...I hardcoded every link on the site with the www. so there could be no accident from someone linking to me without the www and google crawling it that way. It took about 5 months did a site map etc but it has finally been indexed correctly www and begain reindexing the site and pages are coming back.
I had lost the whole site due to this issue.
I mean every link picures, css files anything with a link is now harcoded with the whole url
We as well did this with on all our database driven sites 20+ as well and so far by the Gods Grace we haven't had one site go into this.
We have not set up any 301 redirects as I felt this wasn't the way to go. Hardcoded the links.
I really feel if you will take the time hard code every link on the site as you want it your problem will be corrected or you won't have this problem develop.
just my 2 cents and something that has worked for me
I will be praying for your sons speedy recovery as well as your return to your business.
My observation is that most often my good sites have for example G Page Rank of 4, for both the www and non-www version.
Since ub this commn example they are both listed in the serps (with a combined PR of 8) what would be the benefit of doing the non-www to www when that would likely eventually eliminate the non-www from the serps.
As an end result it would seem you would end up with only the www listed and *possibly* an increase in PR from 4 to perhaps 5 or even 6 (but may not go up at all).
The move would be extremely unlikely to result in a PR of 8 for the remaining listing since a PR8 is almost impossible to achieve anyway, especially for a small website as in my case.
In my view you are better off with two PR4 sites in the serps, with subsequently a double chance of someone finding it, vs just one PR4 (or possible somewhat increased PR).
Of course, my reasoning may not be correct so please give your opinion on this.
Hmm...let's say that before you put your redirect [non-www=>www] in place, that there was another site that was doing a 302 to your www home page, and THAT url was in the Google index showing your title, description and url.
When you finally did your non-www=>www redirect, wouldn't that have caused YOUR pages to show up as dups of the other external site that was indexed as www first?
You were indexed as cleanup.com.
Externalsite.com was doing a 302 to www.cleanup.com and was indexed as such. (example: externalsite.com/linkto.php?url=www.cleanup.com).
When you did your redirect to www, didn't that make you a dup of the above url in Google's eyes?
Just a thought.
Yes, its possible - but I have enough sleepless nights worrying about 301's without bringing 302's into it.
In fact there is more to this story and Matts so called "data push" does seems to be producing some interesting results at the moment.
I will update on the results if/when I can draw some conclusions.
|In my view you are better off with two PR4 sites in the serps, with subsequently a double chance of someone finding it, vs just one PR4 (or possible somewhat increased PR). |
Not at all, currently this is a recipe for disaster in Google. I would redirect using a properly tested 301 redirect from either www to non, or vice versa.
|All I can say here is that throughout all this MSN and Yahoo have indexed and maintained the pages faithfully as www. |
Not the other way around, I'm afraid. I couldn't care less about the PR3 transfer from my www to my non-www, but the reason I have a www >> non-www 301 is that Yahoo stubbornly indexes the www version, which leads to #1 positions going to #14 positions.
|My belief is that if you have a 301 redirect from www to non-www (or the other way around) from day 1, this is okay. You just seem to get killed if you do it at somepoint after the site was launched. If you didn't put in in day one, don't put one in at all is my advice. |
Sounds like good advice with regard to Google, but then I'm back to square 1 on Yahoo.
If I try to read between the lines I get the impression that a well-informed poster like aribicus takes into account the possibility that 301 is (among other things, perhaps) responsible for 6/27.
On the other hand, WikiWidgets also has a www >> non-www 301 and they seem to be ok (more than). Arghhh.....
I think it's time to pose this question to a certain blogger that has a cat, 23 hard disks and a bad digital camera.
[edited by: Martin40 at 8:24 pm (utc) on July 13, 2006]
"well-informed poster like aribicus takes into account the possibility that 301 is (among other things, perhaps) responsible for 6/27"
I don't know about well informed:) But the only way we can get some insight is to share experiences and even experiments. What is funny is in recent months webmasters in bulk have reported very similar experiences (some to a lesser degree and some to the same degree) that mimic very similar traits that we experienced last year. What happened to us was a complete loss for about a year and a half and many hundreds of thousands of dollars lost.
During the time of our drop was the last of the monthly and quarterly timely updates in which we placed a 301 redirect a few months before the drop. Late 2004. During that update in feb of 2005 and another in March there was supposed to be a 302 redirect fix due to hijackings. This is when we seen major 301 problems and our correct www pages disappearing while non-www pages reappeared and 4 versions of homepage were appearing and all sorts of screwed up things in between. Our properly placed redirects should have taken care of the whole kit and kaboodle. It didn't. Our non www pages had no PR so when they came back (supplemental) and the www versions disappeared completely our PR tanked with it and it has been hell ever since.
One of the symptoms was that we would not come up for our unique company name. Another symptom was the the home page would be buried deep within the results. And pages were dropped and come back then dropped again then come back. This page dropping slowly leveled off until a few months ago when it started to happen again. There were other things suspect at the time and we have tested and exhausted and cleaned up most all possibilities except the tons of scrapers and content theives.
We cleaned up everything walking through every single page validating and re-coding a couple of thousand pages. The only other thing I can think of is we have partner authors/sites who submit to us for display on our site. Yes, some of them do get re-distributed but we never minded this as Google treated those pages individually and never penalized the whole site. We checked every outgoing link for suspect "bad neighborhoods" and got rid of them -- Whatever bad neighborhoods are and how we would possibly know what google considers as a bad neighborhood at any moment in time.
We have filed reinclusions but no response. Several times we asked about penalties and were responded with a no. So what is left. Links? All links are natural except for direcories we submitted to. We have thousands of links to the home page and even more throughout our articles. Links are not showing for internal pages and just a few for the home page. Go figure.
We have come back partially with little ranking. PR on our home page went from 0 to a 5. Nothing yet on internal pages. We have noticed that google has been crawling our site like never before mainly because we added an on-site sitemap raising the level of our deep content. (Google sitemaps showed little help as far as crawling - We have been using it since the start). It does appear that google may be doing a supplemental crawl as we finally see some old lingering url's 301'd or removed pages getting crawled.
This is just some of the stuff we have experienced that other webmasters are now experiencing. I don't know the answer obviously. All I know we have begun to promote ourselves in every way we can to make up for the loss. If we can add our link to a site or write content for others we will do it. I don't care if it helps or not in serps. If we get traffic or brand awareness that is all that matters. I have a business to run and a family to take care of. If what I do is legal and morraly correct in that it provides ethical opportunities/add value to our brand...I will do it. There was a time where I was careful not to upset big G. Now it is about survival but not at the expense of my morals.
I would like to add that for every instance of a suspected problem we have come up with and others have noted there can be tons of sites with similarities that are having no problems.
We are nearing the end of what we can do without starting all over. If that were to happen we would have to rebuild and rebrand what we have established over the course of 5 years. I am not willing to do that just yet.
Arbitrary may be correct: "My belief is that if you have a 301 redirect from www to non-www (or the other way around) from day 1, this is okay. You just seem to get killed if you do it at somepoint after the site was launched. If you didn't put in in day one, don't put one in at all is my advice."
I don't know about killed but could be a re-sandbox effect or other temporary effect if you don't do it the way google expects; going the wrong way with the redirect. Although, I have seen a couple of our newer sites get killed just the same and those had the redirect from the beginning. We found a couple of issues with those as in Google adding a trailing slash at the end of a filename such as .php where our Apache servers would still serve up those pages -- a 404 is issued for .htm(l) pages with trailing slashes. Who knows where they came from but people should also watch for this occurance. Those php pages had no query strings at the end so I don't know if it will happen with those. But, if they do plop in those trailing slashes on pages carrying the php extension it will dump your site into supplemental hell as we have seen.
If I knew the site, I might be able to help diagnose more. I guess that's out o' bounds though..
Have you tried other redirect code? There are a ton of them out there. Disable FP extensions? I have seen them screw up even the best configured servers.
Test. Test. Test. I find it difficult to believe that it is the 301 - there has to be another symptom on the server that is leading to this problem.
We crossed paths some time ago when this first occurred. Exchanges some general thoughts passing information to other webmasters and I had a chance to send the site into the G engineers a year ago on your request with many of examples of mess ups we experienced at that time. No information came our way through our sharing. Not getting down at anyone as I understand G needs to protect their algos and I shouldn't have had expectations that any information came our way...sometimes I let it get on my nerves thinking about it.
Now a year later...
PR update today has shown positive results. Internal pages showing a few links, finally, from internal pages hardly any external links which is inaccurate or just not shown on purpose but way fewer that what ever was shown. Most all old junk pages we got rid of a while back has been cleaned up. Our site home page is listed first and we rank #1 for our unique company name (not a keyword name but a proper 1 of a kind name) this happened earlier this year. The only thing left that needs done is to get rid of any supplemental pages that carry the old php URL extension which tend to override the new static pages. These php pages have no PR links and way old cached pages like the are supposed to just seem to be preventing the proper ranking of the correct page. These supplementals may not show under the site command only on a keyword search. Googlebot has been crawling a ton of these pages recently in the past month or two and I do expect that if there is no problems with the 301's set up then these pages should cleaned up - - but we are still waiting on this to happen.
So yes there have been improvements throughout the year. I am thankful for that. I don't know what happened to cause this in the first place nor do I know what was done right on our part (the cleanup, redirects, and whatnot...), or what was done on G's part. I really don't care. I just want to get it all straightened out so that our site can get indexed/ranked on actual merit/value (good or bad) and not influenced by unforseen problems and hiccups (G's or ours). I guess a simple answer back, reinclusion, after reinclusion, contact after contact saying it is your problem webmaster not Google's or vice versa would have been sufficient enough for me to keep digging away or relax and wait (providing more info if necessary). At least then there would have been one direction to follow.
Right now my EGO says not to tell you the name of our site as we sat out this long actually having faith and more patience than anyone I have ever seen have. But the bills my son has accumulated and knowing that other people's livelyhood is at stake (employees) my morals, faith, and beliefs are saying to humble yourself and simply ask once more for help...
"If I knew the site, I might be able to help diagnose more. I guess that's out o' bounds though.."
Not that far out of bounds through. Cutts' blog is a still in bounds. That is one place I do enjoy reading and posting on.
As for the indexing problems.. why some index as www and some as non-www.. I think it depends on how it was indexed.. And personally I think the Google toolbar is spying on what people type or what websites they go to. People often won't bother with the www's so if you have no links to non-www surely this is the only way they must be indexing such things. Also one of my personal sites got indexed some-how despite having NO links from anywhere and me not submitting it or anything. In fact I've had a job getting it back out.. google eventually did (ish).. yahoo is still indexing half of it.. however i changed the content if it comes from a search engines to something rather blank and robot.txt 'd it to go away. Now i only have one page in google (just the main url.. not sure how to dissallow that one)!?!? and 'just' six in yahoo now. Hmm.. one day it will look at the robot.
Also one thing to note.. google and yahoo are the only places my site has been indexed.. and coincidently are the only toolbars i've installed. Hmmm.
As for the 301.. I think there is a big difference between 301'ing pages within a site and 301'ing pages that Google thinks is a different site. It likes to keep everything in 'different' sites very separate, and unfortunately Google thinks the www and non-www are totally different sites. I am now playing with 301-ing some pages I've moved folders and see how easy that works. Hopefully since Google considers it the same site it should work and I wont have the same problem you guys are unfortunatly having.. but we'll see.
[edited by: K199a at 9:27 am (utc) on July 14, 2006]
| This 101 message thread spans 4 pages: < < 101 ( 1  3 4 ) > > |