| This 233 message thread spans 8 pages: < < 233 ( 1 2  4 5 6 7 8 ) > > || |
|A real Google conundrum|
Established site losing all its Google traffic
| 1:07 am on Mar 27, 2005 (gmt 0)|
I have a well-established site that over the past few days has lost almost all its Google referals. I think I know what's wrong but have no idea how to fix it.
First some background. The site is a well-established, deep-information site with many, many thousands of pages and a PR 6 on the home page. While we have attempted to get some links to us, most of the hundreds of links to us are spontaneous from a variety of professionals who find our content useful. Therefore, we're not at all dependent on the "latest" SEO tricks - totally white hat.
Up until this week, we got >15000 Google referals a day. We are not dependent on ranking for "blue widgets" or any other identifiable term - our referals come from thousands of different keywords a day which reflects the diversity of our content. Therefore, only a massive drop in the SERPs across the board can cause a >90% drop in referals, as we are seeing.
We still are in the index with the same number of pages and our backlinks don't seem changed. We still have the same PR showing throughout the site (for whatever that's worth since if there are changes, they probably wouldn't show immediately anyway).
Here's the kicker: Another site we own, let's call it widgetville.com, is showing up ahead of our real site, widgetville.org, in the SERPs when you search Google for "Widgetville". The higher widgetville.com site is shown without title or description. Widgetville.com has been 301 redirected to widgetville.org. Widgetville.com does have a backlink or two out in the world, but not the hundreds that the real site, widgetville.org, has so I don't understand the higher ranking.
If you search for "a bunch of widget words that you find on the front page", three other web sites who quote our mission statement appear on the page and our page doesn't. However, if you click on the link to show "omitted" results, we are listed as the omitted page.
In a way, it seems almost like our home page has been hijacked by our own non-functioning site. And it also seems to be like the whole canonical root problem that trips up some site owners except in our case it is between two domains, not a problem of Google getting confused between widgetville.com and widgetville.com/index.html.
We've had this problem before - a year ago - and I queried Google about the problem. I was told that it was a problem on their end, not mine, and they would fix it. The widgetworld.com listing was removed and within weeks, my traffic grew from a trickle to where I started hiring people to deal with the blossoming new customer base. Now all of that is threatened.
So, anyone want to take a crack at explaining this or giving advice on how to handle it?
I have taken one step to see what happens. I've removed the 301 redirect from widgetville.com and put a simple sentence on the page that says to click on the link to widgetville.org. I did this to disassociate widgetville.com from widgetville.org in case Google was somehow seeing duplicate content from the 301. Not sure how that would happen exactly since that is the prefered method of dealing with pages that are no longer valid, but this whole thing throws me for a loop.
| 1:54 pm on Apr 13, 2005 (gmt 0)|
|what's happened is a broad decline in rankings, almost all of them for non-competitive terms. |
This is called "Google$ optimizing SERPs for revenue". Their revenue, not yours. That can be done only one way - less free traffic to fellow webmaster who doesn't pay up in AdWords, more traffic to those who do.
|Visit Thailand: I am not making any changes and just keep adding content |
Yeah, that is a legitimate business model, right...
| 2:01 pm on Apr 13, 2005 (gmt 0)|
|Yeah, that is a legitimate business model, right... |
This strategy has worked well for us for over 7 years, so I do not see any reason since the change 23/24 March to suddenly start changing things.
If in a few months the situation is still the same then I will take a closer look and review. We still get thousands of referrals a day from G just not the good 5 figures we were getting.
No point in panicking when nobody knows what is happening.
| 2:36 pm on Apr 13, 2005 (gmt 0)|
Three years ago, we had the same problem. We were not well informed, and started some knee jerk. What a mess that became. It took us 6 months to recover that site's traffic 100%.
The worst thing you can do when you see any change in serps, is to knee jerk. You compound the problems, and end up having so many problems, you just about have to go back to scratch, because the number of things that may now be wrong, have become too complex to overcome in a timely manner. That is what we had to do to recover. It was quicker than the tweek/wait repetitive trouble shooting process. Of course, that worked. But, what a huge problem and resolution process it became.
As said by others ahead of me. Sit tight, and watch all the stats, very closely.
| 2:45 pm on Apr 13, 2005 (gmt 0)|
I would agree not to do anything rash, which was Brett's wise counsel.
But definitely don't do nothing. Had we taken that approach, we wouldn't have noticed 301 confusion which may have caused huge problems. Instead, look for unusual behavior and root it out clear problems immediately.
Just don't speculate about hypothetical problems with the new Google algo and try to get around them.
| 2:52 pm on Apr 13, 2005 (gmt 0)|
<As said by others ahead of me. Sit tight, and watch all the stats, very closely.>
Sit tight....For how long?
| 3:02 pm on Apr 13, 2005 (gmt 0)|
Not sure waiting is a solution to this problem!
you need to check through all the entries in
Check all entries that do not belong to your url for a 302 redirect to one of your pages.
If you find any then contact the websites involved
Then go use mcdar.net to look for your home page page title go as far back as you can - by doing this i found a metarefresh that had hijacked my home page in the datacenters, i may not have discovered this for many months if i had not used mcdar.net
also contact the webmasters involved if you find any metarefreshes and ask them to remove them.
If you find both www.mysite.com and mysite.com either mod rewrite or have someone mod rewrite for you.
If you suffer from the above problems contact google ( although the 1000 char limiit is a bugger) just incase they will do something (i actually got my email passed to an engineer, not the usual canned email).
If you want any information on what i did feel free to sticky me
| 3:10 pm on Apr 13, 2005 (gmt 0)|
You said it Broker_Boy.
The double or more indexing of a site by Google does trip the duplicate content filter, drive pages to supplemental status, and under certain conditions split a sites PR thus causing a cascade effect.
Add a few "vanity" domains to the mix or have a large site you might even get classified as a Search Engine Spammer.
| 3:26 pm on Apr 13, 2005 (gmt 0)|
Rewrite rules for Apache in general form can be found in this thread on page 2 :
You might want to read the entire thread and this as well :
| 3:37 pm on Apr 13, 2005 (gmt 0)|
What do you mean by vanity domains?
| 3:38 pm on Apr 13, 2005 (gmt 0)|
It seems that, Google has panelized, through allegra and later at 23/24 th March, all what Google engineers regard as spamming.
Spamming in the eyes of Google covers, but not limited to:
- Sites with duplicates including 302 hijacked sites/pages.
- Authority sites where the keywords/keyphrases densities are high (which is unavoidable for an authority site).
- Black-hat & white-hat SEOīd sites, which Google might have considered as "gaming".
Therfore I canīt see the wisdom in the argument "sit tight..watch..wait" at atime where action is most required.
| 3:53 pm on Apr 13, 2005 (gmt 0)|
I don't think the penalisation was intentional i think the root of the issue is googles inability to pick the 'most important page' and not a intentional effect at all.
Further more the reason it happens to large content sites is that big sites have more links (in general) and therefore more chance to have a incoreect link pointing to the domain to cause the www Vs non www problem.
Large content sites are often at the top of the serps ... it is these sites people try to hijack ....
i believe they are trying to sort out the redirect issue but rather than improving the situation the effect has now been replicated both with 301's and also metarefreshes.
Google is therefore seeing duplicate content where there is none. This is causing the problems not a filter designed to penalise people gaming the system!
Like i said check everything and then recheck it - don't beat up on google isolate the problem and then deal with it as best you can
| 4:20 pm on Apr 13, 2005 (gmt 0)|
That depends. Which came first. the .com or the .org.
Google indexed us on April 4, 5, 6 and 9. We seen some unusual behavior during the period stated above.
They cached our April 4 page, then next view late that day, our March 30 cache was back up. They cached our April 5, cache was updated same day. Then, they cached April 6, and it was fine. Until April 7, when the March 30 cache came back. When we were indexed April 9, cache was updated, and remains updated. April 4-6 cache updates were delayed three days, as expected. Just thought I would mention that. To the topic...
If the .com came first, I think they picked it up from a referring link, that may have been indexed at that time. It had longevity, and it rose above the .org. Because the .org has the real 'value', the next indexing could correct it. That is why I said, wait and see what happens. We have seen this happen to a client site.
The client had jerked away his domain name, and decided to put it in on another host. This was going to disrupt their ranking, so we screamed. We had him ranked top 10 for hundreds of keywords. So, we slapped together a .net site, duplicated the content from the .com with minor modifications to avoid a mirror, and registered the domain.
He had a large client base, pending orders, and needed a live site, with an email address (which went dead as soon as he sent the domain name to the new host). The solution had him back in business in two days. All the search engine listings were going to a dns error. So, I put his link on my main site, and got it indexed the next day, and he had a new listing, and some traffic flow.
Now, when I got the .com domain back on my host, and pointing to the old site, it again was indexed, and he had two listings. Fine, we knew it would be resolved, and it did. The .net eventually fell out, and everything was back to normal.
Then, suddenly a year later, the .net listing popped back up, out of the blue. We determined that there were a couple well-meaning people who had dropped the link to the .net in a forum. Apparently, the forum had gone dead, and a new owner brought it back from the dead. When he did that, the link to the .net was indexed. All I could figure was the .net listing had gained longevity against the .com, because of the dns error.
I think it will correct itself, give it time.
| 5:14 pm on Apr 13, 2005 (gmt 0)|
|This is called "Google$ optimizing SERPs for revenue". Their revenue, not yours. That can be done only one way - less free traffic to fellow webmaster who doesn't pay up in AdWords, more traffic to those who do. |
Oh, please, let's not dredge up the old "Google is trying to sell more AdWords" conspiracy theory again. That might be a valid argument if only e-commerce and affiliate sites (or even pages) were being hit, but such isn't the case.
|Yeah, that is a legitimate business model, right... |
As VisitThailand pointed out, focusing on content and a white-hat/organic SEO approach has been a very successful business model for some of us. My own site's Google traffic remained steady or increased through updates (such as Florida and Allegra) that had many SEOs and site owners gnashing their teeth, and other members have reported similar experiences. You may not be comfortable with an organic "information site" approach, but don't knock it if you haven't tried it. :-)
Re the www vs. non-www problem: That may be an issue, and I've implemented 301 redirects for the www versions of my URLs since they may be helpful and aren't likely to do any harm. But from what I've observed in studying my own SERPs and server logs, I think there's more to the referral changes that some of us have seen than Google's confusion about what may or may not be duplicate content.
Side note: The SERPs are still changing for several keyphrases that I track closely; for example, I've gone up from 21 to 9 for one phrase, and a competitor who was in the top 10 has now dropped out of sight for that phrase, though he's still in the top 10 and I'm way down in the rankings for the inverted version of the phrase. I have the impression that this is more than the normal Googleflux, and it will be interesting to see what happens in the weeks ahead. FWIW, I've had a very noticeable increase in referrals from MSN and Yahoo in the last couple of weeks, so their engineers apparently have been busy, too. :-)
| 7:30 pm on Apr 13, 2005 (gmt 0)|
<FWIW, I've had a very noticeable increase in referrals from MSN and Yahoo in the last couple of weeks, so their engineers apparently have been busy, too. :-)>
The same here as I mentioned in a previous post.
And because I have noticed that you love conspiracy theories, here is a new one:
Yahoo and MSN have figured out that the sites Google removing from top of the serps are of quality to searchers. So they wrote their new algos to target such sites and bring them up to the top of their own serps. :-)
| 7:39 pm on Apr 13, 2005 (gmt 0)|
Is it possible that Google temporarily puts new sites high in the SERPs to see if they do better than the old standbys? If they do, they keep them. If they don't, they put back the old site. Plausible?
| 8:08 pm on Apr 13, 2005 (gmt 0)|
<Further more the reason it happens to large content sites is that big sites have more links (in general) and therefore more chance to have a incoreect link pointing to the domain to cause the www Vs non www problem.>
Furthermore authority sites have in general great numbers of backlinks, mostly because other publishers find those sites useful for their own visitors. Hence they are more subjected to a possible backlinks spam filter.
<Google is therefore seeing duplicate content where there is none. This is causing the problems not a filter designed to penalise people gaming the system!>
The problem isn't just that Google seeing duplicates which don't exist. The main problem is that Google removing innocent sites from their positions on the serps and causing the lost of traffic generated by such positions, because it is seeing imaginary duplicates.
How could this be achieved without a filter?
| 8:18 pm on Apr 13, 2005 (gmt 0)|
|Is it possible that Google temporarily puts new sites high in the SERPs to see if they do better than the old standbys? If they do, they keep them. If they don't, they put back the old site. Plausible? |
Unlikely. In looking at some of Google's recent SERPs, I've seen musty old directory sites that I hadn't thought about (let alone run across) in years.
| 6:06 am on Apr 14, 2005 (gmt 0)|
I humbly think that if G is giving dupe content to www.domain.com and domain.com then that is a problem they must be aware of and need to fix.
I am not changing anything on that for the moment.
| 7:31 am on Apr 14, 2005 (gmt 0)|
"I humbly think that if G is giving dupe content to www.domain.com and domain.com then that is a problem they must be aware of and need to fix. I am not changing anything on that for the moment."
This has been going on for years.
Chuck in virtual hosting as well.
| 8:30 am on Apr 14, 2005 (gmt 0)|
Chuck in virtual hosting? What do you mean by that
| 10:24 am on Apr 14, 2005 (gmt 0)|
I have been reading the posts with great intrest.
My site like many other went from 10,000 daily referes on the 22nd of March to less than 1500 on the 23rd March.
I have made a conserted over the last 8 months to add unique content to my site, writing all content myself adding relevant high quailty relevant links in a uniform manner so as to avoided any link spikes, i never exchange with spam e-mail link campaigns.
I can not like many of you see what i have done wrong, i have noticed today that when i do site:myurl google shows 27,000 results when i only have 10,000 pages. I can only check the first 100 which all are my urls with no duplicate content. I am trying to figure were the other 17,000 pages come from and if this is the reason i have been penalized. I must point out that i still retain a reasonable number of blue widget keyword placements, what i have lost is the smaller unique keywords which generated most my traffic. I cant figure this out as there is a lot less cometion for these keywords and i assumed wrongly this would give me some protection.
If any body has any idea i would be really greatful to here your thoughts.
| 10:35 am on Apr 14, 2005 (gmt 0)|
run the command
site:mysite.com without www at the fron if you see both www and non www entrys you need to mod rewrite so there is a 301 from the other variations to the one you want to use.
The reason you have more pages is probably because the above has occured
| 10:50 am on Apr 14, 2005 (gmt 0)|
Hi Thanks for your adive, i did as you said.
However with the site:url with out the www it still only brings back the www results.
| 10:57 am on Apr 14, 2005 (gmt 0)|
Try Copyscape (Google that up, its good.)
Use that on some internal pages that might have gotten scraped.
Also Google for short search phrases unique to your pages.
You might be surprised what shows up. If one or two sites seem to
have all your content, that should answer your questions. -Larry
| 11:22 am on Apr 14, 2005 (gmt 0)|
Ill give that a go know.
Please dont think i am stupid what is copyscape can you explain, sorry to be a pain.
| 11:32 am on Apr 14, 2005 (gmt 0)|
Its a dupe content checking program hosted online
| 11:32 am on Apr 14, 2005 (gmt 0)|
The others are from scraper sites, directories, and others linking to you. Mostly scrapers I suspect. We have done extensive research on this. We have been wondering if it had changed our neighborhood, due to those links, and lowered ranking on affected sites. Our Google traffic went from 90% to 18%, so we have been working around the clock on the problem.
Until April 4. A reformat of our pages beginning on March 30, proved worthwhile April 4, when the test pages recovered from pages 3 and deeper to top half of page 1, even #1 listing. We held that through the crawl on April 5 and again on April 6.
April 8 we had #1 ranking on a word we have nevered ranked higher than #30 on. Then, we saw sites who were indexed on April 12 and April 13, move ahead of us again, from our April 9 positions, leaving us in the #5 position, where we are still holding tight, with no new indexing of our site.
Interestingly, it was only sites indexed April 12 and 13 that rose above us, but not all sites indexed those days. Some indexed those days, were right below us, which means we out seo'd them on other points, most likely longevity, from looking at them.
That was on one keyword set. On Another, the reformat gave us #1, and we held it through the week. That keyword, we had once been in the #10 position, then lost out due to longevity to much older sites, and dropped to page 3. Then, we got lost in the shuffle and who knows, for many months. The reformat brought us back, higher than ever, and we are holding for a week.
As a note, the reformat was done for two reasons. One, we had lost all ranking at MSN since Dec 18. Two, we had been sliding and bouncing all over the place at Google. The reformat was done to accomplish higher ranking in both those engines, as we are holding almost all #1-10 in thousands of keywords at Yahoo. Yahoo wasn't an issue.
First round attempt, we slid miserably at Yahoo, and very quickly we went back to the drawing board, and created a new format that would address the reasons for the slide at Yahoo. The final reformat worked, correcting all problems in all the engines, with all showing #1-5 on the major keyword sets.
Conclusion from all this testing and reformatting was, it was not the neighborhood change, due to those scraper sites who come up when our url is entered into the search query.
We should be crawled again within the next couple days, and will be interested to see what happens then, at Google.
| 12:09 pm on Apr 14, 2005 (gmt 0)|
Nobody seems to be copying our site just a bunch of very poor search engines type site that links to us. No more than 35.
Could this be the cause.
| 12:27 pm on Apr 14, 2005 (gmt 0)|
Guys! Great news!
First, though, an important life lesson. My life has been crazed of late because of a huge number of personal and business things hitting me all at once. So I've not been reading these forums religiously. And I haven't had the time to think about the business problems that I should have been focused on. But no matter how crazy things get, don't stop reading these forums since broker_boy had my answer.
Had I been reading these forums religiously, I would have realized that Google sees [ourdomain.org...] and [ourdomain.org...] as the same. So we were being penalized for duplicate content on almost all our pages.
Thanks, broker_boy, for your suggestion to look up site:ourdomain.org and site:www.ourdomain.org. We are now 301ing ourdomain.org pages. I know this is kind of a basic SEO issue that has been talked about of late but it's easy to get lost if you don't read carefully.
It will take weeks to months, no doubt, to recover the business (just in time for the summer slump) but I feel much better already.
And remember, read these forums religiously!
| 12:37 pm on Apr 14, 2005 (gmt 0)|
if you have say a group of pages all the same template with same top navagation bar, same navigation bar same title and description in the html but with each page having different content, would they be classes as duplicate.
| 2:22 pm on Apr 14, 2005 (gmt 0)|
|Google sees [ourdomain.org...] and [ourdomain.org...] as the same. So we were being penalized for duplicate content on almost all our pages. |
Umm... It is probably/likely to be the case. However, persoanlly, I wouldn't rush to the street chanting "eureka, eureka..." on that observation alone. Seriously, this could be caused by a whole lot of things, so if I were you, I would continue to explore further just to err on the side of caution. May be it is just me....
I do hope a 301 redirect cures your traffic blues... :)
| This 233 message thread spans 8 pages: < < 233 ( 1 2  4 5 6 7 8 ) > > |