Problems with ranking and traffic in old established sites

Mod's note: Moved from the September 2016 Google Updates and SERPs thread [webmasterworld.com...] as it deserves a separate discussion...

lightnb posted...

@westcoast: what you described is exactly what we're seeing. We're an e-commerce with over 20 years online, and we always follow the rules and get no where. Every time we do what Google says, Google traffic drops. Every time we do nothing, Google traffic drop.

You are not the first to describe this, either. I have now heard of a number of circumstances like ours. The factor that seems to be the same in all of these "constant irrational downward ranking pressure" cases is sites with old AGE and large SIZE. It's my theory that there is some algorithmic bug or algorithmic interaction specific to Google which negatively effects sites in the 15-20 years old range.

Our site's #1 keyword which we held for 19 years suddenly gave way to a new website 2 months ago. This new site has nearly no backlinks and no content. The ONLY thing it has going for it is that its name is "KEYWORD.XYZ". This indicates to me that our site has been hit by massive algorithmic penalties, the type of which we have no insight into.

Our site's second most relevant keyword has slipped from #1 down to #22 over a period of 2 years on Google. Bing still has us #1, Yahoo has us at #2 (and the #1 site on Yahoo has legitimate argument to that spot). The algorithmic penalties are constant, and ranking pressure is steadily down.

My theories are:

1. Backlink complexity: Webmaster tools shows 836,729 backlinks to our site. The root domain has 118,279 backlinks across 1,988 domains. We have a widget (yes, links in it are NOFOLLOW) which account for 300,000 or so links.

120,000 backlinks are assigned to a website that cloned our site a year ago and inserted malware into their copies of our pages (why on earth Google is listing a hacked site that THEY identified as such and are counting backlinks from it when it has cloned our site is beyond me). That hacked site has been offline for months, but Google still retains their URLs, still retains the backlinks, etc. God knows what sort of penalty we are under for that. God know why Google ignored our every attempt at having them remove that site's ranks. Since Google is listing this hacked site's links as backlinks in WMTools, I have no confidence that they aren't penalizing the crap out of our site.

Anyway, over a period of 20 years you accumulate a crap load of spam links. We are in part an informational/educational site, and so a ton of spammers use links to us to try to make themselves look more real. i.e. they'll spam some allergy medication, and then link to a page on our site somewhat related to allergies. Year after year, it's just what happens. We collect a lot of stuff that is clearly not designed to advantage us.

Using one of those "link detox" type sites, I found 480 domains linking to us that looked either spammy or fishy or whatnot. Some are a little fishy, but may be considered good backlinks. The dillema is, do I disavow all or some of these? And if yes, which ones? And who knows how many more were missed because of incomplete backlink data? Can we hurt our site by disavowing these? Google says "yes. yes you can hurt your site with disavow". Disavow when you're talking about 1900+ linking domains is a complete nightmare.

It seems to me to be completely unfair (and increasingly, impractical because of scope) that WE are responsible for identifying 20 years of crap links that other people have linked to our site.

So who knows what sort of penalties Google has levied upon us for using and or not using disavows (I switch back between our 480 domain disavow list and none every 6 months or so because frankly neither seems to help and I keep hoping at some point Google will treat this stuff properly).

301 complexity: Old sites like ours have reorganized, moved stuff around, etc many times. We have extremely large numbers of 301s (all valid, with no more than 2 "hops"). It is my fear that Google has some bug in its 301 code that becomes problematic with large 301 landscapes. Our site has something like a million active 301 URLs. Add in all the 301s that translate domain.com to www.domain.com and other such combos, and we're looking at a few millon 301s. Is Google confident that massive number of valid, legacy 301s are being handled correctly? What about 301s that once existed and were taken down years ago? Are they causing issues today?

404 linking bug: As mentioned, we had 90,000 URLs pop up in WMTools recently. When you click on the "linking from" tab for these, they show URLs that have not contained links to this page for YEARS. In fact, some of our "linking from" URLs have NOT EXISTED for 18 years!

Google thus appears to have a memory of every link a 404'd page has EVER had, and does not update the existence or linking status of those pages. A true "real 404" is a page that still has a REAL, LIVE page linking to it. Those are the URLs a webmaster cares about.

So I am truly worried that Google is counting these "phantom linked from pages" as penalties in some way or another. Google has that data, and shows that data in WMTools. The number of historical 404s old/large sites accumulate over the years can be quite massive too, which is another reason I think this might be a google bug.

Content freshness run amok: We have a TON of content labelled with posted dates starting 1999 and going forward. We do our best to keep such content up-to-date, and have a good system in place for users to report inaccuracies. So, we keep stuff as accurate as possible. I wonder at times if Google has labelled our entire site as "unfresh" because of the large number of old timestamps on some of our posts. It appears that if Google sees 2 dates on a page (created/posted, last updated/modified), it only cares about the created/posted date. At least, that's the one that seems to pop up in the SERPS. I wonder if having old created dates slowly penalizes a site year after year after year. It might explain the constant downward pressure.

Anyway, just some ideas. It does seem to be that Google has some unintended algorithmic issue with some larger/older websites.

The most demoralizing aspect of this is that there is no way to lay out this issue or case to Google. If you get a manual penalty, at least you are told what it is, and have recourse to converse back with Google. Some of these algorithmic penalties, particularly when layered on top of each other and obfuscated seem to be even WORSE than manual penalties. The difference is there is no recourse with algorithmic penalties. There's no feedback, no way to say "hey, this penalty makes no sense". There's no feedback-corrective mechanism, so bugs and unintended consequences could last for years. Intentional black-hat (manual action) gives you a feedback mechanism, whereas following the rules and getting caught up in opaque algorithmic webs does not -- which seems a bit upside down.

<Google needs to take a look at how it> treats older websites with large amounts of legacy 301/404/usage/dated content/history data in your system. As systems have become more complex there is a greater chance for unintended consequences.

.

[edited by: Robert_Charlton at 6:07 pm (utc) on Sep 26, 2016]
[edit reason] moved from another location, fixed formatting, and some Charter issues [/edit]