Has Google demoted Wikipedia?

[aristotle]

Over the past couple of months I've noticed some shifts at the top of Google's results for several keyword terms, in which Wikipedia's ranking dropped a little. Here is a typical example:

Previously:
1. Wikipedia page
2. Big organization page
3. Government agency page
4. My page

Now:
1. Big organization page
2. Government agency page
3. Wikipedia page
4. My page

Of course the details of the rankings changes vary for different searches, but I've noticed several cases where Wikipedia's ranking has dropped. I think someone else here also mentioned noticing this not long ago, but I don't remember who or which thread. So I'm wondering how widespread this might be, or if it's restricted to just a few categories of searches.

P.S. -- Bing also appears to have demoted Wikipedia slightly as well, for some of the terms I watch.

[Samizdata]

Google has taken on a vendetta against Wikipedia and manually demoted it.

A very serious allegation, do share your evidence.

What other rational conclusion is there?

How about a universal search engine conspiracy? From the opening post:

Bing also appears to have demoted Wikipedia

...

[MrSavage]

Google and Bing don't need Wikipedia, they simply need to scrape all the information from Wikipedia. Is that a demotion? Maybe it's a favor. Afterall, more demotion, more scraping means that there will be less server loads for them and less donations required. I think Bing and Google should send thank you letters to the contributors each and every time that info is displayed on the search engine results pages. Something along the lines of, "thank you contributor for your information that is enabling us to keep our users on our pages viewing our ads and ecosystem longer and better than ever before".

[anallawalla]

At work I track a niche in the Australian finance sector and of a thousand or so generic keywords I monitor there are *no* Wikipedia entries (and rightly so). With a rare exception, I get near-100% Australian sites.

However, in my private searches - aviation disasters, historic events, places -- mainly "encyclopaedic" queries - the top result is usually Wikipedia (and rightly so). Yes, the KG is getting more visible, but it is not stopping Wikipedia from being #1.

[minnapple]

Someone else always wins when someone else loses.
It may be informative to hear about such changes, but celebrating another's loses is a poor use of time. Time is better spent on building winning strategies.

[EditorialGuy]

There's really no way to spin this in a positive light. It's legal, of course, since it's creative commons content. However, plundering it to the degree that you create the ONLY YoY traffic drop in their history, without compensation, is just greedy.

This thread is about Wikipedia's supposed "demotion." I think it's safe to say that, if Wikipedia pages now rank lower on average than they did they used to rank, Wikipedia's traffic should be down with or without the Knowledge Graph.

As for the question of why Wikipedia may be ranking lower, that's already been discussed. One possibility, which several of us have mentioned, is a change in how Google measures authority. Wikipedia isn't an authority on anything (it's a megasite that emphasizes breadth, as opposed to expertise on specific topics), so there's no reason why it should rank no. 1, 2, or 3 for queries that can be fulfilled with results from expert niche sites.

For what it's worth, I've seen other crowdsourced megasites (and brand-name generalist information sites, for that matter) lose their dominance for many or most of the queries that I watch. I view that as a good thing, not as a bad thing--for publishers, and also for searchers.

[keyplyr]

Wikipedia editors have been plagiarizing my intellectual property for years, sometimes being so cavalier as to not even reword it at all. For what ever reason, if their bubble is popping, it's long overdue IMO.

[IanCP]

Wikipedia editors have been plagiarizing my intellectual property for years

Or citing it within the copyright rules with a link to your site page as a credible source?

I am aware of a number of instances of my material being quoted as such on Wikipedia - which I deem as fair usage.

I've always taken the view if the Wikipedia reader wants more in depth detail, then they would follow the link back to the page on my site. Indeed many do.

I always consider it as another quid pro quo situation.

[keyplyr]

Or citing it within the copyright rules with a link to your site page as a credible source?

No, they do list my site as an authoritative source on a couple dozen pages, but that's not what I'm talking about.

This started years ago. To fill their pages with content their editors scraped my articles and put my copyrighted content on their pages. In some cases, that wiki page ended up outranking my own page for that same content. Over time, I have complained hundreds of times, sent dozens of cease & desist notices and have started legal proceedings numerous times. Their responses have been blatantly unaccountable. They either water-down my articles, switch around the wording or in some cases remove it altogether. This has become a dead horse since they seemingly will never stop.

Never once has a wiki editor asked my permission to use my property. I'm a college professor and understand very well the fair-use law as it applies to copyright. I'm not concerned with someone citing my work & using a couple sentences. What I'm describing goes far beyond that.

In all this time I have seen Wikipedia sit on top of almost every SERP, detouring traffic that other sites should have rightfully benefited. IMO it's about time they were dethroned, and I hope this demotion is real and continues across all topics.

[MrSavage]

Do those knowledge graph usages cite all the links and clutter/noise at the footer of the pages from which is scraped? Sure, sure. It's give and take. The value of having your content utilized on wikipedia provides reciprocal value? I guess here's to hoping the KG doesn't replace the rankings of the pages themselves. It's smoke and mirrors. It's funny if you think that your contributions were taken off your own site, put into a wikipedia page, and then Google and Bing taking the meat and posting that on their search results pages, thus leaving your reference link and credit somewhere down there on the bottom of the bird cage. The point is if or when the wikipedia pages start getting demoted, it means that the information is still needed, but the middle man is not needed (the middle man being the wikipedia pages themselves). Afterall, it's just about getting that information that everyone is so pleased at providing free of charge and in the spirit of non commercial use. A lot of the web becomes irrelevant thanks to the wikipedia connection to knowledge graphs and brains of Google Now, Cortana and Siri.

[rish3]

This thread is about Wikipedia's supposed "demotion." I think it's safe to say that, if Wikipedia pages now rank lower on average than they did they used to rank, Wikipedia's traffic should be down with or without the Knowledge Graph.

A strong possibility is the earlier suggestion that KG damaged it's user engagement, which then resulted in the demotion.

Panda sometimes works this way as well. Demote you for keywords that are relevant to your site, which screws engagement, as the remaining terms/traffic are less relevant...and here comes the second slap.

Or maybe your theory that Google found them the best thing since sliced bread for years...enough so that they've even lifted their content. And just after doing that, the algo realized "oh, well..you know what, it wasn't really *that* great."

[edited by: rish3 at 5:56 am (utc) on Mar 31, 2015]

[graeme_p]

When someone directly copied text from my site to Wikipedia it was spotted by a monitoring bot and reported to an editor who removed the plagiarized material. I am not saying that Wikipedia does not plagiarism, but that it not typical. Rewriting and a link back are more typical

My object is more subtle. My site that gets is a niche encyclopaedia that is written on the basis of industry knowledge and from looking at primary sources (e.g. academic research) so it is a mixture of a primary and secondary source. If Wikipedia is a secondary or tertiary source (or worse) so it does deserve to rank so high against specialist sites written by experts.

[Ralph_Slate]

The problem with Wikipedia is that they farm their content from sites they consider authoritative, but because of their size and breadth, they take over the authoritative spot in Google. They may not plagiarize other sites, but they certain consume those sites and regurgitate them.

I cannot complete with Wikipedia even when Wikipedia's page is nearly a stub and my page is well-rounded. They come in at #1, and I come in at #2.

[EditorialGuy]

Or maybe your theory that Google found them the best thing since sliced bread for years...enough so that they've even lifted their content. And just after doing that, the algo realized "oh, well..you know what, it wasn't really *that* great."

Google Search is built around an algorithm, not a directory. Algorithms change, and when they do, some sites slip in the rankings while others gain.

There may be a few people who think Wikipedia is entitled by historic precedent to be ranked forever in the no. 1 position for all informational search queries, but that doesn't mean Google is obligated to share their view.

[rish3]

Google Search is built around an algorithm, not a directory. Algorithms change, and when they do, some sites slip in the rankings while others gain.

Huh? When did I say anything about a directory?

There may be a few people who think Wikipedia is entitled by historic precedent to be ranked forever in the no. 1 position

That's not my line of reasoning. My personal opinion regarding Wikipedia doesn't really matter. What matters, imo, is the sequence of events.

Google ranked them well for years, took their content, then demoted them. I suspect the demotion is related to damaged user engagement, since much of their content is now presented on Google. You suspect it's all unrelated, and just a normal algo change.

Neither of us can prove our points. That sequence, though, does match a similar patterns uncovered around the leaked FTC documents. So perhaps there's a third possibility: It was an algo change, but one that was carefully tweaked and coached until the desired outcome was achieved :)

[EditorialGuy]

I suspect the demotion is related to damaged user engagement, since much of their content is now presented on Google.

But wouldn't Wikipedia's user engagement be better without the drive-by searchers who are just looking for the capital of North Dakota or how many home runs Babe Ruth hit during his baseball career? Wikipedia mostly publishes articles, not Answers.com-style nibbles, and without those casual jump-in-and-out searchers, its numbers for time on site, bounce back to search, etc. should be better than they were before Google, Bing, and other search engines began delivering facts and answers on their SERPs.

Not that it matters anyway: Evolution happens, tastes change, and there's no reason why any site (including Wikipedia) should expect the same rankings year after year.

ADDENDUM: Earlier, someone mentioned stubs. One of my biggest beefs about Google Search for many years was the presence of Wikipedia Stubs, empty keyword-targeted "Write a review" pages on sites like TripAdvisor, and so-called product review pages at CNet, ZDnet, etc. that contained nothing but price-comparison widgets. I'm not running across many of those stub pages in Google Search these days, and common sense would suggest that any decline in Google's indexing and ranking of stubs would have an impact on stub-heavy sites' rankings and search traffic.

[rish3]

But wouldn't Wikipedia's user engagement be better without the drive-by searchers who are just looking for the capital of North Dakota?

You keep repeating this mantra, and I understand that's your point of view on the effect the KG is having.

Personally, I think the impact is deeper and more complex than that. For example, if there's tons of KG content, and the user clicks on an organic Wikipedia entry, and sees the exact same content...might that not result in them mashing the back button? (Hey, this is the same exact stuff that's on the page I was just on)

I also suspect there are more than just these two scenarios in play. The effect of lifting a single site's content and pushing it into so many SERP results is certain to have ripple effects.

[Samizdata]

To me, this thread says more about webmasters than it does about Google or Wikipedia.

Join me a trip down memory lane: [webmasterworld.com...]

...

[thereign]

Couldn't agree more rish3, it is obvious.

[thereign]

EditorialGuy I doubt systematically lowering Wikipedia's CTR could possibly benefit its user engagement stats being that CTR is so important in Google's eyes. If a user is not "engaged enough" to click the SERP result then how could the end result be "better" user engagement? This goes against all logic. Better user engagement would move them up, not down. This is not some sign of Wikipedia "falling off" it is a targeted attack against a website that moves more people off Google SERPs than any other. It takes one click to view the knowledge graph page in full, which is the first result. It took one click to view the Wikipedia result in full, which WAS the first result. In both cases the user had to make one click, so it does not benefit the user, only Google. rish3's examplke of the multiple clicks required to get back to Wikipedia once clicking the knowledge graph shows this is just a form of traffic control and user manipulation.

[EditorialGuy]

I doubt systematically lowering Wikipedia's CTR could possibly benefit its user engagement stats being that CTR is so important in Google's eyes.

If a user is not "engaged enough" to click the SERP result then how could the end result be "better" user engagement?

First of all, we have no idea how important CTR is in Google's eyes.

Second, a site's "user engagement" is about what happens after the user has arrived on the site. So, if Wikipedia is attracting a lower percentage of "I just want to know the capital of North Dakota and then I'm outta here" searchers, its user-engagement statistics should improve.

Samizdata: Thanks for the nostalgia trip. Funny, isn't it, how Wikipedia has gone from being the pariah of Webmaster World to the abused orphan? :-)

[netmeg]

I was just thinking that same thing.

[rish3]

Funny, isn't it, how Wikipedia has gone from being the pariah of Webmaster World to the abused orphan?

That's an easy jab at the credibility of people like me, but not really related to the subject at hand.

The about-face in attitudes regarding Wikipedia that matters is Google's...not ours.

Zooming out a bit, it seems clear to me that Google's investors had a certain expectation about YoY growth, based on G's historical success. But then, internet viewership growth flattened out a bit (not flat, but flatter), and the move to mobile reduced ad revenues.

That's exactly when they started finding ways to increase the percentage of ad clicks vs organic clicks. This is more subtle than some of the other moves, but then I think the low hanging fruit is gone.

Maybe they should just start paying a dividend, and calm the investors, before everything above the fold for every query is an ad :)

[Wilburforce]

The about-face in attitudes regarding Wikipedia that matters is Google's

I'm not sure I would describe a drop of a couple of places as an "about-face in attitudes". It is much more likely that a marginal ranking factor has changed that either very slightly weakens Wikipedia's overall ranking, or equally slightly enhances the pages that - for some specific searches, not universally - have overtaken it.

[Nutterum]

Wikipedia suffers from the same symptoms the public libraries do. They are relevant until information accessibility is high enough for them not to be.

And never forget that Google describes themselves as information platform not a search one!

[rish3]

I'm not sure I would describe a drop of a couple of places as an "about-face in attitudes".

Heh. You're leaving out the context that it was preceded by a very notable drop in referral traffic for them...the first one ever in their history.

[Wilburforce]

a very notable drop in referral traffic

I think we should be careful not to conflate separate issues, especially where - as in this instance - the traffic changes you mention predate any widely-noticed ranking changes, and could therefore be a cause but not an effect of them.

My own view is that we are seeing a result of a combination of ranking factors that do not imply a sea-change in Google policy towards Wikipedia, and probably don't directly address Wikipedia at all.

The KG is another matter, and could account for a traffic drop (and subsequent loss of ranking based on popularity), but even here we need to consider the evidence and our conclusions carefully: it isn't clear why the KG would affect Wikipedia proportionately more than other information sources. Also, as the OP mentions, the ranking drop does not seem to be limited to Google.

Have you any specific data for Wikipedia's referred traffic? I ask because the stats you linked to earlier are for unique visitors, which is not the same thing (and probably doesn't actually mean unique visitors for that matter, as it almost certainly includes at least a substantial proportion of robot traffic).

[rish3]

I think we should be careful not to conflate separate issues

That was my exact thought on the side trip "down memory lane".

Have you any specific data for Wikipedia's referred traffic?

Sure: [imgur.com...]

Compiled from data using urls like:

[stats.wikimedia.org...]

Just replace month/year, and look for the paragraph that says "Google referred to our sites, through its services including search, maps, and Google Earth, XYZ page views per day, representing XYZ% of our external page requests."

Edit:

And regarding this:

and probably doesn't actually mean unique visitors for that matter, as it almost certainly includes at least a substantial proportion of robot traffic

If you're talking about the "Wikimedia Report Card" I posted an image of earlier, no, it doesn't. It is based on Comscore data. They have their warts, I'm sure, but I think they are probably sophisticated enough to filter out bots as well as anyone.