How does Google treat forum content?

[forumguy]

Hello,

I'm an old timer in the online world and have been a follower of WebmasterWorld for many years. I run several web communities using off the shelf software that most of us will have used at some point (vBulletin, XenForo, IPB).

Some of these communities are over 10 years old and have hundreds of thousands of threads. Until the end of 2012, we've always done quite well in search engines. I'm quite familiar with SEO and would like to think that they were optimised very well - without any rogue link building or other less than ideal techniques.

For some reason, since the back end of 2012, ALL of the forums I run have taken a big hit in traffic. First a big knock down in Nov '12, but since then there has been a gradual referral decline from Google. We're now at about 1/2 to 1/3 of the traffic on almost all of the communities we run.

I also work with other forum owners and the majority of them seem to have the same problem (9 others). This seems far more than a co-incidence!

Recently Google has also removed the ability to filter search results to discussion forums (from the "search tools" menu).

Is there a push from Google to lower the value of discussion forums and reduce their visibility on the net? I'm aware that badly run forums can be a spam magnet, but these are easily identified. Many forums are invaluable sources of information, and aside from my obvious bias, I actually often prefer seeing search results from forums as I find the information much more helpful (especially when it comes to tech/IT problems).

Top search results seem to include fewer discussion forums and more low-quality articles from the likes of the big adsense article farms (although to be fair, there does seem to have been a slight increase in quality on some of them!). The only exception seems to be sites based on the Stack Exchange platform, which seem to be the only discussion-type site I see doing very well (and deservedly so).

Are there any forum owners here using off-the-shelf forum software that could add anything to this discussion? I would especially be interested to hear from fellow large forum owners.

[thedonald123]

Has anybody been following the story of ask.metafilter and Matt Cutts. I'm pretty sure that ask.metafilter is similar to a forum and they were penalized in November 2012.
Barry Schwartz has a bunch of articles on it.

The date Danny Sullivan figured out with his post named On MetaFilter Being Penalized By Google: An Explainer was on November 17, 2012. It was one of those dates that Google would not confirm an update but one that I felt was an update, Google actually went as far as denying any update on November 16th or 17th.

As you may remember, MetaFilter got nailed by an unconfirmed update on November 17, 2012 that Google denied.
Google has now confirmed that update, 1.5 years later. Not only that, Google's Matt Cutts said on Twitter, in response to my story on Search Engine Land that they are working on updating that unconfirmed update from 1.5 years ago

It seems like this is the update that's being discussed in this thread?

[dethfire]

thedonald123, yes. I went from 298K visits on Nov 14th 2012 to 108K on Nov 17th 2012 and it slowly declined to an average of about 85K today.

[Selen]

It could be a good move from Google's strategic point of view. The only 'missing' link for Google was lack of their own content. In order to produce content, they need people to use Google forums, communities, and Google+. This content may then be a part of the 'answer/knowledge box' in search results.

Still, quality forums appear to be doing well (maybe if they were marked as 'Quality' by a human rater helps). It would be hard to imagine the Internet without quality forums and posters :). I personally like seeing forum results in search because they usually give a new perspective on the subject and often provide links to resources that would be hard to find on your own.

[vandelayweb]

I came across John Mueller talking about forums on this week's Google Webmaster Central Hangout (about the 1 hour 10 minute mark).

[youtube.com...]

He talked about being hyper vigilant with your content and making sure only the premier content gets indexed while noindexing the rest. He had other thoughts like noindexing new users (I have no idea how you would do that). What it sounds like is being a forum administrator is a major headache in this new panda world of ours. Anyway thought this might be helpful to get the opinion from one of the higher ups.

[nrep]

Well this is interesting. We were whacked with the Nov 2012 UGC update and our traffic graph looks almost identical to that of MetaFilter.

I'm excited to see what the result of refining this update will be, but I'm surprised that they've not given some general comments on what it relates to. From what I can tell it seems to hit UGC sites exclusively (primarily forums & Q&A), but mainly quality forums have taken huge traffic hits and only a handful (i.e. StackExchange) have had a huge rise (even if it is a great site).

[dethfire]

vandelayweb, sounds like they are waging war on forums. Such a shame and waste of resources.

[yafdecline]

My UGC site has seen a slight uptick since the middle of June, but I'm guessing it's related to clean-up of technical issues. This came after a drop following Panda 4.0.

What Mueller is recommending isn't a quick fix and sounds incredibly difficult for most forums to implement. I know of some forums that have a no-index setup for zero replies and have yet to see a recovery.

Google seems to be indexing thin content posts in their own forum from new users and they rank. Searched for "forums omitted results from first search."

I'm hopeful that whatever hit forums in November of 2012 will be reversed. With each week, month that passes, it seems less and less likely.

Is there a UGC filter, Panda filter or both that's hitting us all?

[netmeg]

I haven't run a forum myself in twenty years or so, but I've looked at a few, and I wonder if the typical forum architecture is incompatible with what Google wants as far as site architecture in 2014. Meaning, if someone came up with some kind of new architecture for forum software, maybe forums would do better in Google.

(Short version - maybe the problem isn't completely content or even UGC but the way they're put together)

I admit this is nothing but an idle thought. But some of the forum packages I've looked at were kind of awful.

[Planet13]

@ vandelayweb:

I came across John Mueller talking about forums on this week's Google Webmaster Central Hangout (about the 1 hour 10 minute mark).

[youtube.com...]
Thanks very much for the link. I would suggest everyone watch it, whether they run a forum or not.


@ dethfire:

vandelayweb, sounds like they are waging war on forums. Such a shame and waste of resources.

I don't know if they are waging war on forums, but it sounds like they are really turning up the sensitivity to the quality (or PERCEIVED quality) of the content on forums (and other sites).

We really have to see which sites GAINED significant traffic after that Nov. 17th update to get a better picture.

~~~~

I think the question is, "How will forum owners transform their content from 'useful' to 'the best content on the web' for a particular query?"
.

[edited by: Robert_Charlton at 7:54 pm (utc) on Jul 18, 2014]
[edit reason] added quote box, adjusted formatting, to clarify what's what in post [/edit]

[dethfire]

@netmeg I think this a problem for some free software and old software, but the new popular forum software like Xenforo and Wotlab Burning Board use modern techniques and architecture.

@Planet13 I think there is often a disconnect on what google says and what actually happens. We have reports of forum admins doing radical clean ups and mass no-indexing and with no results. This UGC algorithim is a total mystery and admins just end up making desperate attempts which can end up hurting their community more. After all, there are some very popular forums with literally hundreds of thousands of low quality threads and thin content and yet they are doing just fine.

[Planet13]

"After all, there are some very popular forums with literally hundreds of thousands of low quality threads and thin content and yet they are doing just fine."

Well, that might be part of it.

It may be the algo is detecting the popularity of a forum (either through link profiles or traffic referral info they obtained via one method or another) and using that as a significant factor in ranking.

When John Mueller says to curate the content of a forum carefully, it might not be implying that the algo is analyzing PURELY on-page content. It might mean that having better curated content is his (very indirect) way of saying that better-curated content will help generate more off-page authority signals, which are needed for UGC sites.

So to summarize, it MIGHT be that the update that affected forums was looking at off-page / off-site factors more than on-page / on-site factors, but that the way to improve those off-site factors is through better quality content on the site?

I really hope this helps in some way, but it is just a guess.

In the end, the question is, how do you become one of those very popular forums that is resilient to that algo change, too.

[dethfire]

I think you are right. In the end there are thousands of factors and algorithms in play. If a site ranks well for one very important factor, it could outweigh some minor factors it does poorly in.

[nrep]

I've noticed some upward movement in the past couple of days (for forum content) for the first time in years. Anyone else?

[cscgal]

I had twice the amount of expected organic traffic on July 19th and 20th, which was incredibly odd. Nothing unusual either before or after. Something like that never happened before.

[1script]

Another victim of the stealth Nov 17th, 2012 UGC update chiming in, for no particulate reason other than just to say "me too". Also, I didn't use free forum software and the URL structure of my site has been different from what most forum software has as a default. And yes, free forum software is usually ridiculously bad in terms of even basic SEO, but 11/07/2012 update seemed to go after UGC specifically.

BTW, there has been no recovery since then. The traffic is steady but still at the low Nov. 2012 levels, and never above what seems like some sort of a glass ceiling at approx 50% of the pre-Nov 2012 level.

[nrep]

Well, it looks like some UGC sites have recovered, which is interesting.

Here's a news article about a well known one on, which was hit in Nov 2012 and is now back to normal levels of traffic (https://www.seroundtable.com/google-metafilter-traffic-returns-18944.html). Interestingly, this only seems to have happened after it was brought to the attention of Google - no luck for the rest of us.