How does Google treat forum content?

[forumguy]

Hello,

I'm an old timer in the online world and have been a follower of WebmasterWorld for many years. I run several web communities using off the shelf software that most of us will have used at some point (vBulletin, XenForo, IPB).

Some of these communities are over 10 years old and have hundreds of thousands of threads. Until the end of 2012, we've always done quite well in search engines. I'm quite familiar with SEO and would like to think that they were optimised very well - without any rogue link building or other less than ideal techniques.

For some reason, since the back end of 2012, ALL of the forums I run have taken a big hit in traffic. First a big knock down in Nov '12, but since then there has been a gradual referral decline from Google. We're now at about 1/2 to 1/3 of the traffic on almost all of the communities we run.

I also work with other forum owners and the majority of them seem to have the same problem (9 others). This seems far more than a co-incidence!

Recently Google has also removed the ability to filter search results to discussion forums (from the "search tools" menu).

Is there a push from Google to lower the value of discussion forums and reduce their visibility on the net? I'm aware that badly run forums can be a spam magnet, but these are easily identified. Many forums are invaluable sources of information, and aside from my obvious bias, I actually often prefer seeing search results from forums as I find the information much more helpful (especially when it comes to tech/IT problems).

Top search results seem to include fewer discussion forums and more low-quality articles from the likes of the big adsense article farms (although to be fair, there does seem to have been a slight increase in quality on some of them!). The only exception seems to be sites based on the Stack Exchange platform, which seem to be the only discussion-type site I see doing very well (and deservedly so).

Are there any forum owners here using off-the-shelf forum software that could add anything to this discussion? I would especially be interested to hear from fellow large forum owners.

[itsjustme2]

Some of these communities are over 10 years old and have hundreds of thousands of threads. Until the end of 2012, we've always done quite well in search engines.

I actually got out of the fourm scene over ten years ago. Just was not my thing. However, your story sounds familiar to the many very good forum webmasters I've known for fifteen years.

Absent statistics, it's hard for me to tell how aggregate forum traffic has changed over time, making these remarks hypothetical only.

You rightly point out a potential spam problem with forums, but given the fact that Google finds potential spam problems everywhere it looks, the potential spam problems can not be the alpha and omega of any hypothetical devaluation of forum traffic resulting from algo changes.

It's more than a reasonable hypothesis that google conceptualises forums much the same way they conceptualise images.

Over time, google realised that they could keep visitors on their site by increasing the number of pictures they show per page of their image results, and then adding the addition hurdle of a second google lightbox.

The odds of any image in google images being clicked through from the user to the originating site of the image decreased dramatically, thereby changing the aggregate distribution statistics.

Using a tad bit of hyperbole, google images now guarantees Google a distribution of one million page views and one page view for the site originating the image (at one time my images stats, impressions versus click throughs, actually look pretty close to those numbers).

With forums, it's no secret that Google wants the world to sit in its own forum (google plus) every day, all day. Devaluing other forums in their serps represents one strategy for doing so. Individuals looking for information via search, will no longer be sent to forum sites that compete with google's fourm site.

Again, it's merely an hypothesis...nonetheless, it does seem to fit the pattern of the google business model incrementally valuing their products, images, videos, forums, over alternatives.

[paulfitz]

I experienced the same thing, took a hit to traffic across my forums in mid 2013.

I've been working on SEO since then and no noticeable improvement, they seem to have settled on about 30% of the traffic they used to get.

As you mentioned, it's a real shame because the forums often have the best content out there and the serps are being replace with junk a lot of the time.

One of the worst things I am finding is that my adsense revenue is being relentlessly shaved by Google for the forums. 95% of the traffic to the forums is from Google search engine, so real users looking for solutions.

Yet google will shave as much as 75% off the adsense clicks. Some days I look at a site and it will be sitting on $25 for the day, then suddenly, bam, Adsense shaves off some clicks and it's back to $8.

I have come to hate Google and their changes are driving me out of the game. The double whammy of traffic hits, then more revenue hits from Google is destroying my forums.

[EditorialGuy]

I like search results that direct me to forum pages with multiple posts from the same thread (such as Webmaster World pages), but I hate it when Google sends me to a forum where I see only one post (even if it's just a sentence or two) and have to click my way backward or forward to read the entire thread. Is it possible that such "thin content" forum pages have made Google view forums less favorably in general?

[forumguy]

With forums, it's no secret that Google wants the world to sit in its own forum (google plus) every day, all day. Devaluing other forums in their serps represents one strategy for doing so. Individuals looking for information via search, will no longer be sent to forum sites that compete with google's fourm site.

Yes, that's what I wondered too. They've really nobbled image referrals (although I suspect from an end user perspective, they like it, however I'd consider it unfair!).

I like search results that direct me to forum pages with multiple posts from the same thread (such as Webmaster World pages), but I hate it when Google sends me to a forum where I see only one post (even if it's just a sentence or two) and have to click my way backward or forward to read the entire thread. Is it possible that such "thin content" forum pages have made Google view forums less favorably in general?

Most forums shouldn't do this, but vBulletin and some other scripts do have individual post pages that sometimes get indexed (unless you're running VBSEO or have a decent robots.txt). FWIW, I've not got any of these thin pages indexed and prevent them from being displayed at all on my vBulletin sites. There are a handful of threads that I'm sure would be considered thin content, as with any UGC site - but no bulk pages that would be considered thin.

-------

A quick browse of the Google webmasters "groups" page shows that there are quite a few large forums experiencing similar issues. It will be a sad day indeed if UGC is marginalised like this!

[forumguy]

I've been trawling through quantcast profiles (I just searched for "forums") and tried to see if I can spot anything that may help. It looks like the tech forums have taken the worst hit by far, as many of those forums seem to be in rapid decline in the past year. Does this tally with anyone else?

[aristotle]

EditorialGuy wrote
but I hate it when Google sends me to a forum where I see only one post (even if it's just a sentence or two)

Why would Google ever send you to such a page in the first place? Is there something wrong with their ranking algorithm?

[martinibuster]

I have been testing this and Google seems to treat forum content (speaking of ranking only) similarly to any other content. Date of post doesn't make a difference as older evergreeen content can still rank well. I keep a record of ranked phrases, canaries for the coal mine, that I check periodically and there haven't been any recent changes.

Check your outbound links. Google penalized a Sprint forum last year because of spam posts, resulting in loss of traffic.

Other issues may be that more people have moved into your niche and ate your lunch with better backlinks, better content, better titles. I know I ate some forum lunches with a site I launched last year.

Another issue may be that the discussions may not be optimized enough to be competitive. Hard to rank for anything when 20-30% of site titles resemble "Helpppppp!"

Some forum software like vBulletin have had repeated security breaches.

[forumguy]

We've nofollowed all user posted outbound links on the forums and are running a mix of forum software. Likewise, I don't think it's a problem with optimisation - as it's affecting out sites and competitors equally. Our traffic patterns seem to mirror each other.

I suppose what I'm trying to say is that it's not something exclusive to our sites or to a handful, but seems to be the direction that many forums are suffering from (including sites I'd consider to be leaders and ones that are less than good).

It's interesting that you've mentioned the sprint forum (SU?), as that is one of the sites that closely resembles the traffic pattern I'm referring to. Some of the other sites in their network have similar patterns too, starting with the Nov 2012 date.

[martinibuster]

Keeping with the topic of how Google treats forum content, perhaps it might be the medium (forum software) or an editorial decision to lower the trust of UGC. But I have not experienced that. My experience will likely be different from the average community manager though. Perhaps it's not the medium or the content that's the problem but the content topic. Have you checked Google Trends to review what the trajectory of your topic is, flat, up or trending down?

Re Sprint, here's a post from SERoundtable [seroundtable.com] discussing Sprints issues with a spam related penalty.

Maybe we're not talking about the same sprint forum? The one I linked to below is not really a model of good SEO. So probably not a good example for Google excluding forum content from rankings.

[community.sprint.com...]

[yafdecline]

I've seen a similar pattern with my sites, all of which have forums. My largest site has been hit the hardest, losing roughly 70 percent of traffic. The decline started in late 2012. In April, 2013, the trend became increasingly noticeable, with a slight drop around the May 23rd release of Penguin. Outside of a few surges thanks to product releases relevant to my forum topic, the decline has been slow, gradual and painful. Over the past 8 months, I've tried everything possible. I've deleted thousands of posts which might be deemed as 'thin content'. I've removed any and all indexing of member profiles, image based threads and blog posts that might be considered thin.

As for Penguin, I did a ton of backlink research where I find a company who had been creating crappy backlinks to what seemed like an innocent forum post. The user in question was asking about a service. The company built all kinds of shady links. The post was deleted last summer and my disavow updated. I've mass deleted any forum users with zero posts and profile links. I figure, this will break any sort of linkspam coming back.

In April of last year, we received an uptick in backlinks from a scraper. It seems they are using software that merges posts from multiple forums. So now their content might have the same post title, but it's using a mix of posts (often not related), but there's more on the page. This site has 2 million pages indexed and they went online in April 2013. I'm not sure if this is hurting the site.

As with the OP, all links outbound are no-follow. I've also taken the advice of John Mueller who recommends focusing on your niche. As such, all Off Topic discussions are also set to no-index. All affiliate links are no-follow. I've also used Screaming Frog to do a massive clean up. Older forums tend to have lots of broken links outbound. I had a ton of 404 images from photo sharing sites, etc. All fixed.

The result of my work has seen a drastic reduction in indexation, which I take as a good thing. 404s in GWT were off the charts, probably 25k or so. Those routinely go to zero nightly, but Google will serve up old ones on occasion.

As for my reward for this clean up? Now that the holidays are over, I'm back on the natural decline. I may move the forums to a subdomain, hoping that it might boost some of my blog articles. Honestly, I'm out of ideas. I've gone from having a highly successfully site to on the verge of looking for work.

[forumguy]

Have you checked Google Trends to review what the trajectory of your topic is, flat, up or trending down?

This is across many niches, but the majority of ones I've looked at are tech / programming / PC related ones, which are trending highly. The threads are all pretty much on topic, although most do have a small off-topic section.

Maybe we're not talking about the same sprint forum?

That's not the same one as the one I had in mind - I've not seen that forum software before (it doesn't seem ideal!).

-------

yafdecline, it seems we have taken a similar approach. May I ask, did the trouble start around 17th Nov 2012? For many sites I own or track, this is the date that problems start (just before Panda #22).

Also, are you in a tech-related niche?

[yafdecline]

I saw a drop on November 20th, so around Panda #22. It went back up, likely seasonal and held steady through April 1st. Lost about 15 percent by May 22nd and took another 10 percent hit on Penguin 2.0. I've never had issues w/ Penguin in the past. Initially, I thought my problem was specific to Penguin, a slight algo hit from this one spammer and my dabbling with directories years back. I never received a manual penalty, but went all out creating my disavow, just to be on the safe side.

As the decline continued, it became evident from my research that it was either Panda or something else. I had thousands of 404s, the result of years of UGC and things breaking naturally.

Adding insult to injury, I've also scaled back on all of my advertising, thinking that could be having an adverse affect.

Each day it seems as if they take a little back. If there is a spike, they quickly bring me below what becomes a glass ceiling. The traffic is also crap. There are a couple of forum posts that are generating a good portion of the traffic, all of which have high bounce rates. Time spent on the page is high, but the decrease in traffic is making it difficult to lower my bounce rate. I realize it's only one indicator, but if I had my old traffic back, my site vital signs would be more to Google's liking. Over the course of several months, the impact has become devastating.

I think older, established forums are harder to fix. There's so much content. Even when you fix things, it's taking Google a long time to circle back and take note of the changes. Without knowing exactly what's wrong, I'm sure some of the changes weren't needed, pushing the site further into a tailspin. I've gone from 60k daily to 13k.

I'm in the tech sector.

[martinibuster]

PC related topics trending upwards? Really? No offense but I don't believe PC related topics are trending opposite. The opposite is true.

I just checked a variety of pc components like video cards, windows, windows 8 and windows 7. Windows 8 is peaking and starting to go down. All the rest are trending down. Even antivirus is trending down in a big way. Do youself a favor and visit Google Trends to check the actual trends for your niche:

[google.com...]

This reminds me of a tech niche from six years ago where someone was alarmed because traffic was dropping at 15% per year. Turns out less people were using the product and demand for that niche was dropping, replaced by a new product that disrupted the market.

Similarly PC is circling the drain, replaced by tablets and smart phone devices. But even that is fragmented. Things change and one has to not only keep up with trends but jump ahead. I haven't always done that and have had my traffic lunch eaten by upstarts that covered a narrow niche in a more thorough manner.

[yafdecline]

My niche is mobile. Specific term is stable and Google predicts growth. This plague isn't specific to tech, as noted in this thread.
[webmasterworld.com...]

[netmeg]

Tech? Mobile? Not much competition there...

[Mentat]

Same problem here..
Forum content is devalued.

First hit was 2-3 years ago, when Google got in love with Wordpress/blogspot blogs. Now the love is dead.

I have many threads with multiple pages, but the content is not indexed or does not rank at all.

My cache system is resetting each month and I can see that only 20% of the threads are visited. We are talking about 80 000 UV/day (it was 140 000 2 years ago).

Also, get a detailed robots.txt and disable "lofi versions" if you have it.
Robots:
- profiles
- put signature invisible for guests/google bot
- print pages
- calendar
- stats
- help
- attach
- subscribe
- search
etc

In my case, using the disavow file got me a bump up of about 10 000 visitors.
Some idiots created a network of "directories" with a lot of links to me.

All my outbound links are nofollow since... ever.

[yafdecline]

On a smaller site, I moved the forums to a subdomain. As I result, I've seen a boost in traffic. This could be temporary. On my main site, I'm contemplating moving the forums to a subdomain and curious if anyone had any experience good or bad after making a move. I've got high quality original content on the root, so if anything, I'm hoping this would improve rankings for those articles which may have been affected by thin content in the forums. I had planned to make this move at some point, but wanted to hold out for an actual recovery. A temporary bump might have me waiting for the other shoe to drop. Having taken a year plus of kicks to the gut, I'm not too keen on feeling more pain.

I've also heard this may trigger a quicker analysis of the site, which again hope is Googlebot will recognize improvements that have been made.

Thoughts?

[Mudley]

I am a long time forums SEO, and here is what I've observed.

The drop in traffic that I have seen mostly came from changes in the way people search, and the way google interprets the intent of the search.

5+ years ago it was easy for forum topics to rank for a variety of long tail phrases. It was the perfect mix of fresh content that is updated and filled with a variety of natural language. A thread could be a user selling a maxtor hard drive with 6 months left on the warranty, and it would rank and earn traffic for terms like "maxtor warranty information" and other things that are clearly not a match of user intent.

Google has gotten a lot better at this in the recent years.

I think social media has a lot to do with it as well. A lot of forums are shriveling up and slowing down. Less new users, less new content, and less overall engagement.

[thedonald123]

On my main site, I'm contemplating moving the forums to a subdomain and curious if anyone had any experience good or bad after making a move.

Definitely, I recommend moving your forum to a subdomain if you have quality non-forum content on the root.

I am following this discussion with some interest because I have a main site and a forum on a subdomain both contain similiar content. The main site is doing wonderful in Google but the forum is not and never has since 2007. It's always been a bit of a mystery to me why my forum has always been so unsuccessful from an SEO perspective. .
Regardless, moving the forum to a subdomain should let your other content stand on its own without whatever negative effects the forum is causing.

[iref001]

I have wasted hundreds of hours, trying every technique under the sun to get my traffic back on track but nothing seems to work. I lost almost 35% traffic after Google's late September update. I run a big forum with almost 300,000 pages and 50,000+ active members.

Since the last couple of months, 10-20 page websites have taken over and now the crappiest sites with duplicate content are enjoying the rankings. My site is loaded with fresh content everyday but somehow failing to get more traffic.

I was shocked to read the first post as it 100% confirms that Google is playing dirty.
I better stop giving myself sleepless nights and focu my attention elsewhere.

Despite the fact that I am the only website sending 100% traffic to Adsense, Google is not bothered. The top 10 sites dont send a dime to Google and enjoy better traffic. Maybe because, they are so bad that people prefer to click on adwords.

I am glad I came across this post.

[Mentat]

I've done some research.
The problem seems to be that google is accelerating the freshness madness.


My forum is since 2000.

If I search for a specific topic title page pre 2007, I can't find the page!
I get a lot of unrelated pages, but not my page.
The page is indexed. I can see the result if I add my site name to the query.

So, the situation seems to be that OLD content = 0 value.
Same with evergreen content.

Also, a lot of sites linking to me are now dead => not good for ranking in an era when all the new links are nofollow (fear of Penguin).

[netmeg]

If I search for a specific topic title page pre 2007, I can't find the page!

I don't know what your topic is, but I know that as a searcher there are very few topics where a post from pre-2007 would be of use to me. In fact I usually adjust the search to only look within the last year (or month).

[Planet13]

For what it's worth...

I don't track searches systematically the way matinibuster does (but I might start, it's a great idea), but I have noticed that when I search for topics where forums used to rank well, it seems more and more youtube videos are appearing higher and higher in the rankings.

I know this isn't earth shattering news, but it is something to think about... especially if you are thinking of "moving on" to different things.

Also, as a personal aside, I am starting to be influenced in my search tendencies by author photos and the number of google circles someone is in. I try NOT to let that influence my decision on which link to click in the search results, but I would be lying if I said it didn't.

And if I do it, I am sure others do it as well.

I don't know if forums can easily get their author / google profile photo and google circles in the SERPs or not. But it seems like the direction google is headed, so it might be a good idea to see if there is an option for you to explore there.

[dethfire]

Sorry for the necro, but this is interesting because my 4.5m post forum has also had a slow decline for the past two years. I've also tried hundreds of improvements with no reward than further decline.

This tells me it's either something google has decided and there is nothing we can do or people have changed how they search and where they search. I think it's a little of both. Something forums must consider to remain relevant in the future.

If anyone has any more insights since this topic was active I'd love to hear it.

[Planet13]

Purely speculation:

Does the increase in mobile users have anything to do with it?

Are the sites mobile friendly?

Are the topics covered in youtube videos?

~~~
I still see yahoo answers (or yahoo voices or whatever it is called) ranking well for a lot of my personal search queries. I also see lots of forums ranking #1 for other (related to my vehicle or my mobile communication apparatus).

I wish I had an idea to offer.

[dethfire]

Planet, I have seen mobile actively go from 10% to 25% in two years. I did add a nice mobile skin roughly a year ago and no change.

[cscgal]

Large tech forum here. You should all know who I am :) *wink*

Took a massive hit in November 2012 just before Panda #22. Down to about 30% of our previous traffic. Feel free to pick my brain.

[dethfire]

cscgal my stats also show exactly that. I was high flying 2012 fall and then after Thanksgiving I crashed and continued to decline for a couple years.

Considering the amount of similar stories it must be something Google triggered.

[vandelayweb]

Interesting topic. I just plugged webmasterworld.com into SEMRush. Looking at its all time traffic number since 2010, its seen a decline of 111k in January of 2010 to 33k last month. It had a high of 179k (March/April 2011) and a low of 12k (November 2012). I think we'd all agree that this is a high quality forum. Granted, you can't surmise anything by looking at one site, but I think there is something to this.

I wonder if the swell of social media is drawing people away from forums and thus Google discounting their value in the SERPs?