Massive drop in traffic, August 2012

[hitchhiker]

Our site is Q&A, in a specialised field. We use our own custom software, it's similar to stack. Our bounce rate is 55%, which is good for our industry.

Over the last 3 weeks we've lost about 30,000 people a day.

What has changed in the last 3 weeks, what are the new signals? We grew throughout all the panda/penguin stuff.

We don't have VCs or any other backing, this is a real project done by real people. We only run adsense to support it (down 50%).

We've never used any blackhat techniques or tricks. We've followed all suggestions by Cutts and crew since the googleguy days.

We don't track /analyse / focus on SEO / focus on links / we simply focus on content and inovation- it's paid off until this month.

Any idea what algo we might have tripped? Looks like it's time to get back into SEO :(

[mslina2002]

Is there a particular date that you noticed a drop starting or has this all been gradual?

[rango]

First thing is to have a long hard look and see if there are any technical issues that might have caused the drop. Be sure your pages are all reachable by Google, etc. You want to rule that out before you look to the algorithm being the problem

Then have a read of this thread before making a lot of changes [webmasterworld.com...]

If the drop was on a single day, then you'd be looking at Panda as a possible culprit. Last update was July 24th, so check if that roughly coincides. Unfortunately just focussing on content / innovation, etc. doesn't mean they won't sting you.

[netmeg]

On your Q&A site, are the As fairly lengthy answers that go into topics in depth? you have a significant number of one or two line As? Have you tried plugging your Qs into Google to see if anyone else is replacing you, or even Google itself?

Last week when asked about Google providing answers itself instead of sending people to sites, Matt Cutts made comments to the effect that something that could be answered in three seconds, Google may go ahead and provide the information themselves. That websites need to look to how they are adding value to the answers.

Just a thought.

[hitchhiker]

@mslina2002 - steep but gradual, millions of pages - everything google-wise rolls like a slow wave normally.

@rango - yep, june 23-25th saw a 20,000 peson loss, didnt notice it at first, but it was the first largish drop, then it sunk slowly after that till yesterday. No technical issues, but I'm guessing page-speed might be a factor (10s) - though the user can interact with a full page within 400ms -- A lot of extra stuff happens before page-load - i'm changing that to post-load now. I'll read the thread - cheers.

@netmeg - nope, we're absolutely the exact opposite of that - we have experts who are unavailable elsewhere.. our content is 80% 'golden' - but yeah, we're seeing competitors with keyword bulked-up pages all over us. Normal, we never pay much attention to that


--- update: yesterday the slump has stopped? (I hope) - total loss around 35k a day - I'll take into consideration these suggestions, 99% i'll be back on track in a few weeks (best to be blatently arrogant in the face of these things :))

Cheers for your time!

[hitchhiker]

@ rango - re: ranking randomisation - good read (thanks)

if that's it then: We've obviously not blackhat, we've been around forever, AND delivered quality for years and years to get here, we are not gaming the system at all.. there are only a few thousand sites our size.. I DOUBT any of us are dumb enough to start bh'ing anything. It wouldn't make sense.

G - I just want to focus on content - let me.

[rango]

@hitchhiker, considering the dates and the quick drop I'm afraid Panda really is the most likely explanation. In which case, welcome to hell :(

My personal advice - first do a technical SEO once-through; ensure you don't have pages doubled up in the index, etc. Then, all that being in order, focus like hell on finding ways of turning as many as possible of your New Visitors into Returning Visitors.

There's lots and lots of reading on Panda out there; many of it conflicting; much of it pure speculation. My main take-away is to focus on user-experience as much as possible rather than the traditional SEO stuff like title tags, keyword density, etc. Plus, if you go tweaking your title tags too much in response to this, you could find yourself in even deeper trouble (as per that referred thread).

[mslina2002]

june 23-25th saw a 20,000 peson loss,

Yep, you have been hit with the Pandasitus 2.8
That came out June 25.

Our site is Q&A, in a specialised field. We use our own custom software, it's similar to stack.

So it sounds like you have lots of UGC so have you checked some of those pages that have a question posted, but no answers given? Likely candidate for thin content there.

What about user registration pages? Are the noindex, followed or do you allow members to have their pages indexed?

From what you describe, your site reminds me a little of the daniweb site which is also Q&A type format so you could read up about her site: [webmasterworld.com...] where apparently she recovered from Panda.

However I just had a look seem traffic is gone a bit down since a few months ago.

[hitchhiker]

@mslina2002 - Daniweb - interesting stuff, i'll take a look.

Re: reg pages: We only index high quality pages, at one point 4 or 5 years ago, google was trying to index around 2m pages a day - so we cut them off completely. We just index the main stuff now.

I wont stop indexing no-reply/no-answer questions, regardless of G (only a very small percent, as always with Q&A).

I think i've decided to 'just ignore it, and push on' -- in 11 years we've never benefited from analysing this kind of thing too much.

I'm clinging to what googleguy/cutts said back in the day about 'quality' winning out over everything. That strategy makes sense, regardless of these minor swings. I can't predict what G will do, or wants, or whether it will survive itself - I can only adhere to the idea that good content will eventually win out. With our without G (naive perhaps)

[Robert Charlton]

No technical issues, but I'm guessing page-speed might be a factor (10s) - though the user can interact with a full page within 400ms

10s is way too slow. While you don't mention specifically what you're measuring, I wouldn't be surprised if that's your problem. Check what Google measures.

We generally don't discuss tools here, but we did make an exception with this thread. It lists two site speed tools you should consider trying, which will give you some feedback and recommend fixes...

Favorite SEO Tools
http://www.webmasterworld.com/google/4367011.htm [webmasterworld.com]

For some background on where 10s would put you, compare your figures with what other members are seeing...

Site speed as reported by Google, what's the distribution?
http://www.webmasterworld.com/google/4434380.htm [webmasterworld.com]

[edited by: tedster at 1:55 pm (utc) on Aug 23, 2012]
[edit reason] fixed link [/edit]

[netmeg]

I wont stop indexing no-reply/no-answer questions, regardless of G (only a very small percent, as always with Q&A).

I'm clinging to what googleguy/cutts said back in the day about 'quality' winning out over everything.

No reply / no answer question pages being indexed are not going to pass the quality sniff test. (If I were a user and landed on one, I would be pretty steamed)

[rango]

No reply / no answer question pages being indexed are not going to pass the quality sniff test. (If I were a user and landed on one, I would be pretty steamed)

Interesting. We allow these to be indexed also, but only for threads that are less than a year old. My main concern was that if I noindex them to start off with and then that changes, Google might have a hard time picking it up. Or at least we might miss the freshness boost for those.

But then, recently it seems sitemaps are pretty good at forcing another look-in.

[Dymero]

netmeg, seem to me that it's downright insane that a publisher should have to spend time removing noindex tags manually (which would be impossible in the case of hitchhiker's site), or waste server resources to do it automatically. If Google doesn't think a page is going to help a searcher find an answer, it shouldn't be as highly ranked.

Here's an example of Google trying to apply a one-size-fits-all solution to a web that isn't one-size-fits-all.

hitchhiker, I see you said you have Adsense. How are your ads laid out? I know that Google is actually lowering rankings of pages where ad placement is too top-heavy (above the fold), or is otherwise making the actual content difficult to find. You may have to relocate them, even if Adsense placement suggestions seem to indicate these positions are okay.

[netmeg]

Suit yourself.

I have event sites with several thousand events, and we have programmed them to NOINDEX any taxonomy pages that don't have active upcoming events*, because blank pages are not a good user experience.

How do you think a user feels if he is searching for an answer to something and comes up with your blank page that lists a question with no answer? I see this a lot (increasingly on page 2-9, by the way), and it pisses me off no end when I do. If it happens enough, I avoid that site whenever I see it in the SERPs, because I can't trust it.

If you think it's an imposition to remove low quality pages, that's certainly your prerogative. But I would be willing to bet Google pays attention. (And if you're running AdSense on them, by the way, it's against TOS to run it on blank pages as well. And I'm pretty sure just having an unanswered question would qualify as lack of content)


* None of these sites have ever been hit by an algo change so far, and traffic was almost three times higher this year, FWIW

[edited by: netmeg at 1:54 pm (utc) on Aug 23, 2012]

[scooterdude]

@Robert Charlton

The second link goes to the wrong thread I think [thanks - it's fixed now]

I think I also have a speed problem if 10s is way too slow,

My sites are web apps and I finally figured out that all the comparable sites are mounted on high end dedicated servers, some doing sub 2s page loads for cpu intensive , media heavy pages

I am on vps, and its disappointing

[edited by: tedster at 1:57 pm (utc) on Aug 23, 2012]
[edit reason] fixed the link [/edit]

[Shaddows]

seem to me that it's downright insane that a publisher should have to spend time removing noindex tags manually (which would be impossible in the case of hitchhiker's site), or waste server resources to do it automatically. If Google doesn't think a page is going to help a searcher find an answer, it shouldn't be as highly ranked

Google wants to serve the searcher. It does not have infinite resources. If your site is an unsatisfactory destination on too many occasions, Google may well decide to score your whole domain a little lower.

So it's up to you. Self-manage, or let Google make the decision for you.

[zehrila]

Even if you believe your site is Quality and had been a success throughout or has an impeccable record, i am afraid to say Google at same time is not the same old. They are changing their perception of good content, they are in for the money and they are your competitor. They are going to become Yahoo answers and answer to every thing where they see they can help users.

I recommend, take a look at ground realities and change, shift your website according to new trends. If there are thin or shallow page which are not adding up value, it might be a good idea to noindex them, check for internal issues, such as site structure, duplicate urls, how many supplimental pages are auto created by cms etc, how many ads are above the fold, navigation and all that.

Change is the only thing which is permanent, if you refuse to change and adopt to new strategies, it will get harder.

[1script]

hitchhiker, I don't have an answer but perhaps another data point for you. I also run a Q&A type site (much smaller operation judging by your numbers) and it also lost traffic precipitously on Aug 1st. So, there may actually be something about the type of a site that is.

Many people here pointed out the issue with pages that contain a question but not an answer. I do not share netmeg's righteous disgust with such pages. For one thing, they DO have content. Some of the questions we get asked are so esoteric, it makes me wonder how the person expects any kind of answer at all. Even then, someone else looking for a similar issue may get clues from another person's question or at least know that he's not imagining things and someone else in the world is looking for a solution as well. Often times the two people looking for a solution get connected together on such page, even though it does not contain an outright answer.

Still, I have analyzed which of my pages are getting Google traffic (through serious data collecting and log processing, not anecdotally) and I can tell you that Google already has a very strong preference for pages that do contain answers. It may be preference for pages with more content in general or they may able to analyze the structure of the page and see/count the answers (they are formatted slightly differently from the question in my system, also custom-coded). Regardless of how they do it, a page with an answer is already 10+ times more likely to get a visit from a Google's SERP than one without an answer.

So, I am also considering no-indexing pages without answers despite my personal conviction that they do have some value. The only thing that's stopping me so far is that I don't want them to be buried forever in case they do get answers in the future. Does anyone have experience they can share with indexing pages that were previously no-indexed?

One more thing: a behemoth-sized competitor in the field is choke-full of no-answer question pages. They do make it look more like a search page, actually:" we don't have an answer but perhaps you'd be interested in these pages?" - the list follows. You open those and most don't have answers themselves and just send you down the list. I'm anxious to see their August traffic figures on Compete - would be interesting to see how they weathered whatever change we got caught in...

[netmeg]

Does anyone have experience they can share with indexing pages that were previously no-indexed?

I do it all the time with events that get cancelled or postponed and resurrected. Even on my sites that are brand new this year I am not having a problem getting them back in (it's actually more of a problem getting them OUT after they're expired - Google hangs on to an old page like nobody's business)

a behemoth-sized competitor in the field is choke-full of no-answer question pages. They do make it look more like a search page, actually:" we don't have an answer but perhaps you'd be interested in these pages?" - the list follows.

That's marginally more useful, but a pretty slim margin. Again I go back to the user experience. But whatever, the big guys have different rules. That's true everywhere. I could spend time bemoaning the fact that Google doesn't play fair, or I could work to keep my site in good shape and my visitors happy. I try to opt for the latter. I'm not getting any younger.

[Sand]

If you have a question and answer site and you want it to do well in Google, then the best thing to do is moderate the hell out of it to ensure quality. See Stack Overflow for a good example of how to do it right.

As for questions with no answers: think about it from a searcher's perspective. If I'm trying to solve a problem, I want an answer, not confirmation that someone else at one point had the same question I do.

[rango]

Points taken, I'll make it so our question only threads don't get indexed at all. I'd rather be safe than sorry and that traffic really isn't all that valuable in any case.

In fairness though, it's not like Google couldn't just ignore those results. They are capable enough to display the "1 Author - 1 Post" in the description, so if they really thought threads with just questions were a bad result they could just exclude it.

[Robert Charlton]

I am on vps, and its disappointing

The server rarely has that much to do with it. It's the size of the page, image sizes, the scripts you run, and the number of http calls. I have a client who was running on a temp server during his development phase so he wanted to blame slow performance on that, but it was really numerous other issues. We fixed those, and now, even with the development server, his pages are now very fast.

...choke-full of no-answer question pages. They do make it look more like a search page, actually:" we don't have an answer but perhaps you'd be interested in these pages?" - the list follows.

IMO, these can be the worst, particularly if none of the other pages have answers either, which is often the case. They make you waste your time, and it feels like you've been scammed and spammed. I'm sure Google is aware of the problem. It might be they haven't fully solved this problem algorithmically yet, but I'm sure they will.

[1script]

In fairness though, it's not like Google couldn't just ignore those results.

They can and they already do. I've been collecting data about this for most of August and even though one month is probably not very significant statistically, what I see is that my Google traffic to questions with no responses is a meager 1.21% of my overall G traffic which makes them 10 times less likely to receive a visit from Google SERP (about 12% of the questions don't have answers on my site). So, if feels like no matter whether I do no-index or not, it will only affect a tiny fraction of the traffic. It's actually hard to believe that anything having to do with such small part of a site can affect the site as a whole but Google seems to work in mysterious ways...

On the other hand, if it comes with at least some hope for escaping a next Panda refresh, 1% of the traffic is certainly worth it.

[indyank]

stack overflow is a great example of how a forum should be. Any forum could use them as an example. One heck of a site that deserves all the love.

[rango]

They can and they already do

I'm specifically referring to the fact they could ignore them when calculating panda ranks, not to ignore them in the serps. The only real reason to noindex them is to avoid a penalty. But since they can so easily see it's an unanswered question - why would they bother counting it against you or showing it in the serps.

Stackoverflow is a good format for some types of questions - specifically ones where there is such a thing as a "best" answer.

In other fields, it's better to have discussions rather than a definitive answer imo.

[indyank]

Ok. To clarify it is not just about the format but their IA as whole and also about how they moderate and what pages they let search engines to index while ensuring unanswered sections are blocked.

[netmeg]

I'm specifically referring to the fact they could ignore them when calculating panda ranks, not to ignore them in the serps. The only real reason to noindex them is to avoid a penalty. But since they can so easily see it's an unanswered question - why would they bother counting it against you or showing it in the serps.

Cause a bunch of people bothered to allow low quality pages to be indexed in their search engine? Just guessing.

[FranticFish]

websites need to look to how they are adding value to the answers.

So surrounding them with scraped material and ads would be a no-no then?

[Sgt_Kickaxe]

Get specific. Saying traffic is down 50% tells you nothing. Figuring out which pages no longer receive traffic is the first step. Evaluate them and compare them to pages that weren't affected. ARE they the nearly empty pages or not? If the drop is indeed sitewide start looking towards a possible manual action since, obviously, not all your answer pages are low content etc.

[hitchhiker]

I should probably kick in here - cheers for all the feedback, few great reads in there. But I should probably update this a little:

1) Our system is extremely complex, among the many thousands of decisions it makes during render we have rules and processes for:

- no ads on blank pages, infact we dont show ads until 2 replies

- link quality internally never focuses users on 'blank' pages - i dont think we actually have any blank pages! - We auto-301 anything that has expired back to a near match.

- quality assertions based on natural language analysis of our content, it naturally focuses itself on high grade stuff

- browser aware content (low grade browser DO NOT get the same scripts) - i dont care whether Google thinks that's cloaking.

- the 10s thing is a problem with how GOOGLE measure page speed, we'll adjust to suit them. Our interaction time is 400-800ms on average.

- we're not resisting change, or format changes, we actually have to slow down change cycles because it affects END USERS -- we have members who spend thousands of hours, with upwards of 50k posts. We are careful not to let G decide our page layouts.

2) We don't let this kind of thing freak us out - we're already almost back - sways like this are an inevitable part of being on the net - this thread was a bit of a slip up on my part.

3) Im never going to no-index at that kind of micro level - that's google's job. I'm going to =focus= on the million things needed to move forward.

We were 35k down, we're now about 20k down - my guess is we'll be back to 'normal' within a few weeks. It was nothing but a slight change in algos that affected us.

One curiosity is that we have never been hit by any panda or penguin, we always got a (big) boost of traffic during those events.

My guess is whatever hit us now, was a reversal of panda/penguin as had been described over the last year or so.