Zombie Traffic from Google and Traffic Shaping/Throttling - Analysis

[tedster]

We need a dedicated thread to look at this odd phenomenon being reported by a subset of our members. It really isn't about any particular "update" because the apparent signs have been reported since 2008.

I have personally seen just a few examples of traffic shaping and nothing I could really call zombie traffic, but I think it's time for all of us to take the reports seriously and at least give advice on how to analyze what these webmasters are seeing.

To truly make sense of this, we'll need to pull in many areas of Google that we rarely talk about. This ain't your daddy's SEO! Here's a pretty good overview, from 2010: [webmasterworld.com...]

[bobsc]

What I can't understand is - how Geotargeting works? It doesn't!

[tedster]

Geo-targeting basically generates different search results based on the IP address of the user. Of course, it also depends on accurate tagging of the geographical relevance of each website, too.

I think any apparent traffic shaping may be based more on other kinds of taxonomy, not just geographic. Google does build automated taxonomies that they use to tag websites, users, and query types. I honestly can't see them intentionally mis-matching those tags. However, I can see accidental mis-matches occurring. And I can see that there could be statistical testing of these taxonomies, just to verify or question an original automated determination - a sort of machine learning.

Even though that model of mine could explain short term zombie traffic, I can't see why it would go on for months or years - and just for some sites.

[Sgt_Kickaxe]

I wanted to add that, around the time Penguin and the new shopping was released, the 'traffic shaping' patterns shifted and it now seems to occur more prevalently between Thursday and Sunday, previously it was Friday to Sunday.

[diberry]

@tedster, could we kind of put together a list of the various theories here, and what's wrong with them?

--Maybe Google is trying to force Adwords buying on certain phrases. Some people claim this would be too expensive in terms of computing power and all that; others say it wouldn't.
--Maybe Google is trying to give several equally deserving sites their time at the top of the US SERPs, but it doesn't really follow that the traffic during your not-at-the-top times would be totally useless "zombie traffic." It should just be more mixed.
--Maybe Google is testing different taxonomy profiles to find site niches, but that wouldn't take years.

Are there others? What about specific theories on what this could be an accidental byproduct of? Like, what would the algo be trying to accomplish when, as a side effect, it winds up sending certain sites this zombie traffic?

[tedster]

Here's my counterpoint to each of those three theories:

Maybe Google is trying to force Adwords buying on certain phrases.
The "Adwords theory" has been voiced since the beginning days of Adwords, and Google has always passionately denied doing that. In fact, playing around with their core competence (organic search) in that way would destroy Google. Over the years some companies have analyzed huge piles of data hoping to catch Google in a manipulation like this, and it never has held any water.

Maybe Google is trying to give several equally deserving sites their time at the top of the US SERPs
I'd say your counterpoint is right on the money. Intentional ranking rotation would show ups and downs in total traffic, not non-converting zombie traffic at the same essential level. In addition, the webmaster could pretty easily catch it in action - and so far no one has.

Maybe Google is testing different taxonomy profiles
And as we both agree, such taxonomies testing would not show patterns that last for years.

I don't have a site under my management that shows a throttling or zombie pattern of any kind, so I can't really dig into any data myself. I'm hoping someone who feels they have this issue will look more closely at it, segmenting by keyword, geography and time of day - if this is not just some mirage it seems to me a pattern should start to show itself.

[tedster]

Relevant post from diberry in the June Updates thread [webmasterworld.com]:

--For most of today, I was ranking lower on lots of my usual phrases. This evening, I suddenly ranked higher on them. This sort of pattern keeps repeating, sometimes in reverse.
--And within that pattern, there are two fairly distinct SETS of keyphrases. One day, it's all my usual standbys that are fluctuating in position, but the next day it's a bunch of other phrases I've never ranked that well for. Sometimes the keyword set will shift during the day.
--Despite all these shifts, I'm getting a set amount of traffic from Google each day, and it never varies by more than about 30 searches.

[diberry]

Okay, I was wondering if that sounded like traffic shaping, but I'm getting such a low # of searches a day right now that it's hard to establish anything from that small a pool. Like I said in the other post, it's my feeling based on looking at these numbers and words for a lot of years, and this suddenly striking me as nothing I've ever seen before. It may evaporate under skilled inspection.

I'm not very skilled at crunching analytics the way you guys are (segmenting for huh?), but I would be happy to let you have a peek at any stats you think might help, Tedster. Or if it's not that hard to explain to me what I'm looking for, I'll look for it and report back!

_{Edited for clarity.}

[Shaddows]

diberry, that is very similar to what I saw when I first started to believe [webmasterworld.com]* in a Traffic Shaping mechanism (although our daily variation is significantly more)- except I couldn't see any significant ranking change.

Obviously the problem we all have is data access. I am not about to give anyone access to my logs or analytics, and I really do not like giving out site specifics. Especially as I don't have a Traffic Shaping "problem" - my normal traffic is perfectly fine. However...

It occurs to me that Zombie Traffic is not being seen by webmasters with high levels of traffic, with few exceptions. We get up to 7500 uniques a day, 5000 on weekends. Average is around 6200. I suspect that is on the high side for Zombie Believers.

Now, a significant proportion of my Zombies are on tertiary product ranges. In aggregate, these get "normal" traffic in the low hundreds, spiking to around 1000 during heavy Zombie attacks. We have thousands of products in that "tertiary" grouping, so there is plenty of room to wander, but they limit themselves to a small, frequently changing subset of a couple of hundred products.

Whenever we get extended Zombie periods, a referral shift usually follows within days of it disappearing again. The referral shift is usually of our primary (60% of sales) or secondary (30% of sales) products- not where the Zombies were roaming.

The important take-aways are this. Zombie traffic results in changed traffic- that is either a reprofiling (of us, of searchers, of query intent, or all three) or an algo change.

Anyway, my theory is that the reason we can't pin down Zombies is due to the targets they visit. Here's some figures.

1000 uniques a day is substantially less than 2 uniques per 10 product pages, and 0.5 uniques per 10 indexed pages. In aggregate, it is not statistically significant. Big sites (who are more likely to number-crunch) that have relatively evenly distributed traffic are just not going to see this.

I'm not going to go further into my site metrics, as I have posted rather a lot here. Suffice to say my primary product rankings are robust, in high volume niches. My secondaries are robust but less competative. My tertiaries are long-tail (I prefer midtail). The fact all the chaos is in the longtail means my headlines are unaffected- again, I suspect this is rather unusual.

My working theory is twofold, with an untestable assumption thrown in. The assumption is that Google has layers of profiles.

Fist part: Zombies are an intensive "scanning" method- using a weighted statistical traffic profile to sling mud at a site and see what sticks. It is used for LOW VOLUME sites... or sections of sites where that site has been differentially profiled by niche. Read that sentance again, it's important.

Second part: The data gathered is fed back into multiple metrics. Some are page-level. Some are page-group level. Some are site level- even though the site was only partially sampled.

I really don't want to share more until there's some quid pro quo from others with similarly qualitative data.

******
Notes:
1)Google needs to use a certain amount of traffic to get statistically significant results. This minimum level is a higher proportion of small sites' overall traffic, hence more noticable

2)If Google calculates a % of site traffic to use for its Zombie scan, that will be more obvious to those who have a steep distribution curve- and invisible to those with a shallow one.

3)As well as scanning sites, Google could be categorising query intents by showing them multiple sites. Thus, some "Zombies" might actually interact.

4) To anticipate the "netmeg repudiation" that G doesn't "send" anything... For my theory to work, a small, predefined sample would have to be shown different SERPs. Highly focussed A/B testing if you will. Then the rest of the world would see rankings unchanged.
******
*Please, please read that thread. It was written pre-Panda, which means a significantly different set of members were contributing.

[edited by: Robert_Charlton at 6:50 am (utc) on Nov 21, 2015]
[edit reason] fixed typo per poster's request [/edit]

[diberry]

The problem with my site is that while I was getting over 100k visitors per month for a while there, I'm now getting far less since Penguin - like you said, it's not a big enough sample pool to yield any real statistical meaning. In fact, one possibility is that whatever I've been seeing has been going on all along under the surface, and it's only the loss of my most popular keyphrases that has revealed it.

I should also mention I can't see "zombie" traffic. My conversions have slowed to a trickle since Penguin, and I can't see any on-off pattern. It's just mostly off.

Keyphrase shifts

I'll tell you what I'm seeing as best I can describe it, using pretend stuff that I think will give you a good approximation of what's going on. Let's say my site is about jobs, and I have categories about getting a job, keeping your job, getting along with co-workers, dressing for success, getting promotions, your rights as an employee and dealing with stress. Until Penguin, Google mainly just ranked pages from my Dressing For Success category, which were up against pages from fashion websites. This made me scratch my head and wonder if Google was totally confused about the site's purpose, or if I just couldn't compete on the other topics but somehow was rocking it in the fashion topic, but hey, whatever. Traffic was good!

Then Penguin came, and my DFS pages fell to the second fold or other pages entirely. My traffic from Google fell by over 80%. For weeks, the DFS pages were still my main source of traffic from Google, I just wasn't getting as much of it.

Then I started seeing long tails I hadn't been seeing before, same as many others have noted. These long tails were still mainly for DFS pages. Then on May 28, I logged in around 7am as usual, and instead of the usual mix of keyphrases leading to DFS pages, my Google traffic had all gone almost exclusively to the Getting A Job category, which was actually my most popular. I was happy about this - until later that day, when it went back to the usual keyphrases.

That particular day seemed REALLY distinctive to me, and I believe anyone looking at it would agree something was happening on Google. Since then, there are days when it sure feels distinct, but maybe people are just searching for different things according to what they've heard about elsewhere or something.

Ranking Shifts

Yesterday's a good example. My analytics package tracks my average Google ranking, using a string Google puts in the URLs. I noticed at 7am that I'd fallen several positions on all my usual top keyphrases. By 5pm, this had turned around, and my average rank was back up to normal (which suggests it had been ABOVE normal for some searchers, since my search volume didn't increase by much).

Search Volume lack of shifts

The number of searchers Google sends me each day varies by no more than 25% of the highest number. This trend seems to be following a weekly pattern of troughs and valleys that the site has followed for years, it's just now it's about 80% less traffic than they used to send.

THIS, to my thinking, is where it gets odd, given the shifting keyphrases. What are the odds that I would rank exactly equally well on Dressing For Success and Getting A Job, two very different categories? Traffic from other sources (other SEs and social media like Pinterest, Stumble, FB) shows an overall preference for Getting A Job pages, and visitors respond better to it, and it's certainly the topic I know more about. Somehow, no matter how Google presents my site to searchers, my traffic volume doesn't vary by more than 25%. Could it happen by coincidence? Sure, but why is the Google algo the only thing on the internet that thinks these categories perform exactly equally well or badly? It just doesn't pass the sniff test for me.

Feel free to ask for more specifics, and I'll provide what I can/feel comfortable sharing.

I should also mention that I've done the following since Penguin:

--I had a lot of articles with "14 tips for..." or "how to" in the titles, and I've gone through and changed many of those to leave those phrases out. (I only did those to be trendy, but I realize now they were probably a trend because they were good for SEO, and therefore might make me look overoptimized).
--Over the past week or so, I've deleted about 40% of my pages because various signals led me to believe visitors found these particular pages irrelevant, lacking or even irritating.
--I'm very slowly rewriting lots or pages, hopefully to make them more valuable to visitors.

[backdraft7]

@diberry - Sorry if I digress from the Zombie topic for a moment, just thought I'd share an observation, since you mentioned it...the "how to" phrase seems to have some new value placed on it. I lost all my positions that included that particular phrase. Doing a site search, it does not appear to be overly used. Only bigger platform sites seem to have been awarded that phrase, even though the are ridiculously stuffed with the term. What's funny (sickeningly so) is that they removed my organic results completely and optimized my Adwords ads to show up on those pages instead.

They seem to be getting wise to what we sell and know which organic results to pull and replace with cpc ads. Problem is, the adwords ads are not producing either. The end result basically appears as throttling.

[diberry]

Actually, that's really funny... I didn't think to mention this, but the day I had that very distinctly different set of keyphrases - what was so distinctive was how many of them began "how to", which is a phase I've never really ranked for. Looking down the list of searches that had brought me Googlers, it was just "how to..." at the beginning of almost every line. That's what was so odd.

And yes, I've noticed what you're talking about with how to. I find it very suspect, because if you search for "how to [something ehow has no article for]" you will STILL get an eHow article (or several) with some of the same words in it as your top result(s). Even Wikipedia only gets the top spot when they actually have an article on the topic. Even if I knew nothing about webmastering, I'd suspect eHow and Google had a special relationship. It's just too bizarre.

[gehrlekrona]

@diberry, I have notices the same thing. People seem to be asking questions and I think that it is the way google want to go. They said in an interview that they want to give people ansqwers since that is what people mostly go to google to look for, answers.
Someone mentioned splitting up a site in one free and one pay site (buy) and I think the idea is not bad since I think google in first place split up sites in 2 groups, one group is where you can buy things and the other one is for information.
I have totally redone my site and it didn't hurt me but it didn't really help either, as far as I can see so I am guessing that the old site wasn't too bad, even with duplicate and other issues and the new one in HTML5 with semantics didn't make the cut. Maybe it is a site penalty of some sort or they just can't figure out where it is supposed to be in the search.
Right now Google "only" accounts for 40% of my traffic and the other SE's take up the rest (of course).
When it comes to forcing people to use Adwords I think they do, but not the way you might think of it. If I see that my hits disappear then it is not far away from thinking that "maybe I should try Adwords...."

[tedster]

@Shaddows wrote:

Zombies are an intensive "scanning" method- using a weighted statistical traffic profile to sling mud at a site and see what sticks. It is used for LOW VOLUME sites... or sections of sites where that site has been differentially profiled by niche. Read that sentence again, it's important.

Is the bold section a typo or is it missing some words? I can't follow what you're saying there, and I want to.

[edited by: Robert_Charlton at 7:04 am (utc) on Nov 21, 2015]
[edit reason] fixed typo in quote, per Shaddows' request [/edit]

[Shaddows]

Embarrasing. Emphasise a sentance with a typo.

"profiled by niche". Not "nice". A Moderater change would be appreciated.

By way of expansion,
"It is used for LOW VOLUME sites... or sections of sites where that site has been differentially profiled by niche"

So, we span a few niches. In the real world, its a single, integrated business. But online, it really isn't. Take @diberry's example of Employment Advice.

Dressing For Work lives in the ultra-competitive fashion niche. Contrary to diberry's example, lets assume it performs extremely well, dominating the volume searches, all due to excellent content and WH SEO. <Business Suits>, <[Brand] Suits>, <workplace fashion>. That type of thing. Big corporate players in this space, but still being outranked.

Then there is the semi-pro areas, where a load of large and mid-size players compete. Employee rights, getting a job, interview techniques. Loads of trash sites mixed in here. Totally different market from Goog's POV.

Then there's the low volume areas. Keeping a job, getting along with colleagues. Different market again.

Now, you might be rock solid in the competitive Fashion niche, but you're site as a whole is about Employment. Maybe you haven't put as much SEO into "Keeping a Job" although you rank well due to good content, not much (worthy) competition, and sitewide factors.

G thinks "This niche is not very competitive, better establish some qualitative data" and starts running some testing. Some sitewide factors get re-evaluated, impacting parts of the site that weren't tested.

Sites without the breadth wouldn't have their solid traffic in the non-tested areas to rely on. This doesn't explain what happens to their "normal" traffic though, unless its Traffic Shaped away from users that aren't the Test dataset.

[diberry]

Shaddows, that was kind of my instinct of what was wrong with the site, even before Penguin. I had switched from Feedburner to a real email subscription service months before, and noticed I was getting a lot of "content no longer relevant" unsubs whenever I published pages in certain categories.

I had been wondering what to do without destroying my SE traffic when Penguin hit and took out the traffic for me, LOL! So I pruned huge amounts of content to hone in on what visitors seem to like and avoid the stuff they call "irrelevant." I did this about a week ago, so it's way too early to conclude what caused the following changes but:

--Traffic's up about 25% this week from last (Penguin knocked it down by 82%).
--Unsubs are down and subs are up a bit

But I'm having another very distinct fluctuation on Google this morning.

--Suddenly this morning, Google traffic is up 68% from yesterday *so far*. It hasn't been up more than 30% at any point in my many daily checks since Penguin. It'll be interesting to see if it stops dead at noon to keep it down in that level I'm stuck at since Penguin.
--What's driving the traffic is non-exact-word long tail. In the past, most of my long tail searches have been nearly exact word matches for page titles or headers. Today's long tail phrases are clearly well-matched to my resulting pages, but the wording isn't nearly exact. Sometimes it uses synonyms.

If this sticks, it could be a great change. But I'll let you know what happens, since this is exactly the sort of fluctuation you'd expect if the algo is running tests, and it should lead to some kind of lasting change if it is indeed just testing rather than traffic shaping.

_{Edited to update numbers.}

[diberry]

It didn't stick. Through the noon hour, I was up 94% on Google traffic compared to yesterday. Then it plummeted. The exciting long tail searches didn't completely go away, but they were significantly diluted by the old search phrases, and my rankings on the old phrases were lower than the long tails, so the end result was that I'll have less traffic today than I had yesterday, despite doing so great up until the 1pm hour.

I also noticed something interesting, but possibly meaningless: before 1pm, I wasn't seeing as many "secure searches" as usual - that's where the user is logged into the Google account, so Google hides the search phrase from the referral string so analytics packages can't pick it up. After 1pm, it went back to the normal amount. Secure search visitors seem to behave like other visitors, so I don't know why this would happen.

I should mention that my traffic typically peaks on Thursday and troughs on Saturday, so some fall was to be expected this afternoon. But this struck me as a big fall that happened abruptly. I mean, at NO point since Penguin struck have I been up 94% compared to the day before, but even that got wiped out to bring me back down to my usual post-Penguin numbers.

[diberry]

Also, the June SERP changes thread reports a big shift around 1pm PST, 4pm PST and/or 10PM PST yesterday. [webmasterworld.com ] That probably accounts for the keyword shift.

But what I find so odd is that being up 94% until 1pm, my Google traffic dropped sufficiently that I ended up with 1.5% less visitors this Friday than I had last Friday. In fact, except for the holiday dip, there's never more than about 1.5% change on Fridays. Some days vary up to 16% from one week to the next (again, excluding the holiday dip), but that's the biggest variance I can find.

I honestly don't know if this is even close to statistically meaningful. I'm just offering it up in case it helps.

[Bill_H]

< Originally part of the Panda update discussion [webmasterworld.com] >

We are seeing absolute evidence of traffic throttling. I have our impressions chart from WMT it is a series of exactly the same (rounded off) impressions for x days, with a one day drop to y between each repeat, this pattern repeats for four weeks, then we see x days now at now at y impressions repeated four times, with a one day drop to z between each repeat. I would be happy to email anyone with the WMT chart as I can't attach it here. I have to believe that some sort of throttling is occurring using how many times a url can appear in the serp.

Cheers,
Bill
US
E-commerce

[edited by: tedster at 11:20 pm (utc) on Jun 11, 2012]

[diberry]

Thanks for adding Bill_H's post, Tedster!

@Bill_H, what impressions chart are you talking about? I'm not the most savvy WMT user...

[Robert Charlton]

I have to believe that some sort of throttling is occurring using how many times a url can appear in the serp.

Perhaps related... I'm guessing that the recent rollout of search refinements (at least that's how I see it) might also lead to testing that we could see as some form of traffic shaping.

I don't know whether the type of refinements Google is testing will produce easily observable patterns, but see notes here on what I see the Sitelink type refinements being tested are likely to be....

Host Crowding vs Brand Authority
http://www.webmasterworld.com/google/4464096.htm [webmasterworld.com]

I'm initially, at least, not seeing localization effects in the searches, but I performed only very cursory searches and location changes to check that.

[Bill_H]

Tedster:
I strongly believe traffic throttling is occurring by G putting a ceiling on how many impressions per keyword a website receives. Our WMT impressions graph is simply far too consistent to be believed, and seems to reflect some sort of testing.

If I may explain testing. Our WMT impressions graph will show six days exactly at 10k impressions, then one day at 8K impressions, it will repeat that pattern three times, then we see six days at 8K impressions, on day at 6K impression repeated three times and then...

Sorry but there is no statistical way in the world that it can be anything but traffic throttling by throttling impressions. More than happy to email anyone the snip from WMT showing this.

Cheers,
Bill
US
e-commerce

[diberry]

Sgt_Kickaxe said the following in this thread [webmasterworld.com ]:

It actually began late June 5, early June 6 and something that happened, at least to me, is that my peak traffic days have shifted, again.

Eons ago: Thurs to Sunday were the weekly peak traffic days.
3-4 years ago: Sat to Monday became the most trafficked days.
Last year: Sun to Tuesday was the peak.
As of June 5/6: Monday to Wednesday is the new peak, from Google.

Now, I know the entire internet didn't shift how often they use the internet on June 5/6, or likely any other of the times above, so something Panda can do is place your site on a different traffic "schedule". (unless Panda isn't the only "change" Google just made)

We've been thinking of rankings as static, you get x% of the searches for your keyword based on your rank, but that's NOT how the internet works anymore. The rankings are in flux depending on site type (informational vs transactional vs locational) and day of week, keep that in mind with any report you make here... please.

I think this may be relevant. It wouldn't explain traffic shaping to the degree some people are seeing it, but what if we're not only competing for rankings these days, but rankings at prime times of day and week? Like, the value of a TV ad spot goes up depending on the time slot. What if Google's starting to view the SERPs that way? Then their choice for #1 would get the top spot at the top time of day/week, and other top choice sites would get it during less prime times.

If that's the case, then one of my sites (which has not been hurt by any animal updates so far) has just recently developed an odd pattern with the number of Google searches I get per day:

Sunday, June 3: x
Monday, June 4: x-1

Sunday, June 10: x+98
Monday, June 11: x+98

I'll have to see how the rest of the week shakes out, but this struck me as very odd, those flat lines in almost exactly the same place on the graph.

I wish, too, that we knew what traffic shaping MEANS, and whether there's some action we should take when we see it happening. Whatever it is, I wonder if it happens to some sites we're not hearing from, because they come out of it with a higher level of traffic, so they don't feel the need to come in here and wonder what's going on.

[theseanstewart]

Since Friday traffic for one of my ecommerce clients has been solid, rankings have been stable, but the quality of traffic is terrible. Conversion rate is down big time.

[Robert Charlton]

Just a note on this thread that it was reported in the Update thread about an hour ago that Google isn't generating a default location... and several of us have confirmed the same. Original report was on east coast US, and I'm on west coast.

Since many of us assume that zombie traffic and traffic shaping may in part, at least, be geo-related, it could be interesting to see what effects, if any, this anomaly might have.

[diberry]

Good point. And just for the record, Google still knows where I live, so it's not happening everywhere (which might dilute any effects).

[diberry]

Okay, so last night my Google traffic stopped dead at 10:21. That's not typical at all. But this is the very first time I've seen a change from the number range I've been stuck in for so long - and it was a downward turn.

I've been making massive, long-needed changes on this domain - I'd been putting those changes off for fear of wrecking my Google rankings, but now that they're already wrecked, I've gone ahead. So:

--2-3 weeks ago, I deleted about 40% of my pages to tighten up the site's niche. Visitors are liking it, and so are social media and other search engines. Google still kept me in that same range every day.

--This past week, I changed domain names and rebranded to reflect the shift in the site's niche over the last few years. Nothing changed - just the logo and URL. 301 redirect. Google seemed to be ranking the site about the same as the old one until yesterday. I am reading that a lot of people felt there was a Panda shift yesterday, so maybe that's something to do with it.

[timwilliams]

last night, 6/20/2012 at 5:30pm (eastern standard time), my conversion rate tanked (by over 70%)and has not yet return. Like someone flipped a switch, across three different ecom sites. Not seeing any change in amount of traffic, just not converting.

[Sgt_Kickaxe]

Add affiliate links - watch traffic plummet a few days later

Remove affiliate links again - watch traffic return a few days later

Still need proof it exists? I think the above would even hold up in court as evidence.

[diberry]

Sgt_Kickaxe, I read what you were saying about that in another thread. That definitely sounds demonstrable. Now I'm wondering if Google could come up with any reason for doing it that doesn't boil down to tinkering with the SERPs that are supposed to be objective and not play favorites?

[backdraft7]

Can anyone concur that something drastic happened on Saturday June 23 around 2pm CST. Saturday morning had sales on the hour up until that time. Since then, it's been the same old non converting Zombie garbage traffic. The previous week was fine and even a few consecutive double digit sales days.

Sunday and Monday (today) have been terrible...same traffic volume as previous years, but 1/10th the conversions.