Zombie Traffic from Google and Traffic Shaping/Throttling - Analysis

[tedster]

We need a dedicated thread to look at this odd phenomenon being reported by a subset of our members. It really isn't about any particular "update" because the apparent signs have been reported since 2008.

I have personally seen just a few examples of traffic shaping and nothing I could really call zombie traffic, but I think it's time for all of us to take the reports seriously and at least give advice on how to analyze what these webmasters are seeing.

To truly make sense of this, we'll need to pull in many areas of Google that we rarely talk about. This ain't your daddy's SEO! Here's a pretty good overview, from 2010: [webmasterworld.com...]

[backdraft7]

Not just to bump this thread, but I'm now going on a 7 day trend of the EXACT same number of sales for each day in the past week. This is extremely rare and almost a statistical impossibility. It appears either they've got my traffic shaped so perfectly or are so screwed up that it has eliminated a stable 10 year weekly trend of high on Sunday, through low on Friday. I can not figure any reason that they would want to do this other than to control my Adsense income. Bizarre! Worst week since 2005.

[backdraft7]

Just to carry over the sales conversion via traffic shaping theory, I'm not saying that Google uses conversions to throttle traffic, it's just the reverse...they throttle traffic and the natural result is loss of sales. This requires very little computational power, just some simple statistics.

They slash 50% of my highly related terms for no apparent reason. The other 50% that remain are basically equally weighted across the site, I detect no keyword loading using my site analysis software, yet they -100 just half my site's keyphrases. Very suspect.

I have verified no manual penalty via a re-inclusion request.

Now, the question of motivation? That's easy - to force you to increase Adwords spending. I've been a loyal adwords customer for 10 years. So long in fact that I think I may even have grandfathered 5 cent bids on my account. This may be a way to push me to spend more. Believe me, I'm not the only one with this "crazy" notion. I just call 'em like I see 'em.

[diberry]

I think you make a really good case for this. One of the things I think people forget around here is to step back and think like a Google user who just wants to find something and thinks a "conversion" is when you turn a garage into a bedroom:

"I was talking to a group of people at a supper club last night and they asked what I do for a living, I tried explaining it to them, which led to my trying to explain Google update upsets. These non experts instantly knew about periods when Google delivers some real garbage results and they are unable to find what they were looking for or useful sites suddenly disappear. It's during these periods that we WM's experience Zombie traffic."

I can't tell you how many non-experts have mentioned this to me over the years. "I went back to find that site I wanted to show you, but it's gone!" Or, "It disappeared after lunch yesterday, but it came back this morning!" They find this bizarre and frustrating, and Google wants them to love Google's results, right? So there must be *something* that's worth irritating regular users for here.

I think forcing people to spend on Adwords is a very good explanation for why Google might want to do this. TimWilliams suggested in another thread:

"Google needs multiple advertisers for adwords to work in their benefit. If there was only 1 dominate advertiser in a niche that advertiser would not have to bid up their ad. Same goes for the serps, if a serp is dominating the sales then the advertisers go away. It's in google's best interest for everyone to make a little money and no one to dominate. Keep us all hungry but nourished... "

Traffic shaping could keep a whole niche competitive. If you were ranking #1-3 for a keyword already, you might not feel the need for Adwords on that phrase. But if NO ONE is consistently #1-3, then EVERYONE needs Adwords.

In other words, instead of wondering dubiously what value Google could possibly get from doing this to your site, they need to be asking what value Google could get from traffic shaping a whole niche.

And this would take very little effort on G's part for sites that have Analytics. If you can set a goal on your conversion pages, so can they - that simple. If you don't have Analytics, they can still control the amount and quality of traffic they send you per hour or per day easily if they so choose. They're already tracking all this stuff, as we know from WMT.

No one's suggesting they do it to every site - that WOULD take some computing effort. But for certain sites in certain lucrative Adwords niches? I find it very plausible.

[tedster]

I detect no keyword loading using my site analysis software, yet they -100 just half my site's keyphrases. Very suspect.

Now I'm not as clear about your situation, backdraft. I thought it was traffic throttling in general, but now it seems to be keyword related. Care to explain further? The earlier throttling mechanism we talked about (a while ago) was geographic, but keyword throttling would be quite different, wouldn't it?

[gadget26]

Keyword throttling sure makes sense to me. Keep in mind that they have had the mechanism for keyword throttling in place for many years...it's called AdWords. Is it a coincidence that they hid most of the keyword data just before this started? Plus I have noticed that organic traffic often drops considerably right after a single high $ conversion and, after a pause of an hour or two, zombie traffic begins. I'm waiting for one more feature (that's in the works) from my secondary analytics package then I'm dumping google analytics for sure.

Related: On the zombie traffic. Most of the time I get zombie traffic. Once in a while, for a few hours, I get good traffic. ("My turn", as somebody hear coined.) During "my turn", I see a vast difference in both pages-per-visit and time-on-site stats. Over 3x difference sometimes! I've been watching the keywords during both periods. I don't have analytic software that can break keyword frequency down to less than a 24 hour period, which is what I would need. Maybe I'll have to write something.

Anyway, I've gotten to where I can tell which type of traffic I'm getting just by watching the keywords that I CAN see fly by. Zombie keywords come in two types, I think:

One type I call the "hail mary" search. Someone typed in a super-vague or nonsense 1-3 word non-phrase and google just threw some stuff up that sort of might match one of the terms. Why the user clicked on my link I'll never know. I do know that they won't convert.

The other type is sometimes hilariously long long-tail search queries. Usually with hideous grammar and spelling so bad that it's just sad. I mean, really. Who types a whole paragraph into the search box? These are usually how-do-i queries that seek only information...not products.

When it's not "my turn", I never see buying or specific product queries.

As for AdWords. Years ago it worked quite well for me and didn't need much maintenance. But it got increasingly complicated and I ended up tending to it daily and the returns kept going down until I just couldn't make it pay any more. I've dabbled a bit since being hit by the first Panda, but I guess I'm just not smart enough to run the campaigns right. If google is trying to force more people to use adwords, they really need to make it easier to be profitable with.

OTOH, traffic from Bing is consistent, converts well, and is slowly increasing.

Gadget, US e-commerce.

[stuartc1]

I first noticed this effect back In October 2010 ([webmasterworld.com ]), at that time others had already mentioned it. I've still not managed to resolve it.

Some of my observations:

1. I get 2-3 days per week of sales (if I'm lucky), the rest is almost dead.

2. Traffic Cap - traffic volume is almost identical each day (with a slight decline at weekends).

3. The 2-3 good sale days randomly change every few week, but they are always consecutive. For example, for the last few weeks it was fri-mon which was good. Before that mon-wed was good.

4. On good days, the traffic source countries have a higher percentage from the US and UK. On bad days, a high percentage of traffic comes from India.

5. about 80% of the search terms are hidden from analytic (this is obviously due to the new google rules).


Notes:

1. My site contains both free, and non-free products.

2. The traffic share to the free pages has increased, and non-free pages decreased.


Possible reasons:

1. Google has perhaps weighted my site as having a high percentage of free content and sends a larger proportion of non-paying users my way.

2. The traffic cap is reached before much of the good, paying users are sent my way.


Possible solutions:

1. Split the website in two. One for free information, the other for products.


I'm concerned that if I split the site, I may damage it even more. Any thoughts?

[diberry]

4. On good days, the traffic source countries have a higher percentage from the US and UK. On bad days, a high percentage of traffic comes from India.

See, this is the sort of detail that makes it so hard to doubt traffic shaping. What else but search could cause this effect? I can't think of anything.

2. The traffic share to the free pages has increased, and non-free pages decreased.
1. Google has perhaps weighted my site as having a high percentage of free content and sends a larger proportion of non-paying users my way.

It seems to be that offering free and non-free stuff is a good way to build trust in readers. I expect at least some freebies even from major brands that are nothing but shopping sites. So it doesn't make sense to me that Google would be incorrectly classifying you as a content site.

There's another possibility. What if Google is just looking at particularly lucrative Adsense keyphrases, and finding ways to cherry pick who it sends free traffic for that phrase, so as to keep people buying Adsense? When we speculate about why Google would bother with traffic shaping, it's hard for me to imagine another reason. I can't see that it improves user experience. Therefore, it's got to improve something from Google's perspective, and Adsense is most likely.

If that's the case, splitting your site would not help, because all Google wants is for you to buy Adwords.

Just some thoughts. I'm not saying any of this is correct, just offering it up.

[backdraft7]

@tedster - the slashed keywords is post Penguin and an entirely new aspect. The shaping still appears at the lower traffic levels. The past 21 days has had the same number of sales each day +/- 1. (Example: 5,5,6,5,4,5,5,6,5,5,6,4,5,etc.) My Apache volume is no longer a nice bell shape, but rather a flattened square wave shape.

I've run my numbers back to 2003 when I had even lower traffic levels, but had more natural variation and much better conversions. (Example: 5,14,8,7,18,6,8,10,15,9,6,12, etc.)

I attribute most of this problem due to poorly targeted and 'shaped' traffic that began post Mayday 2010.

One thing is certain, I'm not the only one experiencing this phenomenon.

[leadegroot]

Could it simply be actual traffic throttling?
I mean to share the results, rather than attempt to push people to adwords
eg There are 20 decent sites that satisfy query A, so they get rotated around and each get a turn at the 1 - 5 slots.
Its a shame WMT isn't more specific in the data it returns. :(

[tedster]

That's always been one possibility - but no one has ever been able to spot the mechanism that Google uses to accomplish this, if it is actually the case.

[wingslevel]

@leadegroot - looking at the motivation for throttling, I think you are right. Google is in a very precarious position with such a huge market share. Lots of potential anti-trust and other risks, particularly in the current anti-business political climate. The last thing they need is some small webmaster to get on CNN, as the sheriffs in the background are carting his inventory away to the bankruptcy sale, and explain how google has ruined his business and his life. Activist legislators would be all over this with hearings in Washington and proposed new regulations etc. - at this point, google's only potential threat is from Governments.

The real question, and I presume the reason for this thread, is to figure out how they are doing this.

[diberry]

I can think of one way to test that theory. Backdraft7 gets foreign traffic during the zombie periods. If Google's just sharing the love amongst 20 equally great sites, then that foreign traffic should be just what's left over once the US traffic is cut off for a period. In that case, Backdraft7 should see no overall increase in foreign traffic during the zombie periods - just a decrease in US traffic, with the foreign traffic as leftovers.

[backdraft7]

For the moment, I think the entire Penguin fiasco has eclipsed the throttling issue. It's nearly impossible now to analyze the background throttling when you've been totally "throttled" by Penguin.
It's no longer Zombie traffic or throttling because now there is a measurable decrease in long tail. That clearly explains the current loss of traffic and sales. Prior to Penguin, serps looked good, there was no loss in traffic, yet there were very unnatural flat sales periods that had never been observed in prior years.

I have made one observation that points to a possible motivation. My Adwords cost is now equal to or greater than my Adsense revenue. This occurred with no changes on my part. In the past, for a sample week, I might take in $200 in Adsense and payout less than $100 in Adwords. Pretty nice balance I thought...now it's 50/50 and totally tweaked by Google's updates alone. The recent drop in long tail was what really balanced it out. I suspect that when Google felt they were losing the battle, they'd just shut off a portion of my long tail temporarily. Now they just eviscerated my long tail permanently.
Case closed.

[tedster]

The real question, and I presume the reason for this thread, is to figure out how they are doing this.

Yes - both how and under what conditions. It's certainly not a common situation, from what I can see. I've been checking into apparent examples for several years now, and they often seem to evaporate under inspection. The challenge involves geographically localized results, personalized results, on and on.

[diberry]

@backdraft7 Can you look back at pre-Penguin stats to see about the foreign traffic though?

If they're trying to make sure you spend all your earnings right back to them, that's pretty cold, but it's not a new trick. Miners used to get paid in "scrip", which was "money" that could only be spent at the company owned store, where prices were severely inflated. That way, their earnings just went right back into the company, and all they had to show was company provided substandard housing and just enough food to keep their families from starving.

I still wonder what would happen if you:

--Got rid of any Analytics, Adwords, WMT, or anything else Google could possibly be using to know about your traffic
--Started advertising with somebody other than AdWords for your business

I realize that might not all be practical for you right now, but if it's been going on this long, and you're not able to combat it through better SEO, then cutting Google out entirely might be the best longterm strategy. That's assuming you're right about what Google's intentions are here, and I think your theory is plausible, at least.

[wingslevel]

I have been looking at this for years too. I have had a suspicion that some of my sites are throttled. If you look at my google referred traffic, it makes an amazingly consistent picture when looked at over months. Of course, over the last year we have had sitewide separate algos in the form of panda and penguin, so as backdraft pointed out, they tend to trump anything. So our discussion here should try to be net of their effects.

What I am thinking these days is just that the consistency comes from the relationship of off page to on page ranking factors. Off page rules. So, each site gets so much trust weight - skipping for now how google calculates trust - think of page rank as a very dumbed down little peak at this. Imagine a 1,000 page site with home page pr of 3. It might well have just it's home page at pr3, 50 pages at pr 2 and 150 pages at pr 1 - the rest get the white bar pr 0. So 80% of the site might have negligible trust juice - I think these pages, no matter how quality the content and on page optimization, are going to be relegated to the lower tiers of googles index and they will rarely get impression.

In many sites, I think the trust ranking is in fact the throttle. If the above site generated 1,000 more pages of really good content, I think it would get the same number of visitors from google organic search - unless those pages generated good inbound links and other trust signals that would raise that site's overall trust score.

The shift in balance toward off page factors favors the big brands because they have huge trust scores.

[tedster]

I still wonder what would happen if you:

--Got rid of any Analytics, Adwords, WMT, or anything else Google could possibly be using to know about your traffic

You'd need to block any visitor using Chrome. And even then, there's a good chance that some ISPs sell their data to Google. I'd say it's very futile - Google (and Bing) will be watching website traffic and using it in their algorithms.

[diberry]

You'd need to block any visitor using Chrome. And even then, there's a good chance that some ISPs sell their data to Google. I'd say it's very futile - Google (and Bing) will be watching website traffic and using it in their algorithms.

Well, crud.

[timwilliams]

there was no loss in traffic, yet there were very unnatural flat sales periods

I wanted to go back to this for a second. I've been trying to figure this out for a few years now. I drives me mad.

Same traffic, conversion rate plummets.

possible reasons:
1. google doing something funky with the traffic they send.
2. server/backbone issues (slow website will decrease conversion rate)
3. merchant account
4. payment gateway

We have all sites on the same server, so this could be an issue but have not been able to track it down.

we use 2 different merchant accounts. have not been able to track down issues here.

we use one payment gateway for all sites. have seen issues here but nothing that can be consistently pointed to.

[netmeg]

But when you say something funky with the traffic Google sends - Google doesn't really *send* it, presumably the users are clicking on YOUR link in the ten. Nobody makes them click, or sends them someplace they don't want to go. So what could they be seeing that would make them click the link, if they are untargeted zombies?

[timwilliams]

Netmeg, I'm not trying to sell google traffic shaping. Logically the answer is probably a combo of number 2 through 4.

However, if anyone can measure probability it's google. They *can* know what you search is, transaction-al, navigational, or informational. They *can* know which link you are prone to click on.

[diberry]

If the zombie traffic is often foreign, as backdraft7 said, could it be those visitors are interested enough to click the link, but then get discouraged because the site's not in their currency, or the shipping costs would be prohibitive, or something like that? I know I've backed away from sites that really interested me when I suddenly realized they did everything in Euros, and I didn't feel like working out the exchange rate and VAT and figuring up shipping and all that right then (I'm in the US).

[tedster]

None of the possible explanations I have thought of to date would can months and months of such a controlled pattern. Those who are mostly looking at conversions, can you segment your total search traffic during "on" and "off" periods in two ways: by 1) keyword and 2) by geography.

It seems to me that if this is a "real" pattern (even though rare) then shifts in one of those two mechanisms could well be in use. Of course, you may uncover some other mechanism, but there MUST be some mechanism. Otherwise the controlled or throttled traffic explanation starts to sound almost like a fantasy.

If we can nail down a mechanism, then we might have a shot at understanding when and why a website would trigger this kind of treatment.

It would also help if people share what volumes of conversion we're talking about. At lower levels (single digits over 24 hours, say) it's a lot harder to see statistical significance over short durations - so a many day or week pattern would be more important.

[backdraft7]

ATM, my problem is foreign traffic. Apparently the geo-targeting tool does not work as advertised. So, drop 27% from the loss of long tail, then chop the remaining traffic in half. It's no wonder many of our conversion figures are in the gutter. It's obvious that Google is still a very broken system.

[edited by: backdraft7 at 1:19 pm (utc) on Jun 6, 2012]

[bobsc]

It's obvious that Google is still a very broken system.

Geotargeting IS broken(among other things).

[edited by: tedster at 7:59 pm (utc) on Jun 6, 2012]

[tedster]

@bobsc - in what way do you see geo-targeting being broken - and how might that be generating "zombie" traffic for just parts of the day?

[bobsc]

...and how might that be generating "zombie" traffic for just parts of the day?

Different time zones.

[tedster]

Why would geo-targeted traffic from one time zone convert so much more poorly than another? People report that their conversions just "stop on a dime."

[bobsc]

If your US site gets traffic from the UK(for example), but you don't ship there - how can it convert?

[tedster]

Sure, I get that. But why would the drop-off in your conversion stats be a sharp cliff rather than a gradual slope? That's what I've never been able to wrap my head around.