Zombie Traffic from Google and Traffic Shaping/Throttling - Analysis

[tedster]

We need a dedicated thread to look at this odd phenomenon being reported by a subset of our members. It really isn't about any particular "update" because the apparent signs have been reported since 2008.

I have personally seen just a few examples of traffic shaping and nothing I could really call zombie traffic, but I think it's time for all of us to take the reports seriously and at least give advice on how to analyze what these webmasters are seeing.

To truly make sense of this, we'll need to pull in many areas of Google that we rarely talk about. This ain't your daddy's SEO! Here's a pretty good overview, from 2010: [webmasterworld.com...]

[gadget26]

Just as a point of information, I dropped GA several months ago and I still get very clear periods of zombie traffic. (Although my overall conversion rate is slowly climbing. YMMV)

The biggest problem I have in analyzing these periods is that Google seems to be very careful to turn them on/off at odd periods of the day. Since the granularity of most analytics software is one day, it's hard (without custom software) to split out these periods to look for differences.

[tedster]

Some thoughts.

1. Whatever Google is doing here, I really doubt that is intentionally aimed at undermining business for specific websites. Instead it would be a secondary phenomenon - a side effect of something else. Discovering what that "something else" is could provide ideas as to how to avoid it.

2. I know it can take a good bit of work, but raw server logs plus grep can really help zero in on suspect periods. There are also paid raw log analyzers that can make this approach easier. I've built traffic graphs on 10 minute intervals to isolate odd patterns on very busy sites. I've tracked down individual customers with this approach, as well as specific hacking attempts, individual scrapers and bots, and so on. If someone's livelihood is slipping away, it seem reasonable to me to dig this deep into the details.

3. If you think page load times are periodically affecting user experience, try boomerang.js to get that analysis of your actual traffic - it is free.

4. The loss of long tail traffic for specific periods of time can be confirmed or ruled out through keywords in the referrer string.

5. How Google is seeing user intent should also be noticeable through the keyword referrers.

6. With regard to zombie traffic, a lot here cold depend on volume. If traffic and conversions are general low even when they are good, then variations might well be statistically expected and not "caused."

7. Checking what the results page looks like when traffic turns zombie or vanishes might provide a clue. Does a News, Video or Image listing appear and disappear? or move up and down? How about Google Maps or Products?

It is also entirely possible that individual reports here (either traffic shaping or zombie traffic) are not seeing the same thing at all. Analysis could turn up different underlying causes for different sites.

[bluntforce]

I had a page where I was seeing 40% bounce, 7 second average on page time and a 21% exit rate. That wasn't in line with what I'd expect, so I spent some time with on-page issues. End of day, bounce was down to 25%, time on page was up to 31 seconds average, exit was down to 18%.

Changes were visual, brought into conformity with pages that had the desired statistics.

[Martin Ice Web]

To give some food to backdrafts theory.

We see exact the same and to the same time.

This has nothing to do with poeple do not want to buy but with google sending poeple to unrelated pages.
As we see every query to the related site we know what the poeple are looking for and see they are missleaded.
This has nothing to do with google does not know what your pages is about ( we did siloing ).
Long tail queries with => 3 keywords are missinterpreted by google or they ignore one keyword or they replace one or two keywords with "similar" but unrelated synonyms. Addwords is not doing this, it is matching perfect.
Prior to panda we got matching user queries, the users were buying the widgets. This all started with panda 04´12 and then penguin.
To analyze anything with GA is waste of time, the only conclusion to get is that google is not a search engine anymore but a $ printing machine.

[tedster]

Long tail queries with => 3 keywords are missinterpreted by google or they ignore one keyword or they replace one or two keywords with "similar" but unrelated synonyms.

I've also been seeing this in recent months - but not as long as people have been reporting zombie traffic, not by a long way. However, it certainly could be another factor that makes traffic a lot less targeted.

Does this kind of keyword peculiarity start as zombie traffic starts up and then disappear when converting traffic is arriving again?

We see exact the same and to the same time

Are you in the same time zone so the changes match up hour by hour? Or is it the same at local time, wherever your are located? Or do you mean you see it on the same days?

If sites are all affected in the same time patterns, that would be VERY significant, as I see it. We have not been able to establish that as a fact so far, even though we've been discussing this topic for 4 years!

[Martin Ice Web]

tedster,

when zombie traffic starts, the keyword replacements seems to affect more keywords and short tails but the =>3 long tails keys seem to bee affected all time.

We Zombie traffic as:
-foreign visitors
-sudden start of "scrapping machines" like server***u.com
-visitors fetch one page and go back ( 100x bounce )
-start of legal search engines ( yandex, some chinese engines ) scanning the site
The zombie traffic starts with 10 min. with heavy zombie traffic and will slow down the day/next day. Following not converting traffic.


we are located in germany - CEST ( i think backdraft is in the same time zone but we do gave summer time now +1 hour )

[Robert Charlton]

If sites are all affected in the same time patterns, that would be VERY significant, as I see it. We have not been able to establish that as a fact so far, even though we've been discussing this topic for 4 years!

That's something I tried to do in my May 2, 2012 post on this thread, referencing both the May 2012 serps thread [webmasterworld.com...] regarding what appeared to be geo-confused zombie traffic in Australia, which I assumed might have been a test... and also referencing my user-engagement calibration phase? thread [webmasterworld.com...] ...about which I noted....

I've since come to think that traffic shaping isn't just about Panda, and have been thinking for a while that it had to do with all Google testing, much of which I've assumed is about localization.

I see comments appearing on the forums almost always after an update and we've entered what I assume is a (new) calibration phase, and they're essentially always in sync with backdraft7's posts.

To toss out some other factors that might be worth comparing while we're discussing getting more specific about some details...

- Do the sites affected have a high product churn rate... ie, a large percentage of ongoing product or listing changes?

- Are there ambiguities in the place names involved... possibilities for geo-confusion (as I noted earlier in this thread, back on May 2?)

About throttling with regard to time of day, I've seen pretty convincing evidence that relates that to things like lunch hours and search activity, and I don't consider it the same thing at all.

[edited by: Robert_Charlton at 9:56 am (utc) on Sep 12, 2012]

[Robert Charlton]

PS to the above....

Long tail queries with => 3 keywords are missinterpreted by google or they ignore one keyword or they replace one or two keywords with "similar" but unrelated synonyms.

A bell goes off about this one as well.... I think this is Google building those fabled "taxonomies", as tedster so astutely called them. Perhaps mathematically they're "multidimensional paths or flow patterns of user experience", as I've been thinking about them. I've forgotten too much math to say.

=> 3 keywords are missinterpreted is completely consistent with the testing idea, I feel. Perhaps longer and longer tail queries will be going fuzzy for a while. Basically, I think Google is doing what it's done on every algo I've seen it test... it's started with the short, competitive phrases and then gets into longer and more specific phrases. This is also consistent with how artificial intelligence learns. It needs lots of data, and data on long-tail takes longer to gather.

Additionally, I've felt I've been seeing the auto-complete suggestions getting more specific. (Perhaps this is even what's directly or indirectly causing the effect).

All of the above is conjecture, but it's consistent with what I think I've been seeing for a bunch of years now.

[muzza64]

Just to add a bit more fuel to the fire, we recently made some big improvements to the conversion rate of our Panda hit site and at the next Panda update lost the exact proportion of traffic necessary to keep our earnings at the same level as before.

It's probably a coincidence and maybe a result of what we did to improve conversions (although that doesn't seem logical at all). I try to keep paranoid thoughts out of my head, but it does make you wonder......

[backdraft7]

+1 to MIW for the following:

Prior to panda we got matching user queries, the users were buying the widgets. This all started with panda 04´12 and then penguin.
To analyze anything with GA is waste of time, the only conclusion to get is that google is not a search engine anymore but a $ printing machine.

1. I am in the CST time zone (Chicago)
2. I share observations with several site owners in the UK and many times when they say good or bad traffic has started, it hits me 6 about hours later, not usually at the same time.
3. It always moves lower. Since 4/12 I have yet to see a recovery that sticks for more than a few hours or days.

Conversions are still in the gutter today, but with more action in the middle of the night than from all day yesterday.

[tedster]

3 long tails keys seem to bee affected all time.

Then we can throw that possibility out as an explanation for on-again off-again zombie traffic.

sudden start of "scrapping machines" like server***u.com

Does scraping traffic come in showing a Google Search referrer?

start of legal search engines ( yandex, some chinese engines ) scanning the site

Google can't be doing that. It sounds like your analysis has not focused on just Google traffic.

The zombie traffic starts with 10 min. with heavy zombie traffic and will slow down the day/next day. Following not converting traffic.

I thought we were talking about one thing here - traffic that does not convert - and calling that zombie traffic. The only other topic was "traffic throttling".

[tedster]

In the monthly Google Updates Thread, Martin_Web_Ice shared this observation about the traffic cycles that he is seeing:

IP ranges are "crowded" then nothing then "loose" IP range

Martin, thanks for sharing that pattern. Are other members seeing a similar shifting pattern of traffic sources?

If so, how long does each phase seem to last? Also, which phase correlates best with zombie traffic?

[backdraft7]

More apparent evidence of throttling...in GA, the last 3 consecutive friday traffic levels 794, 794, 794.

[tedster]

backdraft - just want to confirm that you're reporting only on organic traffic from Google Search - correct?

[xcoder]

@backdraft

GA?.... is like letting the cat watch the milk...

check piwik...

[backdraft7]

@tedster - yes.
@xcoder - thank you! my project tomorrow is to switch off GA and install piwik.

I have basically nothing to loose at this point. If there were ever an advantage (serps/traffic) to using GA, it's no longer the case. No more third party services for my site. Sorry to say Google, the love is gone.

[tedster]

@backdraft7 - further question. Did the Google Search traffic stop at very different times on each of those three days? O was it pretty much near the end of the time period on each day?

[backdraft7]

@tedster - I can't say at exactly which times, but yes, traffic did stop periodically. I just ran my stats again to catch up and now two Sundays in a row, 1065 and 1067. I've never seen patterns this close. Looks like my site has a Denver boot firmly attached. It's not going anywhere anytime soon.

This pattern seems to be evolving into a tighter and tighter situation. I just had the worst conversion week all time, yet Google reports my traffic UP 22% and has been for over a week. The problem is, my overview graph is flat line, showing no noticeable rise.

[tedster]

Here's my thinking. If traffic shuts off at roughly the same time every day, that's one scenario - and it could possibly be an effect of mano factors. But if traffic stops after a certain LEVEL is reached, that's a very different scenario and strongly does suggest intentional throttling.

It doesn't 100% prove throttling, no - but it does look highly suggestive if the traffic stops at a certain level no matter what time of day that level is reached.

[backdraft7]

If you don't want to here about conversions, skip to next post...
Yesterday saw some real human buyers, but here's the odd part, they arrived in packs (groups).
First group came in with a conversion (sale) at 12:12pm, then another 24 minutes later at 12:36pm.
Then about 5 hours or silence...
Then at 5:15pm, another group of four stated...5:15, 5:32, 5:40, 5:46.
Then nothing for two hours..
Then the final group of two at 7:14 and 7:24pm.

I would guess this is because one of my Adwords ads popped up on a highly relevant page during those periods of times. I have the IP of the buyers, now how can I run those IP's backwards to find out where they came from? This would be a good test case to study, but I can't find a way to track individual IP's through GA.

[netmeg]

Google Analytics has never done that, by design. They have always insisted on no user-specific information in Analytics. You'd have to run your log files through an analyzer (or use a text editor that can handle big files and has a good quick search) to get that information.

Also, if you have ecommerce tracking enabled in GA, you can get that info on a regular basis - I have a set of three filters that I use that will actually display the order number next to the source and medium for each sale.

(However, if you are using Paypal or any payment gateway that takes them off your site to complete the transaction, it either won't work or would take a lot more programming, as far as I know)

[backdraft7]

@netmeg - thanks for the info...BTW, are you familiar with or do you use piwik? I'm guessing that if you use GA, you do not use piwik.

[netmeg]

Nope, never got around to messing with it.

[scooterdude]

IP information is available in piwik and statcounter

Piwik has a Goals tab which you can use to track conversions

[backdraft7]

OK, so now the stupid $64,000 question: Will removing all google tracking from my pages affect my serps? I sure hope not, but maybe, just maybe they use that as another ranking metric.

[diberry]

backdraft, I dumped GA last October. Most of my sites are still steadily improving at the same rate as before I dropped it. That one site that got Penguinized - let's just say, there are several theories about why, but none of them involve "because I dropped GA".

I really don't see how it could hurt you to drop it. Plus, if the Zombie traffic stops after you drop GA, then we'd have near-proof it's something to do with using GA. And if it doesn't, then that'll narrow down our theories a bit, too.

I personally think it made absolutely no difference to me, and will make no difference to you. I removed it not because I thought Google was using my metrics against me, but because they already have more general info about all websites and users than I personally want them to have, LOL. Plus, I switched to Clicky because it specializes in letting you follow a visitor from the time they land to the time they leave - you can practically watch it in real time. You can't beat that for getting a better understanding of user response to your site, and free GA doesn't quite offer anything like that. (Clicky has some freebies and trials if you're interested, and they don't run any search engines!)

[edited by: diberry at 5:39 pm (utc) on Sep 18, 2012]

[backdraft7]

thanks diberry! I guess it's time to make the change and get to the bottom of this Zombie issue. I'm on a dedicated UNIX box, so it should work well.

[ohno]

backdraft, we also see spurts of conversions (much like the Zombies but useful!). We don't have Adwords so it's not that. Take a look at Adfree stats, you get to see the referer string (unless SSL blocked bt Google, why would they do that I wonder?!) & IP address which you can then do some really useful digging on. I ues it in conjunction with GA, real time IIRC is also available on a service called getclicky.
RE:spurts, a few sales in the AM then dead, then sales @ 16:12, 16:17, 16:21 & 16:37 the dead again.

[edited by: ohno at 5:43 pm (utc) on Sep 18, 2012]

[Simsi]

backdraft, I dumped GA last October. Most of my sites are still steadily improving at the same rate as before I dropped it.

If it helps, same experience here.

[ohno]

Side note, doesn't it say it all that webmasters are now scared & paranoid about removing a Google product (free) from THEIR websites?!