"ZOMBIE TRAFFIC" Separating fact from fiction & emotion

[FishingDad]

This recent discussion about "ZOMBIE TRAFFIC" is just utter nonsense. What are people saying, anything worth while or just a communal <snip> because sales are down on the norm? The talk is firmly in the tin foil hat area.

Are you talking about SERPs, if so why, if your positions are dropping then that's that. If positions not dropping are you seriously saying Google is sending you people they know will not buy from you !? REALLY?!

Are you talking about PAY PER CLICK? if so then your talking possible click fraud then, aren’t you?

Giving any constant period on the internet, people buy or they don't buy and there's many many factors why they will one day and might not the next day.

[edited by: goodroi at 5:55 pm (utc) on Nov 10, 2015]
[edit reason] Let's be careful to keep the discussion on a professional level [/edit]

[ecommerceprofit]

Maybe someone already brought this up but Simon_H made me think of this in his recent post. Type in "google testers" - I saw a Google testing blog and all sorts of stuff (testing phones, testing this and that - must include search engine too) - his theory is quite sound and tedster saying years back about quotas...wow...how irresponsible if this was true...we are little guinea pigs...absolute power...testing is fine but to hold the same site in a cage forever is so wrong.

My kids have a hamster...I feel bad for it...my wife and I take care of him and try to provide love (even though we did not want him) because it's so cruel to forget about a dependent creature. Glad I made my cash years back or I would be really depressed right now...feel bad for other people affected by zombies who are not as lucky as I am.

[Simon_H]

Thanks @ecommerceprofit. That's a good point. Google has a 'trusted tester' program, where a community of testers are involved in beta testing search. You can even apply here: [docs.google.com...]

Does anyone have any inside information on this? I'd guess this group tests on live rather than all having access to internal Google servers?

Although I didn't mean to suggest in my original theory that all live testing is manual (although undoubtedly some is given the above). Google would most likely also use extensive automated testing, where pages from a set of test sites are dropped into alternative serps and the impact measured. This is similar to what @robert_charlton proposed earlier in this thread. A Google engineer could potentially trigger this on a particular day or over a particular period, which may explain why multiple sites see zombie traffic on the same days.

[vlexo]

@Simon_H I remember them asking people to sign up for a beta test in Oct 2015 through a Google Docs form - you can see one of those tests here: [deepcrawl.com...]

It was for App Indexing and a beta test in Search Console around app installs from Google search.

[dipper]

Simon_H - but why would this testing "zombie" traffic you speak of replace current "converting" traffic? - are you saying that Google also shape your traffic, and simply exclude regular visitors? - wouldn't this then directly be visible with a loss in rankings, and hence natural traffic?

By Andrey Lipattsev saying they test on live this could also mean that they monitor the results of the natural traffic and that is their test. Or it could mean any number of things. I do find it hard to believe that zombie traffic would replace natural traffic, and all without noticable loss of (obviously visible) SERP rankings. You should delve into that, and into analytics to better understand the natural converting traffic and the zombie traffic - where does it come from, how does it differ, how many pages do they visit, their actions and differences etc etc - there are a number of live website visitor monitors, that would allow you to watch each visitor to your website, see what they do. You might be able to use something like this.

[Simon_H]

@dipper Traffic quotas/shaping is something that's been talked about for a long time; it's not something I've come up with. Many much cleverer people than me have proposed it in the past. If it's real, it most likely happens on the very long tail, which makes it near impossible to measure as it will have minimal effect on shorter tail rankings, whilst allowing it to affect a big percentage of a site's traffic. The cumulative traffic from the longer tail keywords forms ~80% of our traffic, so if Google wanted to try switching us in and out of different longer tail serps, it absolutely would have the effect of changing the quality of traffic without necessarily changing the volume.

[Simon_H]

@dipper I'll also answer the question of "Surely zombie traffic would be seen in analytics as having higher bounce rate, lower time on site, etc and so could be easily identified/measured?"

The answer is that zombie traffic would most likely appear almost indistinguishable from non-zombie traffic. Here's the maths. Let's say that good (non-zombie) traffic converts at 2.5%. Whereas zombie traffic converts at 0.5%. After all, in the scenario being discussed, zombie traffic is still real users, just users who've clicked on your site from a serp where your site probably shouldn't have appeared. This means that 97.5% of non-zombie traffic will navigate the site, fail to find what they want and leave. Whereas 99.5% of zombie traffic will navigate the site, fail to find what they want and leave. The 97.5% of non-converting traffic will behave almost identically to the 99.5%. Therefore, on average, user metrics will be almost identical between zombie and non-zombie traffic.

[mrengine]

And that's a huge assumption of yours that all test subjects should now see normal traffic. Google carries out frequent updates to the core algo, both minor and major, which would need testing at any time. Not to mention that Penguin is due imminently. So I disagree that Google no longer has a need for algo testing.

Simon, I never said that Google has no need for algo testing, so I'm not sure where you got that from. I also don't think it's a large leap to say that if testing was the reason why people were seeing zombie traffic that they should be experiencing normal traffic now. Whatever that normal is should at least but somewhat stable and not produce the massive difference in conversion rate patterns we see with zombie traffic. Google just released a major update, and if the zombie sites were test subjects then those tests for that algorithm change are over. Or are you suggesting that the test subjects will be subjected to perpetual tests, and Google zombie traffic, that adversely impact their ability to compete in a fair marketplace?

[Simon_H]

@mrengine Everything is speculation, so I'm very possibly wrong! But have a look at the conversation between @robert_charlton and I near the beginning of this thread. The theory is that Google keeps a test set of sites and uses them when it needs to, not only during big algo updates. We've seen unnatural zombie spikes in our reports every week or two for up to a year, but it went nuts from the middle of September 2015 at the same time other people saw this. As of late October 2015, the turbulence died down and we returned to spikes every week or two. Then, between the two recent big algo updates, conversions plummeted but traffic stayed the same. Now things have almost returned to normal, but still seeing some oscillating.

Whilst I think this theory makes some sense (Google *does* test on live and I believe they must formalise testing through a set of test sites resulting in test clicks), this theory doesn't properly explain why we and others see zombie traffic on paid results. That's more likely to be human testers or botnet testers clicking on paid by mistake, but even then, that doesn't seem a good enough explanation to justify the huge number of 'bad' clicks.

[mrengine]

Simon, anything is possible and the possibility that zombie traffic is related to testing may be accurate. If zombies are related to testing, it is very harmful to those test subjects. To go from consistent and stable sales one day to days on end with few or no sales over the course of months is not something many smaller businesses can sustain for very long in the current competitive landscape. For many, who are still seeing zombie traffic, these tests may instead test the survival of their businesses.

I would hope Google would understand that live tests may harm the test subjects, and limit the duration of the tests, but Google has pulled some bonehead moves in the past. This is why I don't think it has anything to do with testing. It's been going on for far too long to be testing. I think it has more to do with showing our sites to a different set of users, many of whom are much further away from having buyer intent then we have seen in the past. By showing our sites to different users, it could impact both paid and organic results in the form of zombies.

[Simon_H]

@mrengine Totally agree with first paragraph! But Google has a dilemma. They've admitted to experimenting on live, and that will surely involve clicking on results in the serps, because rankings are affected by at least some aspects of user behaviour and so that would need testing. Which means there will be sites getting test traffic and believing it's real traffic. This may not be the zombie phenomenon, but I do believe this test traffic happens.

I think your theory is not dissimilar to the testing one. With both theories Google is intentionally manipulating traffic to a subset of sites for its own benefit. Whether it's doing this to simplify/reduce investment in its own internal testing or to favour some sites over others to generate additional revenue doesn't make much difference. Neither appears more ethical than the other.

If we assume that zombie traffic affects both organic and paid, then the only explanation I can think of is that it relates to pre-query processing, i.e. RankBrain-type stuff where ambiguous/unique/new long tail queries are rewritten, as that could then be passed to and impact both organic and paid algos. This theory has also been mentioned before, but the problem with it is that it would affect all sites in those long tail serps, not just a subset. Hmmmm.

[ecommerceprofit]

Mrengine and Simon_H - you both nailed it and I'm in agreement. I wonder if there is a way to get Google's attention in this matter? There does not really seem to be a way to write anyone with power to fix. I remember Matt Cutts saying his e-mail is almost impossible to read now...regarding legal action...not really sure this is possible - over the years I have read people saying...hey guys...lets all get together and take on the big company...whoever that company is...never works out...no one really follows through legally or has the power unless they devote their life to the cause through ambush reporting, etc. I don't have the time...or we just hope that some company as big as Google gets irked and gets Washington's attention with their lobbyists...wishful thinking it would happen anytime soon...ughh...

Disclaimer: I have no idea if Google is doing this on purpose...could be just an accident...or it could not...

[Simon_H]

I think @dipper suggested this... The first step is to ask JM/Gary/Matt exactly what testing is done on live, which sites are tested on, how often is it done, how long does it last, etc. Hopefully they'll reveal at least something about what is done.

Coincidentally, Matt just did TWIG 336 (see SER where I've already commented) and spoke around 1:10 about testing. He says that Google initially tests algo changes in a sandbox and then tests on live. Kevin Marks also says they do extensive experimenting on live. My interpretation of Matt's statement is that they test on live *before* full global deploy. Which would support the idea that a subset of sites are getting impressions and clicks from Google testers (human and botnet) as well as the fall-out from real users who would also see those changes on live.

[samwest]

It really is a ridiculous pattern that I'm again seeing.
Around the holidays things settled down a little with more natural daily results. But now near month end, it's back to total conversion blackouts on the same volume of traffic. The other day I had 5 conversions in one hour, then nothing all day. Today I had one at 5 am, one at 6:15 am and then nothing all day...how does that even happen? It stinks so badly of "testing", "manipulation" and "throttling" it's not even funny. Turning Ads on has no positive effect. When it converts, it converts! Other wise, it's a ghost town or tire kickers.

[masterjoe]

I've had a significant increase in conversions the last few days. Daily conversions - no on/off behavior. Hopefully this sticks - I also moved up on a money keyword to #1/#2 which could have something to do with it.

Side note: I also see that some of my old penguin hit site keywords moved up quite dramatically, perhaps Penguin is finally dropping?

[ecommerceprofit]

Zombies are gone today - we are way up in conversions...this is crazy...up so much that we cannot get orders out fast enough then will go down again soon to ghost town...Google has jumped the shark...so wrong...

[ecommerceprofit]

Utah attorney general may be looking at Google...just posted by Bloomberg 2 hours ago...wonder who will be providing evidence...they can decide because I have no idea and cannot say what is happening is good or bad. They have the resources to see...I may think Google has jumped the shark but who am I to know for sure.

[glakes]

Good conversions today. I doubt this will last though. Google throws a dog a bone every now and then, so I'm expecting the rest of the week to be more zombies. I'd love for Google to prove me wrong, but the zombie phases I've seen suggests otherwise.

[IanTurner]

Let me throw in a theory regarding zombie traffic - maybe it has nothing to do with Google.

So look at the possibility of pharming (DNS cache poisoning) and then sending the resulting click stream back to the original site via a botnet (minus of course the conversion clicks)

This would explain no change in overall traffic to a site (and make it much less detectable than pharming without the click stream return - which would show as times of significant traffic drops)

It would explain why only some sites are affected - as the pharmer would need to create a copy of the original site by spidering it (and for some sites this will be more difficult than for others)

It would explain the occasional times of good quality traffic - which would be when the poisoned cache(s) reloads from the DNS source.

[Nutterum]

@IanTurner, how can one check for DNS spoofing? Lets say you are correct and I have a possible Zombie site I want to check. Can you provide some easily digestible resources I can use?

[Simon_H]

@IanTurner I'm just trying to get my head around this! So, you're saying that a user clicks a link to a site (either from Google organic, Google paid or anything else) but that user is on a compromised DNS server so they get taken to a different 'mirror' site instead. So the pharmer is hoping the user will try to buy something and enter their credit card details thinking it's the original site? If so, what would the purpose be of having a botnet navigate the original site? Would that be purely to keep the traffic numbers up to avoid suspicion?

This theory would definitely explain the patterns being seen, but only if virtually every DNS server was compromised. Because the zombie phenomenon is very pronounced (certainly for us) where conversions drop massively, whereas if just the odd DNS server was compromised, then surely the effect would not be particularly noticeable?

[Andy Langton]

how can one check for DNS spoofing?

As a site owner, I'm not sure you can, since it would be your users who are compromised. You could test the security of the major ISPs serving your traffic for potential vulnerabilities, but this alone won't tell you a lot.

One way to get some idea might be to see which ISPs/hostnames appear during zombie/non-zombie episodes to see if there are any patterns.

[IanTurner]

Okay first try reading the wikipedia entry on pharming as good introduction to things.

Yes the botnet would be to lower suspicion and just keep the traffic numbers up. I don't think the pharmer would take credit card details, just send user via affiliate link or cpc ad to another site. If you had significant drops in traffic for an x hour period it would be much simpler to investigate the phenomenon

If you could compromise the originating primary DNS server it would affect all traffic, how close you get the compromise on the downstream side of the nameserver would effect how large a chunk of traffic or affecting a large ISPs DNS cache could have a significant effect.

(I am just proposing a theory that could explain zombie traffic - I don't have any proof yet)

[masterjoe]

I am not a very technical person, so most of what Ian just said went right over my head... however, I really doubt this could happen since everyone is using different servers and clearly operate in vastly different areas of the internet world. To be on as large a scale as what we are seeing (conversions at roughly the same time), there has to be something afoot at HQ. There is no way a bot could interfere with so many different websites which have nothing to do with eachother. Any theory that helps us understand this is great though.

[Simon_H]

@IanTurner Cheers. I have a pretty good understanding of pharming, just wanting to understand how the theory fits with what we're seeing. You initially mentioned that the pharmer would need to create a copy of the original site through spidering, but if they're purely going to take the user via an affiliate redirect or similar to another site, then they wouldn't need to create a site copy. I assumed the site copy theory was because you were proposing the user would be encouraged to enter payment details on the fake/mirror site.

Compromising the primary DNS server would mean that pretty much everyone would get directed to the wrong site and so one would expect this to be reported very quickly by a previous customer or suspicious user. We'd even see this ourselves when trying to visit our own site. I guess it could be argued that our local DNS cache may still point to the correct site so we wouldn't see this, but perhaps one way to tell would be to clear our local DNS cache during a zombie period and try to access the site?

[IanTurner]

Yes you wouldn't need to do the copy if you were just sending people via a redirect, however having a copy of the site would prevent it being spotted by a cursory inspection.

Even compromising the primary DNS server wouldn't necessarily show the wrong immediately to everyone because of caching - it would take some time for the bad info to propagate.

[Robert Charlton]

Without getting into a lot of detail right now, as it's late, I think that Google might be doing a lot of testing to calibrate types of sites that would satisfy different kinds of user intention.

Complete conjecture... they'd be using heuristics perhaps seeded by initial evaluations from their 2015 Quality Raters Guidelines, which certainly describe a lot of different ways of classifying sites, and I can imagine a lot of testing going on.

There are likely several kinds of tests... specific tests on queries to determine characteristics of sites that might belong in each group... and then more general tests of the algorithms developed from these heuristic tests. Again, methodology might follow the scenario I outline in my Nov 17, 2015 post in this thread... and again, I recommend reading (or rereading) this Wired interview by Steven Levy, with Amit Singhal and Matt Cutts...

TED 2011: The 'Panda' That Hates Farms: A Q&A With Google's Top Search Engineers
Steven Levy - 03/03/2011
http://www.wired.com/2011/03/the-panda-that-hates-farms/ [wired.com]

Note that Google's new VP of Engineering, replacing Amit, is John Giannandrea... and one of his big concerns is the need for disambiguation in language, or making more precise the meaning of words. He came from Metaweb and Freebase, where speech recognition was his holy grail, and he became Google's head of machine learning before moving to his new position.

I don't have enough inside knowledge to know how precisely this is going to impact decision tree testing, or where this testing enters public search... but I think it does in several places, and that this is what might produce zombie artifacts. My further guess is that there's a lot of it going on now, and that this is why people may be seeing more zombie type traffic.
</end conjecture>

PS: I think that the appearance of zombie traffic correlates so closely with the beginnings of what Matt and Amit describe as Google's reaction to the shallow content that came with Caffeine, that it's much more likely to be some related testing, and much less likely to be DNS spoofing.

[Simon_H]

@Robert_Charlton The issue with all zombie traffic theories to date is that they don't explain all the symptoms. Starting with what is being seen and working backwards...

This seems to involve (1) traffic volume remaining constant but traffic quality switching in and out, (2) this switching can happen anywhere between once per day up to once every week or two or longer, (3) applies to organic and paid. I'd expand (1) further and say that we have a sufficiently high number of sales that we don't just see conversions switch on and off, we also see the *type* of products bought suddenly change dramatically.

Assuming that this isn't bot/DNS spoofing and that 'zombies' are indeed real users, then I think it's fair to say that the zombie effect is the manifestation of affected sites being switched in and out of certain serps, such that the total daily number of impressions (and hence traffic) remains relatively constant. And because I'm not aware of anyone seeing this effect in the rankings of keywords they monitor, it seems most likely that Google is doing this on the longer tail searches. So Google will try showing a site in a particular set of longer tail searches and will then switch it to instead show in a different set of longer tailer searches. The first set may convert well, the second may not. Or perhaps both will convert well, but result in different types of product selling. Hence, the zombie effect.

The reason why certain sites are being switched in and out is unclear, but it could certainly be due to Google testing on live. However, this doesn't explain:

1. Why this also affects paid results (and it certainly does!)
2. Why this appears to hit only certain *sites* rather than certain categories of search.

Starting with 1, the seemingly obvious explanation is that this effect is pre-query processing. So Google is taking ambiguous long tail queries, translating them into something more understandable (e.g. Rank brain-type stuff) and then feeding the results to all widgets on the page, including paid. And this translation is constantly switching and adapting, hence the zombie effect switching in and out. But that doesn't explain why this hits only certain sites, because if it was pre-query processing, one would expect it to hit multiple similar sites at the same time but that doesn't seem to be the case or there would be a lot more reports of this.

To try to answer that and point 2 above as well, perhaps this *is* hitting all sites, but the impact differs massively between sites. Let's say a site gets 70% of its traffic from these longer tail/obscure queries (e.g. it's been hit by Panda and/or Penguin such that it barely ranks for any competitive keywords). For those sites, the impact of the zombie effect would be enormous. But for a site that ranks for multiple competitive keywords and hence only gets 10% of its traffic from the longer tail/obscure queries, the zombie effect may be barely noticeable. A while ago, I asked if zombie sufferers had been hit with Penguin and/or Panda and nearly all of them said they had. (Although, to be fair, some believed they hadn't.)

So, this all appears to explain things. Except... it still doesn't explain why the effect is not seen far more on paid results, because pre-query processing should impact all sites related to the certain queries and not just some sites. And that's where I'm stuck!

[Simon_H]

Sorry for the multiple posts, but @Robert_Charlton's conjecture may be further validated by the recent hangout with Google Gary and Eric Enge. Eric asked Gary what is happening with Penguin, and he alluded to the fact that Penguin fine tuning is taking far longer than expected and Penguin may be even further delayed past Q1 2016. (Expect to read about this on Monday on SER, etc! It's at approx. 29 minutes in.)

The point is that Google has confirmed they're in the middle of extensively testing Penguin on real sites and multiple Google staff have also recently stated that they test on live. (It's doubtful they have a mirror of the entire web on test server so they have no choice but to test on live.) So if Google does use a test set of sites, those sites will have potentially been seeing on/off zombie traffic from late 2015 up to now.

[IanTurner]

I'm not sure that penguin related changes would create zombie traffic - that is much more likely to create the random fluctuations up and down in visitors that I have been seeing over the last six weeks.

I still think zombie traffic is either Google experimenting with long tail meaning and 'query groupings' or dns spoofing.

I really have difficulty explaining away the 'lack of fluctuation in search visitor numbers' and the 'paid search is also affected' when considering Google testing long tail meaning as an explanation for zombie traffic.

Though on the other hand there has also been talk of people keeping quality traffic and conversions from Bing.

[Simon_H]

@IanTurner Good point. I think I'm wrong on the Penguin side of things. My thinking was that Google was doing unit testing on facets of the new algo rather than fully rolling out new Penguin to a subset of sites, but even with that line of thinking, it still wouldn't produce the zombie traffic we're seeing.