Negative SEO Is Alive and Well

[NYCTech]

I've posted about some of the issues we've faced since roughly the rollout of the "Fred" algorithm in early March. However, we weren't really hit until March 15th, which always seemed odd. We did tons of analysis and fixed everything that could potentially be a problem, but we still weren't seeing recovery. We considered that we may have to wait for a refresh, but that also didn't totally make sense - why would the drop have been so large.

At the same time, we'd found a number of very fishy links to our site, all mirroring pages that linked to other pages within the site. These links came from totally unrelated sites, many of which were hacked. The others were all registered via proxy and clearly spammy. We'd also seen spikes in Webmaster Tools related to "Time spent downloading a page" and in GA related to Site Speed. These perfectly mapped to when traffic declines hit.

We then checked our server logs and found that what appears to be a DDOS attack, with a "Slowloris" component, was to blame. These were launched from other countries (presumably via proxy servers) which changed IP addresses. They were also coordinated with Googlebot crawls so that the slow responses and 500 errors would happen when Googlebot was crawling the site.

All of this worked quite effectively and we've lost more than 85% of traffic. We are now implementing changes to protect against these attacks, and I will happily share results when, hopefully, we recover that lost traffic over the next few weeks.

[goodroi]

Really sorry for your trouble. I am a bit confused and not sure I understand this fully. How did a third party know when Googlebot was visiting to coordinate the attack?

Did you lose all 85% of traffic on March 15th? Is this all traffic including direct traffic or just Google referrals? What date were the spammy links added? What date did the DDOS start?

It has been 6 weeks since your traffic loss and you are just now implementing changes. I am guessing you might not have too many resources and don't want you wasting your limited resources.

[NYCTech]

Traffic dropped a lot on the 15th, recovered a bit, dropped again, et cetera. It's overall been a sort of downward trend, though not a stable one. Each drop follows a DDOS attack.

I think they knew when to launch attacks based on the cache time/date for pages in Google search results. It's easy enough to see when pages are cached, find those patterns, and then attack at those times. It would be easy enough to build a tool to scrape the SERPs in real time as well to confirm an active crawl is happening, especially since Googlebot crawls tens of thousands of pages on the site every day, usually at similar times.

As for the 6 weeks, we implemented many changes earlier on based on flaws we found in the site (things that could, possibly, have been Fred/Panda/Phantom related), it's just that we hadn't recognized this as an attack until more recently. It wasn't all that obvious - we'd see that the server was running slowly, do a manual check, and it seemed fine, so we dismissed it. The attacks were clearly done in bursts and generally during crawls, which made it harder to catch, as well. Resources are somewhat constrained, though not so much so that we weren't able to fix lots of other things - we just didn't think we were big enough to warrant an attack that was clearly targeted to specifically exploit flaws in the site (ie links to our site that encouraged Googlebot to crawl and index pages that only exist because of the addition of strange parameters or pagination beyond that which actually exists, but which are substantively the same as those same pages without such parameters, creating duplicate content).

The spammy links actually started in the summer of 2016. That said, we have tens of thousands of incoming links, so we didn't bother to manually check each of them. Many are also very high authority and trustworthy, which is why I don't think the spammy links had so much of an effect (Google trusts the site and probably ignored these for the most part - we have gotten coverage and links in articles from some of the most respected sites on the internet). It was really once the DDOS attacks kicked in, in force, that we saw the rankings drop, which did also coincide with an increase in the rate of spammy links (apparently they stepped up the attacks against us shortly before this happened).

[goodroi]

Sorry but I am still confused. You are not describing typical Googlebot behavior. Googlebot doesn't really care about precise timing when it visits a page especially when we are talking on the scale of tens of thousands of pages. Googlebot usually spreads out its crawl throughout the day when its crawling big sites because doing high volumes in a small time window could trigger an accidental DDOS.

Theoretically if you are hotlinking images hosted by another webmaster, they could tell when Googlebot is visiting and know when to launch a DDOS. But a slowloris attack is generally a slow attack that needs time for full effect so it doesn't make sense to me as you describe it as a burst attack.

Oh well don't mind my confusion, good luck with your recovery regardless if its negative SEO or Fred/Panda/Phantom or something else :)

[NYCTech]

I hadn't thought about how one could tell by hotlinking images, but, yes, some of the spam sites mirroring ours did so via hotlinked images.

When I am describing the bursts, I mean days (or parts of days) during which the attacks were active - ie they didn't attack every day, usually just once or twice a week. We looked over the last two months at requests that took longer than 95% of other requests. More than 99% of those requests happened on 8 days. On 7 of them, they managed to match when Googlebot crawled well enough that there were big spikes in both "Time spent downloading a page" and "Site Speed" in Search Console and "Speed Suggestions" in GA on those same days. In our case, due to the way our site is set up, we don't time out requests for a very long time (there are places where people can upload images, and 2 GB images are not uncommon, which can take a while), so these requests could go on for minutes, and there could be lots at any given time.

I don't know when these attacks began, but it looks like they probably started in February (possibly sooner, but with less ferocity), so they had some time to take effect, but it was mid March when the big drop happened, and that seems to correspond with them increasing the frequency and scale of the attacks quite a bit.

[brotherhood of LAN]

What kind of hosting are you using, shared/VPS or dedicated? What kind of webserver. Apache/NGINX or something else?

I'd look to mitigate that risk to your site (and its rankings) which would seem to solve your suspicion that this is causing a loss in rankings.

A lot of setups nowadays have NGINX as a front end (which can handle tens of thousands of requests quite trivially) and Apache at the backend for compatibility (mainly for anything that requires .htaccess).

Investigating logs should confirm for you a slowloris type attack, and various Apache modules can also help take care of anything that's grabbing an inordinate number of connections to your server.

Certainly, a client grabbing all your resources and starving Googlebot and others users of connections would cause you issues. I wouldn't quite call it 'negative SEO', though.

[aristotle]

They were also coordinated with Googlebot crawls so that the slow responses and 500 errors would happen when Googlebot was crawling the site.

This seems implausible to me. There can be other reasons for a slowed-down server than the half-hearted DDOS attack that you conjecture.

I've had a lot of experience with slowed-down servers over the years. I have six sites, and have tried at least 5 or 6 hosting companies at one time or another. On a shared server, one bad- acting site can slow things down for everybody. Even though servers may have improved, my GSC always shows a lot of load-time spikes for googlebot crawls. But I've never seen temporary periods of slow googlebot crawls have any effect on rankings or traffic for any of my sites.

[dipper]

@NYCTech - let me guess, adult toy/business related website?

[mosxu]

@NYCTech
how is the recovery going? are hackers still slow downloading your site to affect your rankings?
how did you change server settings to protect against slowloris attacks?

[martinibuster]

we implemented many changes earlier on based on flaws we found in the site (things that could, possibly, have been Fred/Panda/Phantom related),

Theories about Fred and Phantom Have Been Discredited
All the theories about "Fred/Phantom" were wrong. This is a fact. Those were not updates at all and they had zero to do with finding "quality" issues with websites. This has been confirmed out of Google. What happened were run of the mill changes to the core algorithm, changes that happen on a near daily basis.

Difference Between Negative SEO, DDOS & Exploit Bots
What you're describing is not negative SEO. It could be described as a DDOS attack but some or all of it is more likely a standard exploit hunt. That's a bot testing for vulnerabilities in your server and CMS software as well as a password guessing attack. This is very common and happens to any site with even only a few inbound links. Many of these attacks originate in Eastern Europe and various countries in Asia.

Negative SEO is a practice that originated with the gambling industry in the mid 2000's. Is your niche as competitive as the gambling niche? If so then it could be negative SEO. If your niche is not as competitive then the likelihood of negative SEO is infinitesimal.

Red Herrings
In story writing, a red herring is a clue that appears to explain an event. Usually the clue points to an obvious suspect or reason. In life as in fiction, what is obvious is not always the true reason.

Your situation is a textbook example of latching on to invalid theories that once addressed do not solve the problem. You are likely looking at red herrings, especially with the now discredited Fred/Phantom "symptoms."

The first invalid theory was the false Fred/Phantom issues which have been shown to be completely wrong. The second invalid theory was the notion that you're under a negative SEO attack. "Fixing" both issues won't solve your problems because they're not the causes.

It's possible your site is not suffering from those attacks. Check your Google Search Console for crawl errors. If you see those then it is possible that the exploit bots are indeed slowing down Google's crawl. Rogue bots and Google routinely crawl in the early morning hours because the crawl impact is felt less at those hours.

The rogue bots are not coordinating to slow down your Google crawl, it's just a coincidence. Crawls are more efficient when a site has less traffic, that's why bots tend to crawl after midnight in whatever country the server is based.

Solution
Highly likely that addressing the false problems will not solve your ranking issue. The problem lies elsewhere. Find the real reason, look to other issues with the site. It could be software/coding technical. It could be you're already hacked. It could be SEO. Those are the three major causes for a drop in ranking.

Good luck,
;)

Roger Montti

[NYCTech]

@mosxu - So far, we are not recovering, though the speed issues seem largely solved.
@martinibuster - You may be right that some of the problems we identified were red herrings. I have no way of knowing. As for our niche, yes, it's highly competitive and lucrative. It's probably not quite as competitive as online gambling, but it's up there.

When we really began looking into speed, what we saw in Google Analytics was far worse than Search Console, and we started measuring speed using our own code and saw results that seem inaccurate. These may have been designed to feed bad information back to Google. We also definitely had an index bloat issue, with Google indexing about 5 times as many pages as actually exist on the site, and most of the additional ones (when I've used the site: operator in search) were duplicates of thin pages on the site. We've largely cleaned that up and the number of pages has dropped massively, but there's still bloat.

I'm hoping we're going to recover soon, but we'll see.

[NYCTech]

I should note that there are other reasons to suspect negative SEO. We've also found thousands of links we certainly didn't create listed on sites that were pretty clearly hacked and loaded with spam. We've also found a few hundred pages that are displaying content scraped from the site. We've disavowed those links and tried to block some of the scraping bots, but there certainly seems to be a pretty concerted attack on the site coming from somewhere, as those sites displaying our scraped content and links on shady/hacked websites didn't get there all by themselves.

[mosxu]

@NYCTech

I think you have got something. There is a loophole in the system whether left internationally or not it is hard to say.
It is probably a combination of CTR manipulation and slow loading your site that becomes lethal. If you look to my previous comments I have already pointed out that the gambling is very active in CTR manipulation and you pointed put the slow loading speeds for certain IPs which adds up.

I would not worry about duplicate content or else as long as the server is secure it should all be fine.

[seoskunk]

Neg Seo and DDoS attacks and Exploit - different in terminology sure but usually the same person.

[Writerly]

I got a little bit confused here. Aren't both DDOS & Exploit Bots practices of using different unethical methods to ruin the competition’s online visibility and reputation. If that is the case, aren't they the synonyms to negative SEO, or, if you prefer, its components?

[martinibuster]

Aren't both DDOS & Exploit Bots practices of using different unethical methods to ruin the competition’s online visibility and reputation.

No. That is one of the misconceptions I was trying to fix in my post above.

1. DDOS and Exploit Bot activity can be confused with each other because the felt effect can be similar. The intent, methods and goals are different. See the next point.

2. A DDOS is an attack against a website. A Bot Attack can also be a hacking event. The difference between the two is intent and methods.

3. The intent of a true DDOS is to take a site offline. It is generally done as an act of protest or revenge. Perhaps in some cases it is competitive sabotage but legitimate businesses do not conduct themselves in that manner. The method is generally by driving so much artificial traffic to a site that it overwhelms the capacity of the server to respond, rendering the site offline.

4. A Bot Attack for hacking purposes is more common than a DDOS attack. These attacks are looking for vulnerabilities to exploit for monetary reasons. They want to serve rogue ads, they want to use the server to infect PCs to steal passwords and credit cards etcetera.

If that is the case, aren't they the synonyms to negative SEO, or, if you prefer, its components?

As you can see above, that is not the case and they are not synonyms of negative SEO.

[seoskunk]

No. That is one of the misconceptions I was trying to fix in my post above.

I appreciate the sentiment but your wrong, the point is someone engaged in Negative SEO that sets out to damage a website will use all tools at their disposal. Including those not defined by you as neg seo. Negative seo is a broad term referring to person or persons wishing to damage a sites serps position and all of the things listed have that outcome. There is no true intent other than damaging the target website. There are another 5 items I would list under neg seo as well but I don't want to create a shopping list.

[martinibuster]

...the point is someone engaged in Negative SEO that sets out to damage a website will use all tools at their disposal.

The point is that in the real world, for millions of sites worldwide, hacker bot activity is hacker activity, period.

In the real world for millions of websites worldwide they are not experiencing an attack launched by a competitor.

What you are discussing is not the experience of millions of web publishers every day, which very likely may include your own website at this very moment in time as you read this. Hacker bot activity is that prevalent and widespread. Surely you know this?
;)

[seoskunk]

The point is that in the real world, for millions of sites worldwide, hacker bot activity is hacker activity, period.

In isolation I would agree but when you get a combination of hacking, scraping, DDoS, toxic links, XSS and more it becomes Negative SEO.