Best practices for using the Google disavow tool, confirmed

[Robert Charlton]

Ever since Google announced its Disavow Tool, we've had a lot of concerns expressed on the forum here about how best to use the tool to clean up backlinks, how to avoid collateral damage, possibilities of negative SEO with the tool, etc.

In a recent post on his blog, WebmasterWorld senior member pontifex (aka in real life as Ralf Schwoebel, and well-known to many here) shared some information he'd gotten from Uli Lutz, of the Google Search Quality team in Germany, covering some of these questions. This is the first official information I've seen on the subject, and I've gotten permission from Ralf (who has further cleared this with Uli) to pass it on.

Questions Ralf asked included what source of backlinks to look at for disavow candidates, and could he accidentally hurt a friend or good source if he reported a link as bad. Summary of answers he got back (using his translation from the original German) are...

"I would concentrate on the links reported in the Webmaster Tools on Google"
"Do not worry about damaging other people, that does not happen"
"Be aware of the site-wide disavow possibility, it will make your life easier" have been the key sentences in his reply. I am thankful for such a definite answer and thought I’d share it here.

Additional information and blog post here...

Best practices with the Google disavow tool – confirmed
Posted on February 1, 2013 by Ralf Schwoebel
http://sno.pe/b/best-practices-with-the-google-disavow-tool-confirmed/ [sno.pe]

Thanks pontifex!

[fathom]

Ya I went back and re-read what I wrote after which wasn't what I meant.

Good catch! :)

[Jez123]

So start getting organic links elsewhere in your domain or links with different anchors to your affected pages and you'll recover.

So are you saying that google / penguin has just nulled some links it didn't like and to recover all you neeed to do is replace the links that google now ignores with decent links with varying anchors?

You don't go with the general theory that Penguin needs to be refreshed before you can escape it? Don't you think there would have been more recoveries by now if it were as simple as this?

Have you recovered from Penguin? I question as I don't know how or where you are getting your information from.

[Jez123]

Ya... Google's.

They don't mention PENGUIN in their posted information what-so-ever.

This is the reason it's hard for me to take your over CONFIDENT views seriously.

[googlewebmastercentral.blogspot.co.uk...]

No one but google "knows" for sure. You could and possibly are sending people in the wrong direction with your over zealous self belief that you are correct.

There is a lot of grey areas with google but they clearly write about using the disavow tool with PENGUIN on that blog post. What they do not differentiate is the difference between sites affected by the ALGO version or sites affected by the MANUAL version.

Thought I would over-capitalise to MAKE a point as well. :)

[fathom]

No one but google "knows" for sure. You could and possibly are sending people in the wrong direction with your over zealous self belief that you are correct.

There is a lot of grey areas with google but they clearly write about using the disavow tool with PENGUIN on that blog post. What they do not differentiate is the difference between sites affected by the ALGO version or sites affected by the MANUAL version.

Thought I would over-capitalise to MAKE a point as well.

If you forced PENGUIN problems into domains (so you can learn) you might have a track record for fixing these problems though.

[incrediBILL]

You are wrong! I never played with any link exchange scheme. Thousands websites have scraped my contents and they are ranking top on MY CONTENTS! so who played game them or me?

I never said my statement was an absolute, there will always be exceptions.

Not dealing with the scraper problem is like not installing anti-virus and then being surprised when your machine gets infected. That's your fault for not installing anti-scraper scripts, blocking some countries that are the worst offenders, and taking other anti-scraper prophylactic measures to keep the scraping at a minimum.

[Jez123]

If you forced PENGUIN problems into domains (so you can learn) you might have a track record for fixing these problems though.

Is that what you did then?

And you learned that the algo penguin basically disavows the links it doesn't like for you and you add links back to counteract? Why have more people recovered from it then? All links that I have added since Penguin have had no effect whatsoever.

[Kimkia]

So I keep getting a message in WMT about a big increase in Not Found links. When I looked, most of these, about 700 or so, come from an SEO website that "analyzes" other websites for their SEO value.

I did not ask for or pay for these links. They come from pages such as: www.exampleseosite.net/analyzer.mysite.com

All their links appear broken in WMT, by spaces or what looks like deliberate mis-forming of the URL. But the links don't appear broken when I look at their site or the source code.
Now, although these links show up in the Crawl Errors as broken, they do NOT show up as links to my site in WMT.
Can I, or should I, use the Disavow tool to get rid of these broken links?

[fathom]

Is that what you did then?

I am generally quite a #*$!y guy when I am extremely confident on the topic.

And you learned that the algo penguin basically disavows the links it doesn't like for you and you add links back to counteract? Why have more people recovered from it then? All links that I have added since Penguin have had no effect whatsoever.

Googlers give more than enough hints... some are even off-topic but are surprisingly enlightening just the same. The best example for me was this:

Since you all are speculating that Google doesn't mean what it says, it seems a bit fruitless for me to post here, but anyway ... :-)

The information on [support.google.com...] is still valid. Using the rel=nofollow microformat prevents PageRank from passing. It's useful for potentially untrusted links (eg in UGC), it's useful for advertising, and to some extent, for crawl prioritization. There are some good examples listed in this thread - such as event sponsors, etc.. They aren't taken into account when we review sites with regard to paid links or other link schemes, nor are they used for things like authorship. We generally still show them in Webmaster Tools though, since these links may be bringing users to your site. If you're seeing issues with your site with regards to paid links, link schemes, or similarly, then apart from just removing those links (if they're just embarrassing), you could add the rel=nofollow instead, especially if you feel that these links have been useful in driving users to your site.

Does that help? Anything missing?

Cheers
John

I swear on reading that one day it occurred to me that sums up PENGUIN determining the difference between an organic and inorganic link... who is the benefactor?

An organic benefits the patrons of the domain the link resides on.

An inorganic link isn't meant to benefit the patrons of the domain the link resides on it is meant to benefit the owner of the 'link to' domain and the difference defines what PENGUIN does (just like you using the nofollow attribute).

I've put that assumption to the test - many times.

I'm no expert on your domain but if you got PENGUINized for keywordy phrases stop using keywordy phrases.

[Andy Langton]

I have a different view with regards Penguin and links. For one thing, it is extremely difficult to separate out the various algorithmic changes Google makes. Even calendar dates can be problematic since many changes occur concurrently. In a way, it is unhelpful to look at calendar dates and update names, and better to look at symptoms.

In terms of Google's handling of links, my opinion is that certain link activities trigger negative handling of a site, often for specific keywords or specific keyword patterns.

The presence of those negative links is what triggers this problem. However (and it's a big however) there is a side effect of being flagged for those link activities, which is to "taint" other links to a site, even those that could ordinarily be regarded as good links to have. Even if you remove the bad links, you are potentially still left with the 'good links gone bad'. Disavow provides a means to overcome this particular problem.

If your site suffers from the effect I'm describing (and it seems fairly common) then you are not seeing the effect of lost/discounted links, but a negative effect as a result of the presence of bad links.

Of course, your mileage may vary. Look to the symptoms you see is my best advice in diagnosing any problem, whether you call it Penguin or something else.

[Sgt_Kickaxe]

I made the mistake a long time ago of embedding a link to my site, and one to the article in question, within each rss entry in my rss feed. Needless to say as spammers did their thing with my feeds I was picking up links at the same time. Undoing that is a nightmare. I've never bought links but a large number of spam pages are pointing at my site, still.

However there is a huge CAVEAT tied to that... if you didn't get notified by Google that you have a manual review for inorganic link violations - DON'T USE THE TOOL!

I disagree, I received the no manual actions taken email after a complete quality audit and various changes to my site but the incoming links causing my link profile to look questionable aren't manual in nature, they are algorithmic, and for that reason I *hope* the disavow tool helps me out here. I already know there are no manual actions against this site so I won't file another reconsideration.

GWT reports a lot of nasty links, and few good ones, but my PR is much higher than the few positive links could have provided so I am *hoping* that the nasty links reported are brought to my attention in GWT for a reason. Those were my focus with my disavow use.

I'll know over the next few weeks if this helped at all, or hurt.

[ZydoSEO]

I agree with Andy on the following with a slight twist:

have a different view with regards Penguin and links. For one thing, it is extremely difficult to separate out the various algorithmic changes Google makes. Even calendar dates can be problematic since many changes occur concurrently. In a way, it is unhelpful to look at calendar dates and update names, and better to look at symptoms.

In terms of Google's handling of links, my opinion is that certain link activities trigger negative handling of a site, often for specific keywords or specific keyword patterns.

The presence of those negative links is what triggers this problem. However (and it's a big however) there is a side effect of being flagged for those link activities, which is to "taint" other links to a site, even those that could ordinarily be regarded as good links to have. Even if you remove the bad links, you are potentially still left with the 'good links gone bad'. Disavow provides a means to overcome this particular problem.

I have often postulated that such a side effect does exist like Andy mentions where otherwise good links get flagged as bad when problematic linking activities are detected with other inbound links by Penguin. This is similar to Panda where otherwise quality content pages on your site can also be affected by Panda when other lower quality pages trip the Panda Filter.

I see Penguin as more of a "hybrid" between a filter (an algorithmic action taken against a site which when the problem that triggered the filter is corrected, the algo will automatically be lifted) and a penalty (a manual action taken against a site that requires a Reconsideration Request/manual review in order to get it manually lifted). Penguin seems to be like a filter in that Penguin is triggered algorithmically, but more like a penalty in that a Reconsideration Request/manual review is required to get it lifted. This is something new with Google, but enforcing policies by running algo's offline and then applying them through Panda and Penguin updates against their real-time results is also new.

Where Andy and I differ seems to be in what the Disavow Tool achieves. IMO Disavow is logically equivalent to taking down those links. At the moment disavowed links remain in Google's link graph, but essentially get flagged as NOFOLLOW when those links get recrawled over coming weeks after being disavowed.

But whether you take all of the bad links down manually, by using the Disavow Tool, or a combination of both... it's as if the site is STILL flagged as being under the Penguin hybrid filter/penalty. This is why I believe Google is requiring a Reconsideration Request once all questionable links are down (and/or Disavow has had time to process) in order to recover from Penguin. It likely has to be lifted manually (though it was triggered algorithmically).

Many webmasters are afraid to submit a reconsideration request because they know their site will not withstand a manual review, possibly for other reasons. Others don't know how to ID "bad" or "questionable" links so they end up not taking all or enough of the links down that triggered Penguin. So even though they might have removed all questionable links, many sites affected by Penguin never recover. They can't without a Reconsideration Request and passing a manual review of the site (not just for links, but for everything that might be a violation).

It is possible to get Penguin lifted IMO. Some have successfully done so. It's likely that many sites have had it lifted, and don't even know it because they have eliminated a lot of links that made the site rank pre-Penguin. But at least once lifted, the site has a chance of ranking again for those same phrases that they used to assuming good links are acquired in a way acceptable to Google going forward. But as long as your site remains flagged by Penguin, you likely have zero chance of every ranking for those phrases again.

[fathom]

But as long as your site remains flagged by Penguin, you likely have zero chance of every ranking for those phrases again.

So (in your theory) unflagging solves the problem. OK... well...

If you have 1,000 inorganic links and 100 organic links and your original ranks were based on a total of 1,100 links and PENGUIN detected the 1,000 inorganic ones and you lost those you have only 100 links left.

You can't get keyword links back or you'll simply continue to fall victim to the same process but if you wanted to prove that you can unflag a domain you only need to make your problem seem "really small" or far more granular than it current is... (which is something Google says all the time to many, many SEO oriented questions). So in my original premise you had 90% inorganic - if you made that more like 10% inorganic then you can easily see recovery.

But 1,000 inorganic links are powerful links because they are indeed inorganic so just getting 1,000 new organic links will not reproduce the ranks you had you would likely need to produce 10,000 organic links (guessing here since competitiveness isn't a constant across all industries) would produce a recovery impact.

That said,

Some people claim you need to remove links. (OK I'll concede that is one way)

Google suggests removal of link because they may seem embarrassing but they don't actually commit to the instructions of removal for PENGUIN to recover.

They do suggest removing the page and you won't be impact by the links that go to a non-existing page or adding a nofollow will do as well... but they also don't commit to saying these will allow you to recover... they suggest you can do it and you won't have a negative impact.

Every domain has inorganic links (you don't have control over most links developed to you - assumes you are somewhat established) and all you have to do is look at Wikipedia and pick a topic and see the massive amount of (one example) search engine optimization anchors to an SEO page to realize the reason not everyone is impact by PENGUIN is that inorganic links are quite granular on most domains when you compare them to the organic pattern... thus it isn't a problem.

The biggest issue for anyone being impacted by PENGUIN is... + PANDA and/or EMD and/or over-optimization... is a major conundrum.

[edited by: fathom at 9:01 am (utc) on Feb 6, 2013]

[BeeDeeDubbleU]

But as long as your site remains flagged by Penguin, you likely have zero chance of every ranking for those phrases again.

Personally I cannot believe that Google would automate the process of flagging a website in this way (to the extent that it would never rank again without a manual reinclcusion request). It goes against everything they have done in the past.

[fathom]

Personally I cannot believe that Google would automate the process of flagging a website in this way (to the extent that it would never rank again without a manual reinclcusion request). It goes against everything they have done in the past.

I don't either... which is also a good reason they don't do re-RUNs all that often... the re-RUNs have nothing to do with recovery... just better detection.

[fathom]

I disagree, I received the no manual actions taken email after a complete quality audit and various changes to my site but the incoming links causing my link profile to look questionable aren't manual in nature, they are algorithmic, and for that reason I *hope* the disavow tool helps me out here. I already know there are no manual actions against this site so I won't file another reconsideration.

No amount of disavowing will allow you to recover on its own.

You can't have 100% ranks and lose 80% ranks because you lost 80% of your links... then disavow 80% of your links and expect to return to 100% ranks...it isn't going to happen.

You need 80% more links which is actually like 800% more links because organic links are not as useful as inorganic "EXACT MATCH" link anchors... their benefit (organic) is being "risk free".

You're hoping for the end of the world if you believe you'll recover by adding nothing to the equation.

[Jez123]

You're hoping for the end of the world if you believe you'll recover by adding nothing to the equation.

How can you "organically" add 80% links, when not even ranking, AND avoid tripping another filter / penalty? Google has us between a rock and a hard place if you are right.

It all points to the fact that Penguin is the end and better to start again with a new domain or give up. If you are correct.

You obviously don't buy into the the idea that it's juss a penalty and come April it will be a year and they will let some of us out that have done as much as we can to fix what we are guessing to be the issues? I suppose that is what I am / was clinging to.

[tantalus]

"You obviously don't buy into the the idea that it's juss a penalty"

And theres the rub.

The Issue is whether it is a ranking change/addition or a penalty and I believe fathom is advocating the former, which I personally agree with. If true then all those links that were giving you all those wonderful ranking positions before penguin, are just not working any more. So its pointless disavowing them because google has already done it for you.

To be honest I think the whole penguin/links thing is overstated, but maybe thats because I most probably have more of a oop issue than anything else.

BTW I have recovered 90% of my traffic against this time last year. What did I do? Nothing, for the first few months, and then I just carried on as I was before, though I did widen the net a bit.

Still I am not 100% sure it was Penguin. It walked like a penguin, it talked like a penguin, but maybe it was a duck in disguise.

I do think Google has profoundly changed the way it handles citations and takes a much more holistic approach towards a site. (A bit like pageRank but without the "page" bit)

Interesting that just before penguin, masses of links were suddenly included in wmt, and now masses of links are disappearing.

"better to start again with a new domain or give up"

It depends whether you buy into the ranking idea, if you do, then using fathoms numbers you still have a 100 organic links headstart.

[Jez123]

@ tantalus so you think linking is the way out of it?

When did penguin hit you? In April last year or something later?

[tantalus]

As fathom says I'm no expert of your site, I'm no expert full stop.

But theres only one way to find out, just don't be stupid about it.

I got hit just a little bit before penguin officially started.

[flanok]

I have used the disavov tool because of the amount of scraper sites, that seem to link to me.

Or even when there are only 2 sites, the amount of links per site, sometimes 1000,s of links..

This reduces my trust value and dilutes the true diveristy and link profile.

I don't trust Google to recognise these links as autocreated by someone trying to get adsense income from other peoples content.

So I use the tool, with comments to ensure these links are not counted and hopefully the evidence is there should I ever suffer a manual penalty for such a thing

[fathom]

How can you "organically" add 80% links, when not even ranking, AND avoid tripping another filter / penalty? Google has us between a rock and a hard place if you are right.

Well first... I don't have 1 domain - I have a thousand or so. I didn't lose any to PENGUIN but I do sell domains to others for linking purposes and they (mostly) got nailed... which provided me a unique opportunity do alot of testing and fine tuning.

You have to go back to basics to solve your problem... "what is an organic link and what is an inorganic link?" If you cannot resolve these basic questions you cannot recover on your own and it would be best for you to get out of Google optimization altogether because you will simply not develop a risk free strategy.

You obviously don't buy into the the idea that it's juss a penalty and come April it will be a year and they will let some of us out that have done as much as we can to fix what we are guessing to be the issues? I suppose that is what I am / was clinging to.

Mate, I've lost more domains long before PENGUIN to outright bans at almost a thousand with a net worth of a million+... PENGUIN isn't a penalty it's a radar gun that forces you to re-think your link building practices.

Do that and you'll be fine.

You have to re-teach yourself link development or you won't solve your problems... it isn't easy... nothing worth investing in ever is.

[Sgt_Kickaxe]

Well that didn't take long.

As I mentioned in a post above I went through GWT reported links and disavowed the spam three days ago.

Today all of the disavowed links are gone from my GWT and there is no change in traffic, positive or negative.

What there is, however, is a whole new set of spammy links appearing in GWT that weren't being reported a week ago. Also curious is that NONE of my good links, the ones I know about from high ranking pages, appear in GWT.

It appears that Google had already neutralized the effects of the links I disavowed or their impact was minimal.

[fathom]

As I mentioned in a post above I went through GWT reported links and disavowed the spam three days ago.

Today all of the disavowed links are gone from my GWT and there is no change in traffic, positive or negative.

What there is, however, is a whole new set of spammy links appearing in GWT that weren't being reported a week ago. Also curious is that NONE of my good links, the ones I know about from high ranking pages, appear in GWT.

It appears that Google had already neutralized the effects of the links I disavowed or their impact was minimal.

It was reported as a bug by Google (massive drops in shown links in WMT).

You'll never make a change and see any impact in WMT in the same week (rarely in the same month)... adding disavowing is simply forcing a nofollow like attribute to disavowed links... they will always be shown in WMT until they are physically deleted off the resident page.

[diberry]

To be honest I think the whole penguin/links thing is overstated, but maybe thats because I most probably have more of a oop issue than anything else.

Penguin is not just about links, that's for sure.

[RockyMtnHi]

The whole concept of chasing your tail (Google's algorithm) to include disavowing links is a fools errand. There are more ways to bring in relevant traffic.

If Google's algo allows a competitor to spam your site with bogus links to take you out then they are doing a disservice to those honestly trying to make a living.

It doesn't matter though because their motivation is income for Google, not those trying to make a living.

If they truly wanted to help these people then they'd change their algo to just ignore bad links, but I'm not holding my breath.

[edited by: Andy_Langton at 5:10 pm (utc) on Feb 13, 2013]
[edit reason] No links please, see charter [/edit]