WEBINAR: Google Penguin 4.0 is live - Real Time Penguin Insights and Outlook
Since Google made various statements following the launch of the Penguin 4.0 update, the controversy around Google Penguin 4.0 increased. Join the webinar and have Christoph C. Cemper - the founder of LinkResearchTools (LRT) and Link Detox answer all your questions.
Common mistakes and how to do it right
The Google Disavow Tool has been out for years and while webmasters rarely used it in the beginning, today nearly every SEO heard about it or used the tool at least once.
The disavow tool is surrounded by many myths and conspiracy theories: the tool doesn’t work, the tool has a negative impact on a website’s rankings, Google is using the tool to hunt down SEOs. We have heard them all.
This article is not about some SEO conspiracies, but rather about the right way to use the Google Disavow Tool. Why? Because it works!
However, we see common mistakes of SEOs every day, not knowing that disavowing the wrong links can cause severe damage to the rankings.
Read this article and learn:
- When and how to use the Google Disavow Tool
- How to create the perfect Disavow file
- How to do a Disavow File Audit
- Extra tips about disavows
Christoph C. Cemper
Here's what you can read further...
- Common mistakes and how to do it right
- The introduction of the Google Disavow Tool
- Google ranking factors
- When and how should I use the Google Disavow Tool?
- 1. Identify the right links to disavow
- 2. Create your Disavow file
- 3. Upload your Disavow file
- 4. Boost your Disavow file
- 5. Constantly Review Your Disavow file
- 6 Extra Secrets about Disavows
The introduction of the Google Disavow Tool
On October 16, 2012 – only a few months after the release of the first Google Penguin, Matt Cutts from Google proudly announced the release of a new tool, “the Google Disavow Links Tool.”
Its introduction has led to heated debates among SEOs around the world, and many preferred not to use it in the beginning since Google stated that the “vast, vast majority of sites do not need to use this tool in any way.”
The release of the Google Disavow Tool generated controversial reactions in the SEO world.
Many SEOs were scared of using the tool, since Google made clear that using it can do you harm if used incorrectly – which is true. We will discuss the importance of the correct use later on.
By introducing the Disavow Tool along with harsher Penalties, Google managed to shirk responsibility, dumping it onto webmasters. And in the light of the rising trend in negative SEO, every webmaster is, sooner or later, likely to get to a point where there is just no way around using the Google Disavow Tool.
Especially in the era of Penguin 4.0 the need for a regular Disavow has risen, even if there were rumors saying that Google now has less need for users disavowing bad links. Like with all fake news, the rumors tend to spread widely while the clarification does not reach as many readers, making it possible for the rumor to live on. By no means is the Disavow Tool useless. Google is still not able to find all spammy links automatically, so it is YOUR responsibility to get rid of them.
But as mentioned above, you could cause severe damage to your rankings if you don’t know what you’re doing so you have to make sure you use it carefully. We often come across cases where webmasters tell us they “started disavowing something here and there.” This is certainly not the right approach.
There are common mistakes, sometimes even as simple as the wrong file format, which will make Google ignore your Disavow file completely, turning all your work to dust. To prevent that from happening, we prepared a safe step-by-step guide for you.
Read on and learn how to use the Google Disavow Tool to disavow links properly.
Google ranking factors
Your ranking in Google depends on many things, but one of the main ranking factors is certainly your backlink profile. Google tries to rank their search results according to quality and relevance of the websites. This relevancy, in turn, is based on the popularity of the content. The more visitors you have and the more people refer to your content by sharing or linking to it, the higher the chance that Google will assess your website as relevant. And thus, you ranking will move to the top. Google puts it as follows:
"In general, a link from a site is regarded as a vote for the quality of your site."
The more good “votes” (meaning powerful backlinks) your site gets, the better your ranking. But this can also work the other way around: bad links to your website can affect your rankings in a negative way. In 2016, Google’s Search Quality Senior Strategist Andrey Lipattsev confirmed links as being the #1 ranking factor, alongside content:
"I can tell you what they are. It's content and links pointing to your site."
If you have links from bad sites with poor content, Google might associate your website with such sites, and thus classify you as spam too. This is why you need to analyze your backlinks and get rid of unwanted ones, either by having them removed or by disavowing.
When and how should I use the Google Disavow Tool?
Some people claim that using the Google Disavow Tool should only be considered when you file a Reconsideration Request after a Manual Action.
Well, this cannot be true: we have seen thousands of proven examples where disavowing risky links has led to a recovery from an algorithmic Penguin Penalty – where you don’t even have the option to file a Reconsideration Request. Therefore, we are convinced that you can use the Google Disavow Tool in both cases, a Manual Action and an algorithmic Google Penguin Penalty.
Also keep in mind that the rule “better safe than sorry” applies here. If you don’t want to risk receiving a Penguin Penalty in the first place, then you should proactively disavow risky links. This way, you can continue with your online business without having to worry about possible drops in your rankings.
Another question we often hear is whether you can use the Google Disavow Tool straight away or whether you should contact the webmasters first.
Google officially stated that you should use the disavow tool after trying to reach out to the webmasters, specifically asking for the links to be removed.
The only way to get a full understanding of your backlink profile is a thorough analysis of your links. You can compare it to a medical check. Unless you get a careful check-up done by a doctor, you have no idea about your condition. The same goes for your backlink profile.
In your first step, you should gather as many of your backlinks as you can. You can check out some of your backlinks using Google Search Console, but you will surely need to see more than what Google shows you. This is why you need professional off-page SEO tools. Link Detox will aggregate data from 24 different data sources. In addition, it allows you to upload your own backlink lists, if available. By connecting your Link Detox account with your Google Search Console, you will add yet another data source to your report. Read here how you profit from the GSC integration.
The Link Audit is certainly the most difficult part of the disavow process. It is also the most time-consuming task as you will have to review your links carefully and in detail. However, the Link Detox Screener will speed up your manual review considerably. You can browse page after page from within the toolkit, and set important actions on-the-fly. Watch a short video of how the Screener will simplify the Link Audit.
But before you start your manual review, it’s recommended to start off by cross-sorting the backlinks according to a combination of relevant metrics. Grouping your backlinks in clusters will give you a better overview and allow you to work more efficiently.
You could, for instance, first review all backlinks that derive from pages with a high DTOXRISK, low Power*Trust value, and a negative link growth. If all of these metrics apply, chances are it’s not exactly a top-quality link. Of course, you can combine other metrics to filter only those backlinks that you care about most.
2. Create your Disavow file
After you completed the backlink audit and identified the backlinks you wish to disavow, you can start the actual disavow process.
2.1 Required format
First of all, let’s take a look at the format requirements by Google.
- File type must be .txt
We see people, often Mac users, having problems with their submission because they don’t create files in the correct formats. Don’t use .docx, .xlsx, .rtf, or the like. Google will not accept such files. Also some text editors cannot display the Disavow file correctly. We see Notepad sometimes has difficulties opening the file correctly. You could use other text editors like Wordpad or Sublime instead.
- The file must be encoded in UTF-8 or 7-bit ASCII.
- The file size mustn’t exceed 2 MB.
- Make sure you list one URL/ domain per line.
- You can disavow a particular URL or an entire domain. If you have many unwanted backlinks from a particularly spammy sites, for instance, it might be better to disavow the entire domain. Matt Cutts confirms this mistake where “people are trying to take a scalpel and pick out individual bad links in a very granular way,” while “rather than a scalpel you might be thinking of a machete” (~min. 01:00). If you wish to disavow at domain level, add “domain:” in front of the domain name: so to disavow example.com add the line domain:example.com.
- It’s recommended to disavow sub-domains separately.
- Last but not least: you have to make sure you added all the backlinks you wish to disavow. You should neither miss any of the links you identified as risky before nor should you add a full list of your backlinks.
Make sure you comply with the format requirements because. Google will not take your Disavow file into consideration otherwise.
If you want to make your life easier, avoid potential mistakes and focus on more important things than file formats, you should use professional tools to create the Google Disavow file automatically for you.
With just one click, Link Detox will export each and every link you identified as “Disavow” during your Link Audit in the exact format required by Google.
2.2 Example of a Disavow file
This is how a Disavow file would look like. Lines beginning with a # character indicate a comment. We will speak about comments in the Disavow file later.
2.3 Naming Conventions
Google doesn’t require any particular naming conventions, but for your sake, we strongly recommend you give your Disavow file a meaningful name. This is important because you need to keep track of your Disavow files in case you want to edit previous versions. In Link Detox, your Disavow file will be created automatically under a meaningful name, namely: project-dtox-domain-date.csv
3. Upload your Disavow file
As soon as you prepared your Disavow file, you will have to upload it to the Google Disavow Tool.
If you use Link Detox to create your Disavow file, you will be forwarded from within the toolkit to the correct login page by simply clicking on “Google Disavow Links”.
If you don’t use Link Detox, log in to Google Search Console account and got to the Disavow Tool with this link: https://www.google.com/webmasters/tools/disavow-links-main
Please note that at this point you will be overwhelmed with warning messages by Google. However, this shouldn’t irritate you. If you went through a thorough Link Audit before, there is nothing to be scared of.
1. Select the domain you want to upload a disavow file for.
2. Click on “DISAVOW LINKS”.
3. Confirm the warning message that the Disavow Links Tool is a tool that can cause serious damage if used incorrectly (see screenshot). To confirm click again on “Disavow Links”.
4. Once you passed that hurdle too, the Disavow Tool will confirm your upload immediately. You should see a message similar to the screenshot below, confirming the upload and showing you your submission results:
Please note that the result submission will state the exact file name of the submitted file. This is why Link Detox generates meaningful file names automatically for your Disavow files.
After completing this step, you uploaded the Disavow file successfully.
5. Click the “Download” button to download the most recent Google Disavow file again.
6. Select “CSV” as format and download.
7. You will receive a file with a file name DIFFERENT to the one you saw, containing only the domain name and some numbers. We highly recommend you rename that file BACK to what it was called. This will help you maintain your project’s consistency when doing round-trip disavows.
8. Upload that Disavow file to your next Link Detox report (along with other custom link data files).
Your disavowed links will now be shown as such in your future reports.
4. Boost your Disavow file
It is a common misconception that after uploading the Disavow file, Google will immediately start crawling those links for you. The Disavow is only a recommendation to Google to not consider some links on the next crawl. Google’s crawling schedule is based on their own priorities and they will not change anything about it every time they get a new Disavow file.
Also, it takes about 48 hours for all the Google bots to take notice that there is a new Disavow file for your domain. After those 48 hours, it can still take weeks or even months until Google re-crawls your disavowed links and takes the Disavow into account.
Do you really want to wait for that long?
Our experience has shown us that some people do. You can’t imagine how often we hear from SEOs that they uploaded their Disavow file a week ago and nothing has happened. This is only one of many common link audit mistakes. Let’s give you an example here: a fellow SEO has become a victim of a negative SEO attack for his site selling sunglasses. After two months of constant spamming he received about a hundred links from spammy automated scraper sites, so he decided to add these links to his Disavow file – good so far. A week later, his rankings started to fall. He immediately reacted with a conspirative post on his website, claiming that it was the Tool’s fault and the links were not that bad and he should have kept them.
A more logical explanation for that drop in rankings would be:
- The Disavow file was not taken into account yet
- The drop happened due to the spammy links that have been there for two months
- The drop could be the first indicator for an upcoming Penguin Penalty
You should keep in mind that every day there are thousands of spammy links that are generated by automated sites and that people upload in a new Disavow file. Your hundred links you just disavowed don’t bother Google much, as they only represent a tiny fragment of the huge amount of disavowed links they get every day. This is why you need to think of a more sophisticated way of getting your disavowed links crawled.
A good way of doing that is using the Link Detox Boost. It usually takes weeks or even months before Google even sees you uploaded a Disavow file. Every day of a Penalty will cost you money, so we strongly recommend you don’t waste time. Using Boost will make Google really take the Disavow file into account. This will speed up the recovery considerably. We have even seen cases where Boost helped to do so within only 3 days!
Link Detox Boost by @lnkresearchtool works. Love it. That is all.
— Bill Hartzer (@bhartzer) December 22, 2015
You can start Boost directly in Link Detox. Running the tool is simple and will only take you a minute. Read here how to run Boost in only 4 steps.
After running Boost, you will see a table with the last Google Crawl Date of each backlink.
As soon as all links have been crawled, you can go ahead and submit a Reconsideration Request in the case of a Manual Action.
If you do that before, Google will not see that you disavowed the link already, and your request is likely to be declined.
5. Constantly Review Your Disavow file
You must constantly review your Disavow file since the web changes every day and you sure have some very good links in there. A website that looked suspicious or had no links one year ago may now be owned by someone else, or it may have been cleaned up. If you included that domain in your Disavow file, you might now want that link back.
Here are some examples for potential reasons to un-disavow links.
Link Detox (DTOX) offers the possibility to do a Disavow File Audit. Just open a new report, enter your domain, select “Disavow File Audit” and upload your Disavow File. You might find some links in there that have become great! Remove them from the Disavow File and get all that good link juice back.
Read more about how you can audit your Disavow file.
You should also keep in mind that we live in the era of the Real-Time Penguin. Remember that sunglasses-example mentioned before? That happened almost a year before the Penguin 4.0 update, and still, his rankings dropped within a week. Imagine what would have happened with those spammy links if they had been counted in real-time!
That makes Link Risk Management and the need for proactive disavows even more important for your online business. Google is crawling the web on a daily basis. Maybe not all of your links at once, but you still don’t want them to find out about spammy links before you put them in your Disavow file. You should at least update your Disavow file on a weekly basis. If you have a large backlink profile and you’re getting new links on a daily basis, you should probably disavow on a daily basis.
But how to find out?
We offer a great tool called Link Alerts (LA) that notifies you about all your new links on a regular basis. We recommend that you implement those reports in your weekly link audit routine and check all those new links for their risk – and then put them into your Disavow file. If your site witnesses a negative SEO attack, please don’t act like in the example mentioned before and wait for two months until you update your Disavow file. Do it right away and save yourself the troubles of seeing your rankings slowly dropping and spending nights worrying about your online business.
6 Extra Secrets about Disavows
1. Comments in Disavow files: Nobody ever looks at the comments in a Disavow file because it never gets to the hands of a member of the Google spam team. In this Google Hangout, John Mueller confirmed that these files are “processed completely automatically”. Nevertheless, some webmasters like to include comments. You might want to use comments if you are not the only one working on the disavow file, and you want your colleagues to see what you changed. This is why Link Detox supports a comment function within the toolkit. In case you want to include comments to your Disavow file manually, make sure you begin each comment line with a # character.
2. One file only: Google accepts one Disavow file only! Uploading a new Disavow file will overwrite all previously uploaded ones. If you want to add links to your Disavow file, you have to make sure that you don’t overwrite previous versions by mistake. If you create your Disavow lists manually, you should keep track of all your files by giving them meaningful file names!If you forget to include a previously disavowed link to the latest version of your Disavow file, Google will count this link again, which could once again cause a Penalty.Link Detox maintains your Disavow lists automatically for you. It combines all previous versions into one. This way, you can always make sure you work with the latest version.
3. Disavowing is not removing: If you disavow a link, it does not mean the link is gone. It only means you ask Google “not to take certain links into account when assessing your site.”
4. Disavow file is not binding: Also keep in mind that Google considers your Disavow file as a recommendation to not count particular links, not instructions. It does not mean that Google is obliged to ignore these links. If Google’s evaluation differs from yours, they might still count. This is why removing a backlink is surely the safest option. What is gone, is gone.
5. Disavowed links remain visible: If you disavow a backlink, it will still appear in your Google Search Console. So don’t get scared if you still see them. It will only be removed if the link is deleted. Link Detox works the same way. You will be able to see the disavowed links in your report, but they will be marked as such, highlighted in blue and have assigned a DTOXRISK of 0.
6. Disavowing NoFollow links: Google regards disavowed links as kind of NoFollow links. But this, in turn, does not necessarily mean that NoFollows don’t need to be disavowed. To the contrary: one of the two schools on NoFollow links believe that they actually can – under certain circumstances – impact your rankings. Either way, a risky link remains a risky link – whether it’s a NoFollow or not.
Today, probably everyone will at some point be required to submit a disavow file. No matter whether you have bad backlinks from the past or new risky links due to negative SEO attacks: if you cannot remove your risky backlinks, you need to disavow them BEFORE Google finds out.
But we see many mistakes:
Many SEOs panic and start disavowing anything hell-for-leather, just because they are trying to diminish the DTOXRISK as fast as possible. I have even seen cases where all links with medium DTOXRISK got disavowed in bulk like crazy – without a manual review.
Well, if this sounds familiar, let me give you one advice: disavowing the wrong links can and will harm your rankings just as well. A good ranking requires good backlinks. If you go ahead and carelessly disavow all links with medium DTOXRISK, for instance, your backlink profile will not look natural. And this will be the reason for a Penalty.
Reviewing your backlinks individually before actually disavowing them is just indispensable. But luckily, there are tools out there that make it easier. We also don’t recommend to anyone to wait for weeks or months until Google finally takes your Disavow file into account. There are better ways. Use them!
When was YOUR last update of your Disavow file?
Did you ever do an audit of your recent Disavow file?
With the Superhero Plans you can perform link audits, link cleanup, link disavow boost, competitive research, professional SEO and backlink analysis for your own or your competitor's sites.
You can avoid a Google Penguin Penalty! Learn all about the Real-Time Google Penguin Update in this free webinar.
Join our 21 Day Link Strategy Training below
Signup to receive snack sized bits of knowledge about Link Research Strategies, SEO Tactics & features of LinkResearchTools platform and technologies.