I’m going to preface this by saying that in theory (and the most common way) you can bulk redirect all of your 404 error pages straight to the homepage, and many people do that especially within WordPress by using a plugin.
I will cover why you may not want to do this for SEO purposes later in this guide, and for the purpose of why we’re here I’m going to be diving into various tools that would be worth you checking out.
So, the pesky 404. It’s no use for anyone, right? Wrong. Just because you have a 404 error page, does not mean that you should be redirecting it straight to the homepage without looking at whether it holds any authority, and what that could do for your SEO strategy.
Many site owners take these tactics to the extreme, for example by finding expired domains and redirecting a whole host of external ‘dead’ URLs which hold authority, in theory boosting the authority of the target site. What you’re doing here is much more natural, as the 404’s are going to be internal to your site.
What I’m trying to get across is that you could have an old page that very recently started to serve a 404 status code, and it could’ve had lots of high-quality, relevant backlinks that were really boosting your SEO efforts. If you have a bulk redirect protocol in place, this could be missed and instead of being redirected to its new alternative that would massively benefit from the external link equity it would go straight to the homepage. By redirecting straight to the homepage, you’re potentially missing out on a relevant boost that you can give to singular pages which otherwise may struggle to obtain links organically.
Anyway, overly-extensive preface over. Let’s have a look at how you can discover, analyse and then match your 404 error pages and make the most out of them for SEO purposes.
Redirect process to analyse and retain SEO value
For this process you’re going to need the following tools. Note that this process is extensive, so if you want me to take a look at your site or would like to enquire about my technical SEO services please just drop me a message.
Tools required:
Process:
- Use the SEMRush audit tool to find 404’s
- This is a quick, easy way to find 404’s if you’re already using a tool like SEMRush, or an equivalent. Go to Site Audit > Top Issues (your 404’s will be in here).
- Cross reference these findings with Screaming Frog to ensure the correct number of 404’s are being collected. Once your crawl is completed go to Response Codes > Client Error (4xx) > Export
- You can do either of the above, or both. I do both just to cross-reference across more than one data source/external tool and ensure that I have the analysis of each URL and their authority (if external links are present). So for example we can go to Ahrefs and paste our 404’s into the ‘Batch Analysis’ tool, then export them to see their individual URL rating. If they have solid, singular authority, then these may be the URLs we can to match like-for-like and not just blindly redirect to the homepage.
- Manually visit the URLs – Is the 404 status code being served e.g. is this actually a 404 and not being shown as an error?
- Find soft 404’s from within Search Console and add them to the list. Go to Index > Coverage > Error. Any soft 404’s will be flagged in this section, and you can export them.
- Also export a full site crawl via Screaming Frog. These will be the URLs we want to match against the 404’s, so these need to be live. Go to Internal > HTML > Export. Then, open the Excel file and remove any status codes that are not ‘200’.
- Add your soft 404 URLs to this 404 list, if applicable.
- Remove all data other than ‘Address’, ‘Status Code’ and whichever form of a domain rating your tool may have given to analyse the authority of each URL from your bulk export. It’s also easier if you remove the initial site path too, so you’re left with:
- Name the 404 tab as 404 and the 200 tab as 200s.
- Do the same for the full site export, so you’re left with just the URLs and their status codes (which need to be 200).
- Format the 404 data in a table and call it something like ‘errorcodes’. Do the same for the 200s tab data and call it something like ‘liveurls’. Simply use Format as Table > My table has headers:
- Create a new tab called ‘Fuzzy’.
- We can now use the Fuzzy Lookup tool to exact-match URLs and see our like-for-like opportunities. Go to ‘Fuzzy Lookup’ and ensure the left and right tables are selected, along with ensuring you’re only matching ‘address’ to ‘address’. In the output columns, you then want all outputs as you’re going to need to filter the data. Keep the number of matches as ‘1’, and move the similarity threshold as you see fit (I usually reduce this slightly to 0.3):
- Once Fuzzy has done the matches, you filter both by similarity and also the URL Rating – Can you see any opportunities for URLs that are like-for-like matched and also have a high URL rating? So for example if we filter by ‘URL Rating’, we can see that we have an exact match, from a 404 URL that has retained some authority:
- I then create another spreadsheet with just the 404 and the 200 URL’s, matched based on the Fuzzy results. This can then be used for adding the redirects within WordPress.
- Using the ‘Redirection’ redirect plugin, either manually match the 404 to the recommended redirect URL, or upload the redirects in bulk using:
- Tools > Redirection > Redirects
- ‘Add New’ to add a singular URL for matching
- ‘Import/Export’ to import a file of all the redirects to add in one go. The plugin also provides the desired doc format:
Or, as discussed at the top of this guide and if you’re not dealing with manual matches or simply don’t have fuzzy or don’t have the time, you can use plugins to bulk redirect all 404’s to the homepage.
This isn’t something I recommend, especially as you could be redirecting a 404 URL with authority that could be better used on a like-for-like basis. But, the option is there if you’re just looking to get 404’s resolved quickly. It’s not recommended for SEO purposes, but it is an option for resolving numerous 404 issues quickly.
How to redirect 404 error pages in WordPress without a plugin
The most common way (especially away from WordPress) that site owners or developers deal with forms of redirects is by utilising a .htaccess file.
Your .htaccess is actually hidden, so you’re going to need to use an FTP like Cyberduck or FileZilla in order to access this. Then, if you still can’t see it, you’ll need to request to see hidden files. You then edit this file the same way you’d do a disavow file, using a plain text editor – here’s an example image of what a .htaccess redirect file looks like:
Source: WooCommerce
Essentially, you’re specifying the rewrite URL from the old to the new URL. If it looks complicated, I’d strongly urge you to stick to a plugin approach.
Although, if you do have a WordPress site, using a plugin is much easier in my opinion. It’s ultimately a personal preference. For people with redirection experience with .htaccess, naturally they’ll use that approach. If you’re not familiar, using a plugin will no doubt be easier!
Recommended free plugins for 404 redirects in WordPress
I’ve already shown the plugin I use for 404 error redirection in the initial step-by-step process, but if for whatever reason you don’t want to use this plugin then the following are just as good:
- All 404 Redirect
- 404 redirection within Yoast SEO
Conclusion
Hopefully this guide shows you the (not so) secret world of singular URL power, lurking in the graveyard of your 404’s. The Fuzzy Lookup tool also comes in really handy when doing URL redirects for website migrations, for example if you’re moving from one CMS to another and need to redirect all of your URLs. That’s certainly a guide for another day though!
