Finding broken links

Broken links make your readers unhappy! They make search engines pretty grumpy too, and can cause your search engine rankings to drop if left unattended.

This is a pretty low priority though - so while we'd recommend it as a housekeeping exercise every 2-4 months, there are better places to spend your day-to-day energy.

We recommend:

  • Deepcrawl or Moz: these are super-powerful, and comes with a bunch of SEO tools. You can take advantage of these reports as part of an SEO package from Performance Foundry, or sign up to your own account.
  • Screaming Frog -- App, free for 500 pages. Our other paid recommendation would be to upgrade this to the pro version.
  • Broken Link Check — Online, limited to 3000 pages. If you run this too often it can put extra stress on the server, which can trigger a security protocol, which will block your IP address from using this going forward. More often than once every two months is overkill, anyway!
  • Validator This online tool looks basic but does the job! By default this tool will only check the homepage, so make sure to click the “Check linked documents recursively” checkbox so that it will check the whole site. A recursion depth of 3 should make sure most pages on your site are checked without overloading the process!
  • Xenu Link Sleuth — Windows only.
  • Integrity — Macintosh only.

I get lots of errors, or the tool is blocked

A note on link checking tools and our security tools: We practise rate limiting -- if a bot hits our server too fast, it'll trigger a captcha notice. This protects you from all sorts of malicious attack, but can cause false positives with the tools above (except Moz). If you see a lot of false positives, or if the tool stops working, it isn't a problem with your site, it's your testing tool acting like a attacker -- and our security systems block against that.

What can you do? We may be able to whitelist the tool. Find out the IP range the tool uses and send it to Support. 

Most tools allow you slow down the rate at which the tool scans. If you have been blocked, try halving the resource use by scanning fewer links per second or increasing the time between crawls.

Why you should avoid the Broken Link Checker plugin

There's a WordPress plugin called "Broken Link Checker" that does a similar job. The only problem is that it hobbles your website speed, expands your database tables, and runs riot with our caching systems. It's also one of the number-one crashed tables we see on larger sites. That means your MySQL database will slow right down or stop responding all together.

WE DO NOT RECOMMEND BROKEN LINK CHECKER ON OUR SERVERS. Please don't run it on your site.

Please avoid "the WP Broken Link Status Checker" plugin too, for the same reason.

What do all the error messages mean?

The most common issue is 404 errors. We have a  write up on 404 errors and how to address them here.

We deal with the other common errors and how to address them here.

---

Looking for a host that knows WordPress backwards and forwards? Talk to us about our Managed WordPress Hosting Solutions so you can get back to running your business. 

Still need help? Contact Us Contact Us