Improving our broken link tests

One of the most unique features Oh Dear offers is the broken links and mixed content crawler. We will crawl your site for all links, reporting any broken pages to your defined notification channels.

Recently, we encountered degraded performance with our crawler service due to a breaking change in an underlying library called Guzzle. This caused HTTP 505 responses on the first page of each site, blocking further crawling and resulting in false positive reports. Although our uptime service showed everything was okay, some customers were incorrectly informed about unreachable pages in their broken links report.

Making sure this problem does not happen again

We have over 1,200 tests that run on GitHub Actions with every pull request. To make sure all these tests are executed fast, we mocked all requests that happen inside our broken links and mixed content tests. Although this results in fast tests and consistent results (no network or external conditions), it missed this particular issue because that particular issue only occurred when making real network requests.

We added tests that perform real HTTP requests using our crawler service to validate end-to-end functionality under real-world conditions. This way, we'll make sure we don't hit this particular bug again.

Additonally, we posted a status update and responded to support tickets to inform users of the issue and began our investigation.