What is the Oh Dear crawler doing in my logs?
If you landed here, your access logs probably look something like this:
10.0.1.1 - - [28/Nov/2025:08:17:48 +0000] "GET / HTTP/1.1" 200 845 "-" "OhDear.app (+https://ohdear.app/docs/features/broken-links-detection)" 10.0.1.1 - - [28/Nov/2025:08:17:48 +0000] "GET /login HTTP/1.1" 200 9155 "-" "OhDear.app (+https://ohdear.app/docs/features/broken-links-detection)" 10.0.1.1 - - [28/Nov/2025:08:17:49 +0000] "GET /pricing HTTP/1.1" 200 11540 "-" "OhDear.app (+https://ohdear.app/docs/features/broken-links-detection)"
That's us. We show up in logs for one of two reasons:
- Your site is monitored by Oh Dear, so we crawl it for broken links, mixed content, or sitemap validation.
- One of our customers has a monitor with outgoing links that point to your site, and we verify those still work.
We throttle ourselves and respect your robots.txt by default, so you shouldn't notice any load issues. If you ever do, let us know and we'll look into it right away.
What is the Oh Dear User-Agent?
Our crawler identifies itself clearly, with a link to the docs that explain what it's doing. The exact string varies slightly by check type:
OhDear.app (+https://ohdear.app/docs/features/broken-links-detection)
OhDear.app (+https://ohdear.app/docs/features/mixed-content-reporting)
OhDear.app (+https://ohdear.app/docs/features/sitemap-monitoring)
OhDear.app (+https://ohdear.app/docs/features/application-health-monitoring)
All of them start with OhDear.app, which is handy if you want to match on a prefix in analytics or firewall rules.
Want to allow us through a WAF like Cloudflare? Check our Cloudflare whitelisting guide.
Curious to put Oh Dear on your own site? Start a free trial, no credit card needed.
