AI monitoring in Oh Dear allows you to use artificial intelligence to verify virtually anything about your websites and services that traditional monitoring might miss. Instead of pre-defined checks, you provide natural language prompts describing what you want to verify.
⚠️ This check is currently in closed beta. If you'd like to try it out, please apply to get access to the AI monitoring beta.
How AI monitoring works #
Unlike traditional website monitoring that checks for specific metrics like uptime or performance, AI monitoring works by describing your check in plain English, eg. "Check that that our homepage displays 5 products, links to blog posts and a call to action. If it does not, send me a notification."
You can ask the AI to fetch webcontent and DNS records. We make sure that the AI has access to the internet so it can verify your requirements.
Getting started #
To use AI monitoring, you need to create a new monitor in Oh Dear and select "AI" as the monitor type. You then provide a prompt that describes what you want to verify. You can choose how often the check runs (from every minute to once every two months).
For the first 10 AI calls, you can make use of Oh Dear's internal AI account for free. After that, you'll need to set up your own API key from a supported AI provider in the AI configuration screen.
How to construct a good prompt #
When writing prompts, be as specific as possible about what you want to verify. The more detail you provide, the better the AI can understand your requirements and provide accurate results. Your prompt should have these elements:
- The URL to check
- An exact description of what to look for
- What to do if the conditions are met or not met (e.g., send a notification)"
- A clear description of what should be included in the results
Here is the example of a good prompt:
Check that our homepage https://example.com displays exactly 5 products, each with a title, image, and price. Ensure there is a call-to-action button labeled "Shop Now" that links to the /shop page.
If all these conditions are met, the check is ok, and you don't need to send me a notification. If any condition is not met, the check fails, and you need to send me a notification explaining what is wrong.
Also, in the check result include the titles of the products displayed on the homepage.
In general, the more specific you are, the better the results will be.
The capabilities and limits of AI monitoring #
Our AI monitor can fetch web pages and DNS records. It is als aware of the current date and time, which is useful for time-sensitive checks.
The AI can only fetch up to 5 web pages has maximum execution time of 60 seconds. Therefore, avoid asking the AI to crawl an entire website or perform complex tasks that require extensive processing time.
You can also ask the AI to send notifications. When the AI determines that a notification should send, we'll send it via the team notification destination and the notification destinations configured for that monitor.
When executing a web request (e.g., fetching a web page), the AI can use different HTTP verbs (GET, POST, etc.) and add custom headers to the request.
Currently, the AI does not have access to a web session, and it cannot type. So you can not ask it to log in or perform multi-step interactions that require maintaining state. We are working on adding theseNo capabilities in the future.
More prompt examples #
Here's some inspiration for other prompts you can use. The AI can check live websites in real-time. It is also able to use our notification system.
Make sure the homepage of mysite.com lists at least five products, includes a login link, and has a registration page link. Send me a notification if any of these elements are missing.
The AI has access to DNS records, so you can also use it to verify your DNS configuration:
Verify that all MX records for spatie.be point to Google. If they don't, send me a notification and show all current MX records in the result.
We support POST requests with custom payloads and headers:
Send a POST request to mysite.com/api/my-endpoint with this payload:
first_name: "John"
last_name: "Doe"
Include this HTTP header:
my-custom-header: my-custom-header-value
The response code should be 200. Send me a notification if the response code is anything other than 200.
Of course, you can also use it to verify your robots.txt or sitemap.xml files:
Verify my robots.txt on https://mysite.tld allows Google but blocks ChatGPT's AI crawler
The AI monitoring system has access to several tools to help verify your requirements:
- Web page fetching: Can retrieve and analyze any web page content. You can specify which HTTP verb to use (GET, POST, etc.). You can also instruct the AI to add extra headers to the request.
- Date/time awareness: The AI knows the current date and time for time-sensitive checks
- DNS record lookup: Can verify DNS configurations and changes
Limitations and considerations #
- The AI can only fetch 5 web pages per check execution. Do not ask it to crawl an entire website, as this you check will be halted.
- Each check execution has a maximum time limit of 60 seconds
- AI monitoring is designed for functional and content verification, not performance monitoring, for which our traditional checks are better suited
- Prompts should be specific enough for consistent results across multiple runs
Check results #
AI monitoring provides two possible results:
- OK: Everything is working as expected according to your prompt
- Failed: Significant problems found that require immediate action
Notification behavior #
AI monitoring notifications work just like other Oh Dear checks:
- You'll receive notifications when checks change from OK to Warning/Failed
- Recovery notifications are sent when issues are resolved
- Notification channels can be configured per check (email, Slack, webhooks, etc.)
- Repeated failures follow the same escalation patterns as other checks