Working with sites in our API

We'll assume you've already got the API authentication settled and you have our API key with you (this is the Authorization header in all the examples used below).

Get all sites in your account #

The /api/sites endpoint lists all your sites in your account.

$ OHDEAR_TOKEN="your API token"
$ curl https://ohdear.app/api/sites \
    -H "Authorization: Bearer $OHDEAR_TOKEN" \
    -H 'Accept: application/json' \
    -H 'Content-Type: application/json'

As a result, you get the details for every site you've added.

{
  "data": [
    {
      "id": 1,
      "url": "http://yoursite.tld",
      "sort_url": "yoursite.tld",
      "label": "your-site",
      "team_id": 1,
      "latest_run_date": "2019-09-16 07:29:02",
      "summarized_check_result": "succeeded",
      "created_at": "20171106 07:40:49",
      "updated_at": "20171106 07:40:49",
      "checks": [
        {
          "id": 100,
          "type": "uptime",
          "label": "Uptime",
          "enabled": true,
          "latest_run_ended_at": "2019-09-16 07:29:02",
          "latest_run_result": "succeeded"
        },
        {
          "id": 101,
          "type": "broken_links",
          "label": "Broken links",
          "enabled": true,
          "latest_run_ended_at": "2019-09-16 07:29:05",
          "latest_run_result": "succeeded"
        },
      ]
    },
    {
      "id": 2,
      "url": "https://yourothersite.tld",
      "sort_url": "yourothersite.tld",
      "label": "my-site",
      "team_id": 1,
      "latest_run_date": "2019-09-16 07:29:02",
      "summarized_check_result": "failed",
      "created_at": "20171108 07:40:16",
      "updated_at": "20171108 07:40:16",
      "checks": [
        {
          "id": 1,
          "type": "uptime",
          "label": "Uptime",
          "enabled": true,
          "latest_run_ended_at": "2019-09-16 07:29:02",
          "latest_run_result": "succeeded"
        },
        {
          "id": 2,
          "type": "broken_links",
          "label": "Broken links",
          "enabled": true,
          "latest_run_ended_at": "2019-09-16 07:29:05",
          "latest_run_result": "failed"
        },
        {
          "id": 3,
          "type": "mixed_content",
          "label": "Mixed content",
          "enabled": true,
          "latest_run_ended_at": "2019-09-16 07:29:05",
          "latest_run_result": "succeeded"
        },
        {
          "id": 4,
          "type": "certificate_health",
          "label": "Certificate health",
          "enabled": true,
          "latest_run_ended_at": "2019-09-16 07:29:02",
          "latest_run_result": "failed"
        },
        {
          "id": 5,
          "type": "certificate_transparency",
          "label": "Certificate transparency",
          "enabled": true,
          "latest_run_ended_at": null,
          "latest_run_result": null
        }
      ]
    }
  ]
}

Let's drill this down, because the 2 sites that are listed each have different checks.

The first one is an http:// site, without TLS encryption. Since we can't monitor certificates or mixed content, we only check the uptime of the site and the broken links.

That's why there are only 2 checks item for that site.

The second site is an https:// site, and it gets all the available monitoring. More on each check in our dedicated checks page.

Get a specific site via the API #

If you don't want to retrieve all sites, you can get a specific site if you know its ID. The example below will get the details of site ID 99.

$ OHDEAR_TOKEN="your API token"
$ curl https://ohdear.app/api/sites/99 \
    -H "Authorization: Bearer $OHDEAR_TOKEN" \
    -H 'Accept: application/json' \
    -H 'Content-Type: application/json'

The resulting pageload of both calls above is a single site.

{
  "id": 99,
  "url": "http://yoursite.tld",
  "sort_url": "yoursite.tld",
  ...
  "checks": [
    {
      ...
    },
    ...
  ]
}

Notice how this payload does not contain the data block (the API call to retrieve all sites does).

Return properties #

The response gives you a lot of details per site. Here's the specifics;

  • id: an internal identifier to Oh Dear
  • url: the URL you used to submit this site to Oh Dear, including the protocol.
  • sort_url: the url without the protocol and without the www. prefix.
  • friendly_name: the friendly name of a site
  • label: contains the friendly_name if one is specified, otherwise this contains the sort_url.
  • team_id: every site belongs to a Team, this is its ID. A team can consist of multiple members (including yourself). Since you can belong to multiple teams, the corresponding team ID will be shown for every site. To know which team this is, have a look at the user info API call.
  • latest_run_date: the date & time of the last time one of the checks in this site has run.
  • summarized_check_result: this gives you an overview of the health of your site at a glance. Possible values are succeeded, warning or failed. To get the details of the status, you'll have to check the output of each check.
  • uses_https: whether it has an https:// prefix.
  • checks: an array of all checks that have been added to this site. More details on the checks page.
  • broken_links_check_include_external_links: boolean, whether the crawler will check for external links or not.
  • broken_links_whitelisted_urls: array of whitelisted URLs that should not be reported if they contain broken links.
  • created_at and update_at: timestamp values for when it was created and when the site's properties were last updated.

More properties will be added over time.

Add a site through the API #

To add a site, create a POST call to the /api/sites/ endpoint. Here's an example to add the site https://mybrandnewsite.tld to Oh Dear.

$ OHDEAR_TOKEN="your API token"
$ curl -X POST https://ohdear.app/api/sites \
    -H "Authorization: Bearer $OHDEAR_TOKEN" \
    -H 'Accept: application/json' \
    -H 'Content-Type: application/json' \
    -d '{"url":"https://mybrandnewsite.tld", "team_id":"1"}'

Remember that we love JSON? Your request payload should also be valid JSON. The actual payload, when formatted, looks like this.

{
  "url":"https://mybrandnewsite.tld",
  "team_id":"1"
}

Mandatory fields are the url we should monitor and the team_id where this site should be added to. To find out the teams you belong to, see the user info API call.

If the API call succeeded, you'll be given output like this, showcasing every check that was added.

{
  "id": 173,
  "url": "https://mybrandnewsite.tld",
  "sort_url": "mybrandnewsite.tld",
  "team_id": 1,
  ...
  "checks": [
    {
      "id": 560,
      "type": "certificate_health",
      "enabled": true
    },
    {
      "id": 561,
      "type": "mixed_content",
      "enabled": true
    },
    ...
  ]
}

When creating a site you can control which checks get enabled. You can do so by passing an array with one of these check types:

  • uptime
  • broken_links
  • mixed_content
  • certificate_health
  • certificate_transparency.

Here's an example payload where a site is created with only the uptime and mixed_content checks enabled.

{
  "url":"https://mybrandnewsite.tld",
  "team_id":"1",
  "checks": ["uptime", "mixed_content"]
}

For more details on each check, have a look at the checks API endpoint.

Error handling #

If an error occured, you'll be given a non-HTTP/200 response code. The resulting payload might look like this.

{
  "message": "The given data was invalid.",
  "errors": {
    "url": [
      "The url field is required."
    ],
    "team_id": [
      "The team id field is required."
    ]
  }
}

If you've got both the url and the team_id in your payload, but the data is invalid, you'll get a detailed error message.

{
  "message": "The given data was invalid.",
  "errors": {
    "url": [
      "You should enter a url that starts with either \"http://\" or \"https://\"."
    ],
    "team_id": [
      "You do not belong to that team."
    ]
  }
}

Deleting a site #

Use the DELETE method to delete a site through the API. In this example, we'll delete the site with ID 1.

$ OHDEAR_TOKEN="your API token"
$ curl -X DELETE https://ohdear.app/api/sites/1 \
    -H "Authorization: Bearer $OHDEAR_TOKEN" \
    -H 'Accept: application/json' \
    -H 'Content-Type: application/json'

Notice how we're calling the DELETE method on the /api/sites/1 endpoint. The 1 in the URL determines which site to delete.

You can add a single URL to our crawler whitelist with an easy API method. This is primarily used to quickly add just a single URL, for instance in an overview-screen where you allow users to whitelist URLs.

If you want to bulk-add several whitelist URLs, or remove whitelisted URLs, please see the more extended update-broken-links-settings API call.

$ OHDEAR_TOKEN="your API token"
$ curl -X POST https://ohdear.app/api/sites/1/add-to-broken-links-whitelist \
    -H "Authorization: Bearer $OHDEAR_TOKEN" \
    -H 'Accept: application/json' \
    -H 'Content-Type: application/json' \
    -d '{
          "whitelistUrl": "https://externalsite.tld/page1"
        }'

Modifying site settings #

You can control whether or not our crawler should check for external links or stick to internal pages only.

Additionally, you can update the site settings and add whitelist URLs that are known to be broken (or out of your control), so we no longer report if we find broken pages listed on them.

Note: this lists the source URLs, not the destinations.

$ OHDEAR_TOKEN="your API token"
$ curl -X PUT https://ohdear.app/api/sites/1/update-broken-links-settings \
    -H "Authorization: Bearer $OHDEAR_TOKEN" \
    -H 'Accept: application/json' \
    -H 'Content-Type: application/json' \
    -d '{
          "broken_links_check_include_external_links": true,
          "broken_links_whitelisted_urls_string":"
https://externalsite.tld/page1
https://externalsite.tld/page2"
        }'

The inputs are as follows:

  • broken_links_check_include_external_links: boolean value, either true or false
  • broken_links_whitelisted_urls_string: string value, new-line separated list of fully qualified URLs that we should whitelist and not report on

Both can be used independently, if you just want to update the broken_links_check_include_external_links setting, there is no need to also submit the list of whitelisted URLs.

Was this page helpful to you? Feel free to reach out via support@ohdear.app or on Twitter via @OhDearApp if you have any other questions. We'd love to help!