How to find broken links with On-Page API?

A broken link is a hyperlink on one page that refers to a different page that’s broken. In other words, a broken link is only a reference to a broken page, but not the page that responds with a 4xx or 5xx code.

Broken links make it difficult for Googlebot and other search engine bots to crawl your website. What’s more important, broken links may harm not only SEO but also user experience. When a website visitor clicks on a hyperlink and gets to a non-available resource, they may abandon the site at all.

Either way, you’ll definitely want to discover broken links on your site and fix them.

To find broken links with DataForSEO On-Page API, you need to start with registering and getting your api key that you should use for authentication.

Learn more about Authentication in our docs >>

Once you’ve completed that, you can get a list of broken links in three simple steps.

1 Set a task using a POST request: https://api.dataforseo.com/v3/on_page/task_post.

For example:

[
  {
    "target": "dataforseo.com",
    "max_crawl_pages": 10,
    "tag": "some_string_123",
    "pingback_url": "https://your-server.com/pingscript?id=$id&tag=$tag"
  }
]

2 Use the task’s ID from the API response to make a GET request to the Summary endpoint: https://api.dataforseo.com/v3/on_page/summary/$id

In the task results, find the broken_links field. If it shows a value greater than 0, our crawler found the indicated number of broken links on the target website.

3 To retrieve the list of broken links, use the task’s ID for a POST request to the Links endpoint: https://api.dataforseo.com/v3/on_page/links. Note: you should use a filter as in the example.

[
  {
    "id": "07281559-0695-0216-0000-c269be8b7592",
    "filters": [
      "is_broken",
      "=",
      true
    ]
  }
]

Note that in the response of the Links endpoint, the source page that’s sending the link will be indicated in the link_from field, while the broken link will be provided in the link_to field. The page_from and page_to fields will indicate the relative URLs of the respective source page and broken page.

Alternatively, after obtaining the Summary, you might want to first review all check-ups for all crawled pages, and get a list of broken links after that.

If that is the case, after step 2, set a task to the Pages endpoint:

https://api.dataforseo.com/v3/on_page/pages

For all returned crawled pages, you will get all metrics related to page performance including the broken_links check. If it is true for a specific page, it means our crawler found broken links on that page. You can then use the page URL to retrieve the list of broken links for it.

Simply indicate the necessary relative URL in the "page_from" parameter in your POST request to the Links endpoint. For example:

[
  {
    "id": "07281559-0695-0216-0000-c269be8b7592",
    "page_from": "/apis/google-trends-api",
    "filters": [
      "is_broken",
      "=",
      true
    ]
  }
]

Recommended read:

How to find broken images with On-Page API?
How to find broken (4xx/5xx) pages?
What are broken backlinks/broken pages?