Push the boundaries of what DataForSEO can do
Learn about the latest updates

Subscribe To Our Newsletter

Back

Set Disallowed Subdomains in On-Page API!

February,24 2023 19:14:25

Improvement
On-Page API
Set Disallowed Subdomains in On-Page API!

We have an update to share! We’ve added an option that enables you to better control the crawling process with On-Page API.

Now you can specify subdomains disallowed for crawling by simply using the disallowed_subdomains parameter. For example:

[
  {
    "target": "site.com",
    "max_crawl_pages": 10,
    “disallowed_subdomains”: ["status.site.com", "docs.site.com"]

  }
]

This update is particularly useful for websites with multiple subdomains. It allows you to focus on crawling and analyzing the main domain with its most important subdomains while ignoring less important ones.

Note: to use disallowed_subdomains, the allow_subdomains parameter should be set to true.

Don’t forget to check our docs to handle the new parameter correctly.

Embed DataForSeo widget on your website


Embed code:
Preview: