Push the boundaries of what DataForSEO can do
Learn about the latest updates

Subscribe To Our Newsletter


Crawl the list of URLs with OnPage API

August,19 2021 14:32:06

On-Page API
Crawl the list of URLs with OnPage API

Our team is glad to announce a new feature in On-Page API!

We’ve added the field that allows you to scan up to 20 specific URLs of a single website.

The specified URLs will be crawled prior to other webpages of a given domain.

Thus, primarily you can scan a specific list of URLs you are interested in, rather than all the pages on the site in a row.  All it takes is to specify the priority_urls field in the following way:

        "target": "dataforseo.com",
        "max_crawl_pages": 5,
        "priority_urls": [

Feel free to check the documentation.

Learn more

Embed DataForSeo widget on your website

Embed code: