SERP API can now automatically stop crawling once the domain or page you specify is found in search results, helping you save time, resources, and API funds.
What’s new
We’ve added a new feature to the DataForSEO SERP API – stop_crawl_on_match array – where you can specify up to 10 target domains and/or URLs. The API will automatically stop crawling as soon as the first match is found for one of your specified targets, reducing unnecessary page fetches and API usage. If the target is not found, the crawl will continue as usual up to the indicated depth.
This parameter is available in Google Search Lite SERP API and in the Organic SERP API type of the following search engines: Google, Bing, Yahoo, Baidu, Seznam, Naver.
How it works
Simply add the stop_crawl_on_match array to your API request body as in the example below.
"stop_crawl_on_match": [
{
"match_value": "dataforseo.com",
"match_type": "with_subdomains"
}
]
The match_type parameter supports three options:
domain– match the specified domain exactly (without subdomains);with_subdomains– match the domain and all its subdomains;wildcard– match by pattern.
For domain and with_subdomains options, the match_value must be provided without https:// or http:// (e.g., dataforseo.com).
For the wildcard option, you can define an absolute URL like https://example.com/blog?tag=1$ ($ means “ends with”) or a relative URL, e.g. /example.com/blog/*.
Note that if more than 10 objects are set within the stop_crawl_on_match array, the API will return an error.
Read more about using this new parameter on our Help Center >>
How it helps
This feature is especially useful after Google removed the num=100 parameter, and limited the number of organic results to 10 per page. By stopping the crawl early, you can maintain accurate rank tracking while minimizing extra page requests and costs.
Combine the stop_crawl_on_match with existing parameters like depth, max_crawl_pages, and "search_param": "&start=N" for maximum control and efficiency.
Get started
Ready for a faster, leaner, and more goal-centered rank tracking? Check out the updated SERP API documentation for details and examples.