How to use new crawl control parameters in Google Organic SERP API

In Google Organic SERP API, we enhanced control over the stop_crawl_on_match feature with three new parameters: target_search_mode, find_targets_in, and ignore_targets_in. These parameters enable you to fine-tune when crawling stops and which SERP elements are checked for target matches, providing greater precision in Google rank tracking.

How do these parameters work?

By default, the stop_crawl_on_match parameter terminates the crawl when at least one of the specified domains or wildcard values is found in SERP. The three new parameters that work alongside stop_crawl_on_match extend crawl control beyond this condition.

Let’s take a closer look at these parameters:

1. The target_search_mode controls whether the crawl stops when any target domain or wildcard is found, or only when all targets are found. This parameter has the following string values:

• any (default value) – the crawl stops when any single target is found;
• all – the crawl continues until all specified targets are found.

2. The find_targets_in allows you to specify an array of SERP element types that should be checked for target matches. Only the element types you select will be evaluated against your targets. Without the find_targets_in, all first-level SERP elements with domain and url fields will be checked for targets.

In the find_targets_in array, you can specify any of the following SERP element types: organic, paid, local_pack, featured_snippet, events, google_flights, images, jobs, knowledge_graph, local_service, map, scholarly_articles, third_party_reviews, twitter.

Example: "find_targets_in": ["organic", "featured_snippet"]

However, the array cannot contain the same element types as the ignore_targets_in parameter.

3. The ignore_targets_in parameter is the opposite of find_targets_in. It specifies an array of SERP element types that should be excluded from target matching. All other element types will be checked for matches.

The following SERP element types are supported: organic, paid, local_pack, featured_snippet, events, google_flights, images, jobs, knowledge_graph, local_service, map, scholarly_articles, third_party_reviews, twitter.

Example: "ignore_targets_in": ["paid", "images"]

Similarly, the array cannot have identical element types as the find_targets_in parameter.

Keep in mind that all three new parameters can be used only when stop_crawl_on_match is specified.

Example use cases of new crawl control parameters

With these new parameters, you can set various custom conditions for stopping the crawl. Besides, you can use parameters individually or combine them to achieve more specific results.

Here are three powerful examples of how you can use them:

1 Track targets in organic results only. Specify "find_targets_in": ["organic"] to track a website’s organic position while ignoring all paid results and extra SERP features:

[
  {
    "keyword": "seo api solutions",
    "depth": 100,
    "stop_crawl_on_match": [
      {
        "match_value": "dataforseo.com",
        "match_type": "domain"
      }
    ],
    "find_targets_in": [
      "organic"
    ]
  }
]

2 Multi-competitor analysis. You can customize the request to find all competitor positions before stopping the crawl and checking only selected SERP elements. To do that, specify “target_search_mode”: “all” and find_targets_in with relevant SERP elements:

[
  {
    "keyword": "email marketing tools",
    "depth": 100,
    "stop_crawl_on_match": [
      {
        "match_value": "mailchimp.com",
        "match_type": "domain"
      },
      {
        "match_value": "constantcontact.com",
        "match_type": "domain"
      },
      {
        "match_value": "sendinblue.com",
        "match_type": "domain"
      }
    ],
    "target_search_mode": "all",
    "find_targets_in": [
      "organic",
      "featured_snippet"
    ]
  }
]

3 Exclude irrelevant elements. You can track positions across most SERP features, but exclude a few specific types. Specify ignore_targets_in array with irrelevant elements to ignore them in the crawl:

[
  {
    "keyword": "vpn for pc",
    "depth": 100,
    "stop_crawl_on_match": [
      {
        "match_value": "zoogvpn.com",
        "match_type": "with_subdomains"
      }
    ],
    "ignore_targets_in": [
      "paid",
      "third_party_reviews"
    ]
  }
]

As you can see, with new parameters, you can try different crawl control options to optimize tracking and make it more cost-efficient. If you have any additional questions, don’t hesitate to contact our 24/7 customer support.