Site icon DataForSEO

How to Analyze SERP Competition for Any Keyword with DataForSEO Labs API

Anatolii

Sizing up the competition for one keyword used to mean opening the SERP, scrolling through the top 10, reading the domains by hand, then redoing the whole thing in a private tab to make sure the rankings weren’t personalized. It doesn’t scale. A proper SEO SERP analysis API replaces that loop with a handful of structured calls that return the same intelligence as JSON: which domains rank, how visible they are, what keywords they overlap on, and how contested the SERP really is. This guide walks through four endpoints in our DataForSEO Labs API that, called in sequence, tell you what you need to know about the competition for any keyword.

Why a dedicated SEO SERP analysis API beats scraping the page yourself

A single Google SERP today carries roughly 20 distinct feature types. Those include featured snippets, AI Overviews, People Also Ask sections, local packs, shopping carousels, and more. Google keeps changing how those features work. A parser that handled every element cleanly last quarter is probably missing something this quarter.

That’s the part most teams underestimate when they decide to scrape SERPs themselves. The crawler works on day one. By month six, someone is patching the parser every Tuesday, the proxy bill is climbing, and the captcha-solver invoice is its own line item. Meanwhile, the feature you were supposed to ship, the part customers actually pay for, hasn’t moved.

The DataForSEO Labs API skips the raw-page problem. Instead of parsing HTML, you get pre-aggregated competitive intelligence about the keyword and the domains ranking for it. No parser to maintain, no proxy pool, no captcha pipeline. That aggregation is what makes the workflow below possible in the first place.

The four DataForSEO Labs endpoints that do the heavy lifting

A SERP competition analysis is rarely answered by one call. The question “who is competing for this keyword and how hard is it to beat them?” breaks down into four sub-questions, each handled by a different DataForSEO Labs API endpoint:

Call order matters. Here’s the workflow we recommend for analyzing a single keyword.

A four-step workflow for analyzing competition on a single keyword

Each step uses one of the endpoints above and feeds the next.

1. Score the keyword difficulty. Send the keyword to Bulk Keyword Difficulty before anything else. A score of 12 and a score of 78 should lead to very different research depths. This call accepts up to 1,000 keywords per request, so you can also use it as a triage step across a whole keyword list.
2. Pull the top SERP competitors. Pass the same keyword (or a small seed cluster) to SERP Competitors. You’ll get back a ranked list of domains with the visibility metrics you need to identify who actually owns the SERP rather than who happens to land on it once.
3. Profile each top competitor. Take the highest-visibility domains from step 2 and feed them, one at a time, into Competitors Domain along with your own target. The response tells you how much organic and paid overlap exists, and where the rival is winning that you aren’t.
4. Find the keyword gap. For the most relevant rival, run Domain Intersection between your domain and theirs. The keywords that come back are SERPs where both of you rank. That’s the cleanest definition of contested territory you’ll get.

The example below shows step 2 in action. The cURL snippet calls the SERP Competitors endpoint for one seed keyword and returns the top five ranking domains.

# Replace login and password with credentials from https://app.dataforseo.com/api-access
# base64-encoding the "login:password" pair satisfies HTTP Basic auth in one line
cred=$(printf "login:password" | base64)

curl --location --request POST "https://api.dataforseo.com/v3/dataforseo_labs/google/serp_competitors/live" \
  --header "Authorization: Basic ${cred}" \
  --header "Content-Type: application/json" \
  --data-raw '[
    {
      "keywords": ["seo serp analysis api"],
      "location_code": 2840,
      "language_code": "en",
      "item_types": ["organic"],
      "limit": 5
    }
  ]'

Endpoint used: api.dataforseo.com/v3/dataforseo_labs/google/serp_competitors/live. See the SERP Competitors endpoint documentation here.

What the response gives you and how to use it

The response is structured JSON. The snippet below shows the full envelope with non-essential objects collapsed so the meaningful fields stand out.

{
  "version": "0.1.20240101",
  "status_code": 20000,
  "status_message": "Ok.",
  "time": "0.2104 sec.",
  "cost": 0.0105,
  "tasks_count": 1,
  "tasks_error": 0,
  "tasks": [
    {
      "id": "...",
      "status_code": 20000,
      "status_message": "Ok.",
      "time": "0.1846 sec.",
      "cost": 0.0105,
      "result_count": 1,
      "path": [ ... ],
      "data": { ... },
      "result": [
        {
          "se_type": "google",
          "seed_keywords": ["seo serp analysis api"],
          "location_code": 2840,
          "language_code": "en",
          "total_count": 87,
          "items_count": 5,
          "items": [
            {
              "se_type": "google",
              "domain": "example-competitor.com",
              "avg_position": 4.2,
              "median_position": 3,
              "rating": 612,
              "etv": 1840.55,
              "keywords_count": 1,
              "visibility": 0.0418,
              "relevance": 0.92,
              "keywords_positions": { ... },
              "metrics": { ... }
            },
            { ... },
            { ... },
            { ... },
            { ... }
          ]
        }
      ]
    }
  ]
}

A few fields inside each item in the items array carry most of the analytical weight:

Most teams sort by etv descending, then filter for relevance above 0.5 to drop the generalists. That gets you from a list of ranking domains to a list of competitors worth analyzing.

Common pitfalls to avoid

  1. Forgetting to set the location parameters. The endpoint requires either the location_name or location_code. Leave both off and the request fails. Pass the wrong one and the SERP analysis comes back for the wrong market: US results when the customer cares about Germany. Set the location explicitly every time. There is no default.
  2. Treating keyword difficulty as an absolute score. Keyword difficulty in our responses is logarithmic from 0 to 100. The gap between 60 and 70 is much larger than the gap between 10 and 20. Compare relative scores within a keyword set rather than applying a flat cutoff across unrelated topics.
  3. Ignoring the item_types array. By default, the endpoint considers a wide set of SERP elements. If you only care about organic competition, fetch the "item_types": ["organic"] as the cURL example does. Otherwise, a domain that dominates the local pack or shopping carousel can show up as a “competitor” you can’t actually compete with.
  4. Sampling too few seed keywords. A one-keyword SERP Competitors call surfaces who ranks for that exact phrase. A 10 to 20 keyword seed cluster surfaces who consistently ranks across the topic. For most competitive analysis use cases, the cluster view is the one you want.

Wrap up

One keyword’s SERP shows you who is competing, but only if you can read the snapshot at scale. The four endpoints above give you the parts you need: how hard the keyword is, who ranks for it, what those rivals look like across their full footprint, and where your domain overlaps with theirs. Integrate once. The SERP collection, the parsing, the aggregation underneath is something we maintain so you don’t have to. Sign up and try for free. Every new account gets $1 in credits, which is enough to run all four endpoints on a real keyword and see the whole workflow before committing to any volume.

Exit mobile version