So, you’re trying to develop a quality solution and empower SEO experts with comprehensive and reliable data. Given that Google definitely has all the data on keyword searches, it would have been logical to go knocking on Google’s door to get the most accurate data for your tool. Yet, Google doesn’t like revealing many insights for SEO research, such as precise search volume – a vital metric for evaluating keyword popularity. This is the point, many keyword research tools stumble upon.
You can stay cool though. In this article, we’ll walk you through the pros and cons of the three common approaches to getting search volume for an SEO tool:
1 Using Google Keyword Planner (GKP) Data
2 Using GKP data + Clickstream
3 Using GKP data + Impressions
Approach 1: Google Keyword Planner Data
You must have heard that after the 2016 Keyword Planner update, the search volumes are displayed as ranges for accounts with no active campaigns, making the data nearly meaningless.
Even though it’s possible to obtain more granular data by adding the necessary keywords to your plan, Google’s search volume still won’t be flawless. Shortly before the “ranges”, the SEO community was hit with “combined search volumes” for groups of similar keywords:
To get distinct search volumes for close variants, you should additionally select the exact match type in the monthly forecast settings.
Overall, the whole process of adapting GKP data to keyword research purposes has grown too complex. That’s why many SEO experts would rather go with paid professional tools which provide already unscrambled Keyword Planner data and allow analyzing substantial keyword volumes much faster and easier.
Many SEO tools are taking the most straightforward approach to getting search volume and are building their keyword research functionality upon GKP data. For example, Rank Ranger displays search volume for keywords as “Google’s estimate of the number of times a keyword is queried per month”, and the data in their reports “is based on Google’s Keyword Planner tool for all locations and all language settings.”
Unfortunately, Rank Ranger does not provide local search volume for some reason. However, it is possible. We can provide you with Keyword Planner data specific to particular countries and languages in DataForSEO Google Ads API as well as historical search volume data in DataForSEO Labs API.
At the same time, we do not recommend using search volume from Keyword Planner alone. Here’s why.
On top of “ranges” and “combined search volumes”, there’s also a “buckets” issue in GKP. The thing is, around 80 logarithmically distributed search volume values are repeated across all keywords. We’ve found this out based on the analysis of our extensive database of over 3.5 billion keywords. Other studies report roughly the same findings.
Accordingly, keyword research tools running on GKP aren’t reporting the most accurate stats, just like Google. In one of its blog posts, Rank Ranger says “… we currently have gaps in our data for certain niches.” They also share: “…we are currently actively looking for a reliable clickstream source to either replace Google Keyword Planner or to at least fill in the data gaps.”
As you can see, relying solely on Keyword Planner data leaves neither SEO software providers nor its users completely satisfied.
On the other hand, ditching Keyword Planner at all does not seem a reasonable solution to the problem. Even if GKP doesn’t indicate precise values, it does show some popularity estimation, and quality search volume data for Google search engine requires giving some credit to the original data source. At the same time, if you want to provide your users with higher granularity, it’s best to complement Keyword Planner values with data from other sources.
Approach 2: GKP Data + Clickstream
One of the common approaches to distill search volume from Google Keyword Planner is combining its data with Clickstream, as Rank Ranger is looking to do.
Clickstream is anonymized data from real-world searchers collected through a vast variety of plugins, extensions, browsers, and other programs. Data aggregating companies purchase Clickstream from multiple sources, refine it, and resell to other parties, such as Ahrefs.
Different parts of Ahrefs functionality make use of Clickstream. Keywords Explorer, in particular, is employing it as an additional data source for decluttering Keyword Planner search volume. As they explain:
“…clickstream data is super useful when you cross-reference it with data from Google Keyword Planner. Which is exactly what we do here at Ahrefs.”
The screenshot above illustrates fully well how helpful Clickstream can be. On the other hand, it can be quite intricate to deal with, and even harmful.
The same Ahrefs article says: “…raw Clickstream can’t be considered as a reliable source of data, even if you find all re-sellers and buy all clickstream data that is available.” Point is, Clickstream represents only a small fraction of data on keyword searches because not many people are using data-collecting applications other than Google, Windows, or iOS. Accordingly, matching Keyword Planner values with those from Clickstream data is a highly elaborate process, which usually requires machine learning and sophisticated algorithms.
Another issue with Clickstream is the reliability of data providers. You must have heard about the Jumpshot shutdown and Hitwise closure following. So, if you go with Clickstream, you wouldn’t want to make your software dependable on one provider and see it cease operation. Accordingly, you’d need multiple datasources, which adds up to the next downside.
Unfortunately, acquiring Clickstream data is very expensive. Integrating even one data source will be costly, while you need several providers to make sure your supply isn’t suddenly put on a halt.
This brings us to another approach, which is tailored to lower budget spending, yet allows refining Keyword Planner data.
Approach 3: GKP Data + Impressions
While GKP shows a quite blurry search volume, especially for high-volume broad terms, there’s another metric – impressions from Ads Performance Forecasts – that may bring more clarity.
Google’s official definition of impressions is “how often your ad is shown,” and it’s counted “each time your ad is shown on a search result page or other site on the Google Network.”
Sure enough, impressions cannot be a substitute for organic search volume, but they can help un-grouping close variants and providing more realistic numbers to your software users.
SEO PowerSuite is one of the tools that have already successfully implemented this approach. They “…take search volume buckets provided by Google Ads and use a combination of Forecast settings to point to an exact number within this range.”
Combining search volumes with impressions data from Ads Forecasts isn’t a piece of cake. The accuracy of your estimation will depend on the specified bid, search partner policies, and position of the advertisement.
Ensuring maximum possible precision will take some machine learning and refinement algorithms, close to how it works with Clickstream data. What’s also similar, impressions can as well point to only a portion of real-world searches, because ads are run for a handful of keywords. Still, impressions will do a great job at indicating more exact values for head keywords.
All in all, correlating impressions with search volumes from GKP is a much cheaper alternative to Clickstream. Here at DataForSEO, we can supply you with impressions data through Ads Traffic By Keywords API.
Conclusion
In the end, there is no correct answer to the question “what’s the right way to calculate search volume.” Yet, we can safely say that Google’s search volume alone doesn’t make keyword research any easier.
Below you can see a quick comparison table with data from the tools we mentioned and the approaches they take to getting search volume values:
| Keyword |
Rank Ranger (GKP data) |
Ahrefs (GKP data + Clickstream) |
SEO PowerSuite (GKP data + Impressions) |
| seo | 823,000 | 301,000 | 114,010 |
| search engine optimization | 823,000 | 38,000 | 57,450 |
| search engine optimisation | 823,000 | 2,100 | 10,820 |
Note: search volumes were checked for the US location, English language as of May 11, 2021.
Worth pointing out, you shouldn’t try comparing data between the tools, as each applies its own algorithm and data sources. Instead, look into the data for given keywords within one tool and consider how accurately they reflect the most popular term.
As you can see, the most accurate options are combining search volume from Keyword Planner with Clickstream data or impressions data. The question is how much you’re willing to pay for higher accuracy. In the case with Clickstream, you should expect whopping expenses, while getting impressions to refine search volume will cost you considerably less.
For example, with DataForSEO Ads Traffic By Keywords API., you can get impressions data for 1M keywords paying only $75 or $50 (depending on how fast you need the results). You can also check 14K keywords for free upon registration for trial, and start impressing your users with more granular data right after a quick integration.
IMPRESS ME