Every developer has their own dreadful stories about dealing with other people’s code. Misunderstanding of functions output, managing huge data flows between various systems, unstructured data challenges – building sufficient interoperability while struggling to get significant volumes of data from point A to point B is such a complex problem sometimes.
At this point, a consistent application programming interface can make things a lot easier. Simply put, APIs build a convenient “bridge” between non-compatible applications, which may have different programming architecture. It doesn’t matter how different applications are – APIs will form a connection between parts of the software that need to exchange data with each other.
Different solutions are designed to serve different software apps. Nevertheless, APIs are commonly understood to be connected with web apps (they are often called REST APIs or RESTful APIs). Data requests to such APIs are made via HTTP(S). At the same time, structured data normally returns either in XML or JSON file formats.
In the framework of SEO tools development, APIs are used to integrate search results, keywords data, information about on-page errors, and so on and so forth. In simple terms, you get the data you need straight to the application. Moreover, that data is typically structured, readable and can be accessed only by authorized users.
However, this is not a manual for those of you who plan to build an SEO tool. So, you need to have some basic understanding of data structures and programming. You can check out the interview with the CTO of RankActive to learn more about the SEO software development.
How do APIs work?
Even though APIs are widely used in software development, their primary working principles are still aspects to be clarified. First of all, one should understand that a typical REST API and an ordinary website are not that different. Both use the same HTTP(S) protocol to communicate data and fetch files. On the other hand, though, the main difference between them lies in the presentation of data (REST APIs usually use JSON and XML, while web pages have an HTML source code).
There are hundreds if not thousands of different API solutions that one can use for different purposes. Need a repository of engaging content? YouTube API will help you to search for videos, retrieve standard feeds, see related content and even upload videos.
Interested in weather forecasts? AccuWeather API provides extended daily forecasts for the desired location, satellite images, info on tropical cyclones and more.
Although some APIs are free and don’t use any form of authentication to make calls, most of SEO-software APIs require this information before they can be used.
Authentication is normally done with a so-called “token,” which can be either retrieved from a server (using a username+password combination) or provided after the registration is completed. A lot of API providers also use OAuth protocol to allow the connection via Internet service functions, such as those provided by Facebook or Google.
You may try to send a simple API request by typing in the following URL in your browser: http://freegeoip.net/json/18.104.22.168
This request will return the following response:
As you see, the response is provided in plain JSON. You can try out sending a similar request with your IP address.
That kind of request method is called GET, which is by far the simplest way to retrieve information via the REST API. Apart from GET, there are other methods to request data from a web server: POST, PUT and DELETE.
- POST method is used to store some information in a database. For example, you make POST requests to a backend script on a server whenever you log into a website.
- PUT request is used when you want to update some information in a database. However, this request normally goes typically hand-in-hand with the POST because you need to store a new piece of info in order to rewrite an old one.
- DELETE has an obvious purpose – it removes an object from a database.
There are other HTTP(S) methods defined under HTTP(S) specifications, but these four are used across all REST APIs you would ever encounter.
Now let’s take a look at how APIs function. Note that in this article we are talking only about REST APIs.
- Data gets stored in the database of an API provider.
- You set a task to an API endpoint using your credentials.
- API reaches out to a database and extracts requested data.
- You request data from an API through an endpoint.
As you see, it’s not rocket science. Or at least, it is so from a user perspective.
APIs for SEO software
Although there are lots of APIs one can use to develop SEO tools, almost all of them are based on a list of common principles that we’ve already described above: you authorize within a system, make a call, and get your results.
However, only a handful of APIs is applicable to the needs of SEO software development. Let’s go through the most widespread solutions to see what APIs can be used to devise SEO tools.
Social Media and SEO has been a trendy topic for a very long time now. Everybody’s talking about the importance of Twitter, Facebook, LinkedIn and other networks in the SEO framework.
You may use Twitter API to get search results, create and manage ads campaigns, check out engagement metrics, send direct messages, etc. Nevertheless, only a handful of these features is used in SEO software.
First things first, Twitter’s Search API is handy when it comes to analyzing trends and topics. Secondly, Twitter’s Engagement API can provide some valuable insights into what people would or wouldn’t like to see in their feeds; there are currently sixteen kinds of engagement metrics you can use to empower your tool.
Google AdWords API
Unless you’re new to the basics of search engine optimization, there’s no need to explain what Google Adwords is. However, AdWords API seems to be the uncharted waters for many SEO enthusiasts.
Google AdWords API allows integrating data from AdWords platform directly into applications. It not only helps to manage various AdWords accounts and campaigns more efficiently, but also provides tremendous opportunities to developers of third-party software including automated generation of keywords and destination URLs, key marketing metrics (Search Volume, CPC, Competition), location targeting, customizable reports, and so on and so forth. What’s more, the SOAP interface of AdWords API is supported by six popular programming languages, comprising PHP, Java, Python, .NET, Ruby, and Perl.
If the process of authorization weren’t too complicated, AdWords API would make an excellent tool for SEO software development. The point is that one needs to get a developer token in order to use pretty much all Google’s APIs. Although this process itself is not that difficult, the so-called “token review team” rejects all applications that don’t comply with the Terms & Conditions and the Required Minimum Functionality. In fact, 99% of SEO software providers don’t pass token review (any SEO software is considered to be a third-party tool). The good news is that you can apply multiple times: who knows, maybe eventually you will succeed.
Google Analytics API
Analytics API would be useful for those who need to generate detailed ranking reports for keywords and websites’ landing pages. Simply put, it gives a possibility to display the data from Google Analytics and generate reports. What’s more, Google allows to return up to seven different dimensions, including operating system, country, city, landing page, keyword, and device.
Unlike Adwords API, Google Analytics is free and doesn’t require high-level technical expertise. Nevertheless, it works only with websites that you own (or have access to their Google Analytics profile).
Google Custom Search
Google Custom Search allows creating custom search engines that search across particular websites or web pages. On the other hand, it cannot give a precise image of how Google SERP look like for a specific query. Hence, it doesn’t represent any value for SEO tools.
You can use Google Ad Preview Tool to check results pages for a certain keyword, but it involves a lot of manual work (specifying SE, location, language, etc.).
It’s worth noting that Google used to have SOAP Search API. It allowed to access Google index data with vast volumes of queries, which could be used for further analysis and processing, but Google removed it a long time ago.
Making your first API call with DataForSEO
DataForSEO uses REST APIs for interchanging data, meaning that data exchange is done via the HTTPS protocol.
Let’s make keyword research based on the dogtime.com domain, select its most popular keyword (e.g., keyword with the biggest search volume) and get SERP results for it.
We’ll be using Postman. For those of you who aren’t familiar with it, Postman is an easy-to-use HTTP(S) client explicitly developed for testing web applications.
2 Download Postman Examples from DataForSEO Docs.
3 Launch Postman and import DataForSEO collections (JSON file you downloaded at the previous step).
4 Open DataForSEO collection and launch the “kwrd_for_domain_tasks_post” file (find it in the Keywords Data API folder).
5 Specify your DataForSEO credentials in the “Authorization” section.
6 Head to the “Body section” and adjust parameters. In our case, we’ll be looking for keywords that are ranked for the dogtime.com domain in Great Britain.
7 Click Send to request data. Don’t forget to copy your task id; you’ll need it to retrieve the results later.
8 Add a new tab and get back to the DataForSEO collection. Find the kwrd_for_domain_tasks_get/$task_id file and launch it.
9 Modify the GET URL line with your task ID and click Send. Note that you may need to repeat the authorization process.
10 Now we have a list of keywords ranked for the dogtime.com. Let’s pick the one with the highest search volume. In our case, it’s “puppies for sale near me.”
11 Let’s get SERP data for the selected keyword “puppies for sale near me.” In order to do that with Postman, launch the srp_tasks_post file.
12 Head to the “Body” section and adjust parameters. In our case, we’ll be looking for SERP data for the “puppies for sale near me” query. We can also narrow down the search by specifying an exact location; we’ve chosen Brighton, U.K., but you can try it with any other place.
13 Add a new tab and get back to the DataForSEO collection. Find the srp_tasks_get file and launch it.Then modify the GET URL line with your task ID and hit the Send button.
14 Here’s how a simple response of SERP API looks like.
The response is plain and clear. You get URLs, titles, snippets, and all other important data about the results for a high-volume “puppies for sale near me” query in Brighton.
Let’s wrap it up
In this blog post, we’ve looked at how modern APIs work and found a few ways they access data on the web. We hope that it helped you to better understand the basics of the application programming interface and how they can help to devise an efficient SEO software.
If you need more information on this topic, we’d recommend checking out “RESTful Web APIs: Services for a Changing World” by Leonard Richardson, Mike Amundsen, and Sam Ruby. The book will provide you with everything you need to know about the theory and practice of using APIs.