The basic principle of data collection with DataForSEO APIs is pretty straightforward:

1You set a task using any Task POST endpoint of SERP API.

2The API processes your request for some time, depending on the chosen mode or specified priority.

3Once the task is completed, you use its ID to send a request to the GET endpoint and collect the results.

However, knowing these basics isn’t enough when working with large volumes of data.

Depending on how many daily tasks you deal with, your data collection method will differ.

This help center article will walk you through different methods of collecting SERP data from SERP API and help you choose the optimal solution for your specific needs.

Use case #1: your system requires delivering instant results

If your system depends on real-time data, you should use the Live method of delivering results.

Its main advantage is that this method is fast (11 seconds turnaround time on average), and you don’t need to make a separate GET request to retrieve the results. However, there is also a significant drawback — you can set only one task at a time.

Note that the Live method is more expensive than the Standard method. You can learn more about the prices for creating tasks with the Live method on the pricing page.

Working with the Live method is easy:

1Set one task and send the POST request to any live endpoint of SERP API.

2Wait up to 11 seconds for the API to process your request and provide the results.

This method is suitable for a small number of tasks: approximately up to 50 tasks per minute or several thousand tasks a day.

Use case #2: you deal with a medium number of tasks and don’t require instant results

If you don’t need instant results, use the Standard method. While it is slower than the Live method, it enables you to work with larger volumes of data — you can set up to 100 tasks per one request.

The working process is simple:

1Set up to 100 tasks per request and send your requests to the Task POST endpoint.

2Send a request to the Task Ready endpoint every minute (that way, you can collect IDs of up to 1000 tasks per minute).

3Use the IDs of completed tasks from the Task Ready endpoint to collect the results from the Task Get endpoint.

This method is great for a medium number of tasks: approximately up to 1000 tasks per minute or 100,000 tasks per day.

Use case #3: you deal with a large number of tasks and don’t require instant results

If you need to collect SERP data in large volumes, you should leverage pingbacks and postbacks.

Commonly, pingback is known as a callback function, and postback is referred to as a webhook or HTTP push. In DataForSEO API, the pingback function is a notification sent to your server upon the completion of a task, while postback is a function that sends the results to your server.

Using pingbacks and postbacks will help you avoid getting confused and ensure you don’t lose any important piece of data.

Pingbacks and postbacks will enable you to work with more than 1000 tasks per minute or 100,000 tasks a day.

You can learn more about these parameters in this help center article.

The working with postbacks looks the following way:

1You set up to 100 tasks per request, adding the postback_url to each request and specifying your server’s URL in it. Don’t forget to add the postback_data field and specify the datatype you want to be sent to your server. Possible values: regular, html, advanced.

2You send your request to the Task POST endpoint.

3Once the tasks are completed, the API sends them to your server.

4Your server processes the results.

If there is an issue with sending the results, you can collect the tasks that weren’t sent to your server using the Task Ready endpoint.

As for pingbacks, the working process is also simple:

1You set up to 100 tasks per request, adding the pingback_url field to each request and specifying the URL you want the notifications to be sent to.

2You send your request to the Task POST endpoint.

3Once the tasks are completed, the API sends their IDs to the URL specified in the pingback_url field.

4Your server processes the requests, saving the IDs of the completed tasks.

5You collect the IDs and use them to retrieve the results from the Task GET endpoint (or your server retrieves them for you).

In case you have any questions regarding building the optimal app architecture for your specific needs, don’t hesitate to contact us — we will be happy to help.