Merchant API: Getting Big With E-commerce Data. Part 1: Google Shopping API

Merchant API: Getting Big With E-commerce Data. Part 1: Google Shopping API

"Google

One of the most popular activities on the Web is shopping. This is not surprising given that we can shop anytime — at home, at work, and even without getting out of bed.

It’s only natural that retail businesses had followed up with new needs of their buyers and moved their operations to the Internet. It didn’t take long for a new trend to take its stake of the market: in fact, by the end of 2015 e-commerce sales had already amounted to 8% of total sales.

E-commerce shares

No wonder Google has built its own e-commerce platform. Launched in far 2002, Google Product Search (known as Froogle back then) quickly took off and eventually became the dominant player on the online retail market.

However, as the platform evolved, it became necessary to provide developers with a resource to use its data for creating shopping apps.

The Search API for Google Shopping: Evolution and Collapse

Google launched Search API for Shopping in 2010. It allowed developers to build their applications for running product search and extracting necessary information (such as prices, product descriptions, etc.) from Google Shopping results pages. This API displayed outputs proceeding from data gathered from users’ Merchant Center feeds. What’s more, back then Google could offer a separate Search API for Shopping, which would allow collecting product’s information programmatically.

Nonetheless, all that was too good to last long. In 2013 Google decided to shut down Search API for Shopping, leaving thousands of retailers without a reliable source of the e-commerce intel.

We’re deprecating our Search API for Shopping, which has enabled developers to create shopping apps based on Google’s Product Search data. While we believe in the value this offering provided, we’re shifting our focus to concentrate on creating a better shopping experience for users through Google Shopping. We’ll shut the API down completely on September 16, 2013. – Google Official Blog

Moreover, Google never announced they have plans to provide a replacement. So, the door to Google Shopping data was closed. Permanently.

DataForSEO Google Shopping API: Shifting power back to the community

When Google first announced they would shut down its Search API for Shopping nobody could imagine how big of a gap emerged on its place.

That, however, doesn’t mean the connection point between Google’s e-commerce platform intel and large retailers was lost. The gap got quickly filled up with third-party data providers that pretended to “have an ultimate substitute for Google’s API.”

Here at DataForSEO, we decided to invest our efforts in building something that an overwhelming majority of e-commerce APIs are not capable of – namely, providing our clients with a possibility to retrieve huge volumes of data on a regular basis.

We are able to crawl Google Shopping results pages and deliver real-time results with exceptional speed.

So, before we take you through all the capabilities of DataForSEO’s Google Shopping API, let’s figure out how Google Shopping works.

Google Shopping: What you need to know about it

Since Google Shopping is a paid service, no surprise it’s powered by AdWords. However, there’s also another service involved – viz., the Google Merchant Center. While the first one represents the control panel for your shopping campaigns (including bids, budget, sales analytics and more), the second one is used for managing your product feed.

While the setup of Google Shopping ads is somewhat similar to the standard AdWords ads, there’s one thing that makes them different: feed mastery. At Google Shopping, keywords don’t matter. Google takes into an account your products, your bids, and your website to define what queries can trigger your ads.

Nevertheless, in some cases only queries that contain the “buy” keywords tend to trigger Shopping ads most of the time. For example, let’s try searching for a strawberry jam with the most similar query.

Strawberry Jam SERP

As you see, the results page consists of featured snippets, but there are no Google Shopping ads. Let’s change the query to “buy strawberry jam” and run the search one more time.

Buy Strawberry Jam SERP

Now we have products from Google Shopping on the results page. According to Google Support Page, click through rate of Shopping ads is 300% higher compared to traditional text ads.

We’ve long been working on adding support for Google Shopping rich snippets into the Extra SERP API. It’ll be released in the next couple of weeks.

That doesn’t mean that Google Shopping is just another rich-snippet block in SERPs. Although it may sound a bit left field, Google Shopping is a stand-alone e-commerce platform extending far beyond the traditional ad-based business model. In fact, by adding a Shopping tab to the SERP and building a unique interface for it, Google had quickly repositioned itself as a place where people can actually buy and sell things instead of just talking about how this or that product is greater-cheaper-faster. The recent news about Amazon shutting down its ads on Google Shopping is yet another confirmation of that.

Here’s what you’d see if you clicked on the Shopping tab:

Strawberry Jam SERP

Results page consists exclusively of paid product ads. You can also click on certain results to get more information about a particular product, including its reviews, sellers, similar items, and details.

It’s also worth noting that Google Shopping is currently available in the following countries: Argentina, Australia, Austria, Belgium, Brazil, Canada, Chile, Colombia, Czechia, Denmark, France, Germany, Hong Kong, India, Indonesia, Ireland, Israel, Italy, Japan, Malaysia, Mexico, New Zealand, Norway, the Philippines, Poland, Portugal, the Netherlands, Russia, Saudi, Arabia, Singapore, South Africa, South Korea, Spain, Sweden, Switzerland, Taiwan, Thailand, Turkey, United Arab Emirates, Ukraine, the UK, the US, and Vietnam.

DataForSEO Google Shopping API: What is it capable of?

Although our Google Shopping API can be used for the number of different purposes limited only by your imagination, we must admit that it’s been created for helping retailers to extract competitors’ prices straight from Google Shopping results pages. No wonder, it’s what it does best so far.

So, let’s make competitive price analysis for a “Hartley’s best strawberry jam” using its Google Shopping ID and DataForSEO Google Shopping Shops API.

Note that Google Shopping ID is different from the standard GTIN or MPN identifiers. It’s a unique set of numbers assigned to products by Google and indicated in the URL of the product’s page.

Product Id Google Shopping

1 First things first, you’ll need to configure the request to Google Shopping API. Use Google Shopping ID of the product you’re interested in. Also, don’t forget to specify the location and Search Engine ID. We recommend checking our extensive documentation beforehand.

Request Sample   
{
    "data":
        {
            "874654654":
                {
                    "se_id": 2933,
                    "loc_name_canonical":"London,England,United Kingdom",
                    "product_id": "15056200872717257258"
                }
        }
}
Response Sample
{  
    "status": "ok",
    "results_time": "0.0771 sec.",
    "results_count": 1,
    "results": {
        "874654654": {
            "post_id": 874654654,
            "product_id": "15056200872717257258",
            "task_id": 861406261,
            "se_id": 2933,
            "loc_id": 1006886,
            "status": "ok"
        }
    }
}

2 The response you’ll get reflects the info extracted from the “Online Shops” table at the product’s Google Shopping page.

Strawberry Jam Shops

Request Sample

https://api.dataforseo.com/v2/merchant_google_shopping_shops_tasks_get/861406261

Response Sample

{
    "status": "ok",
    "results_time": "0.0448 sec.",
    "results_count": 6,
    "results": {
        "organic": [
            {
                "task_id": 861406261,
                "post_id": "874654654",
                "se_id": 2933,
                "loc_id": 1006886,
                "product_id": "15056200872717257258",
                "result_position": 1,
                "result_datetime": "2018-07-02 12:45:21 +00:00",
                "result_title": "Hartley's Best Strawberry Jam - Min Basket Order £25.",
                "result_seller_name": "Tesco Grocery",
                "result_base_price": 1,
                "result_total_price": 7,
                "result_currency": "GBP",
                "result_shop_stat": 4.6,
                "result_shop_stat_type": "stars",
                "result_shop_stat_count": null,
                "result_details": null,
                "result_shop_ad_url": "https://www.google.co.uk/aclk?sa=l&ai=DChcSEwjtw531uIDcAhVcZBkKHSr1AYMYABABGgJsZg&sig=AOD64_1opRHmdsLeOVkZ8J3_sM3608AYsg&ctype=5&q=&ved=0ahUKEwjSspv1uIDcAhVBYlAKHTy8CHAQ2ikIJQ&adurl=",
                "result_se_check_url": "https://www.google.co.uk/shopping/product/15056200872717257258/online?&hl=en&gl=GB&gws_rd=cr&uule=w+CAIQIFISCXXeIa8LoNhHEZkq1d1aOpZS"
            },
            {
                "task_id": 861406261,
                "post_id": "874654654",
                "se_id": 2933,
                "loc_id": 1006886,
                "product_id": "15056200872717257258",
                "result_position": 2,
                "result_datetime": "2018-07-02 12:45:21 +00:00",
                "result_title": "Hartley's Best Strawberry Jam - Min Basket Order £25.",
                "result_seller_name": "Ocado",
                "result_base_price": 1.6,
                "result_total_price": null,
                "result_currency": "GBP",
                "result_shop_stat": null,
                "result_shop_stat_type": null,
                "result_shop_stat_count": null,
                "result_details": null,
                "result_shop_ad_url": "https://www.google.co.uk/aclk?sa=l&ai=DChcSEwjtw531uIDcAhVcZBkKHSr1AYMYABADGgJsZg&sig=AOD64_28p88wmQcRPlogS6U_ROzUBZSfcg&ctype=5&q=&ved=0ahUKEwjSspv1uIDcAhVBYlAKHTy8CHAQ2ikIKQ&adurl=",
                "result_se_check_url": "https://www.google.co.uk/shopping/product/15056200872717257258/online?&hl=en&gl=GB&gws_rd=cr&uule=w+CAIQIFISCXXeIa8LoNhHEZkq1d1aOpZS"
            },
            {
                "task_id": 861406261,
                "post_id": "874654654",
                "se_id": 2933,
                "loc_id": 1006886,
                "product_id": "15056200872717257258",
                "result_position": 3,
                "result_datetime": "2018-07-02 12:45:21 +00:00",
                "result_title": "Hartley's Best Strawberry Jam - Min Basket Order £25.",
                "result_seller_name": "Waitrose",
                "result_base_price": 1.66,
                "result_total_price": null,
                "result_currency": "GBP",
                "result_shop_stat": 5,
                "result_shop_stat_type": "stars",
                "result_shop_stat_count": 845,
                "result_details": null,
                "result_shop_ad_url": "https://www.google.co.uk/aclk?sa=l&ai=DChcSEwjtw531uIDcAhVcZBkKHSr1AYMYABAFGgJsZg&sig=AOD64_32C1fFTaRPpWCp6RN492ChqvIjyw&ctype=5&q=&ved=0ahUKEwjSspv1uIDcAhVBYlAKHTy8CHAQ2ikILQ&adurl=",
                "result_se_check_url": "https://www.google.co.uk/shopping/product/15056200872717257258/online?&hl=en&gl=GB&gws_rd=cr&uule=w+CAIQIFISCXXeIa8LoNhHEZkq1d1aOpZS"
            },
            {
                "task_id": 861406261,
                "post_id": "874654654",
                "se_id": 2933,
                "loc_id": 1006886,
                "product_id": "15056200872717257258",
                "result_position": 4,
                "result_datetime": "2018-07-02 12:45:21 +00:00",
                "result_title": "Hartley's Best Strawberry Jam - Min Basket Order £25.",
                "result_seller_name": "BritishCornerShop.co.uk",
                "result_base_price": 2.65,
                "result_total_price": 5.14,
                "result_currency": "GBP",
                "result_shop_stat": 4.7,
                "result_shop_stat_type": "stars",
                "result_shop_stat_count": 4486,
                "result_details": null,
                "result_shop_ad_url": "https://www.google.co.uk/aclk?sa=l&ai=DChcSEwjtw531uIDcAhVcZBkKHSr1AYMYABAHGgJsZg&sig=AOD64_3zljGx7Wsso3krq-fAnWEC0QHCPA&ctype=5&q=&ved=0ahUKEwjSspv1uIDcAhVBYlAKHTy8CHAQ2ikIMQ&adurl=",
                "result_se_check_url": "https://www.google.co.uk/shopping/product/15056200872717257258/online?&hl=en&gl=GB&gws_rd=cr&uule=w+CAIQIFISCXXeIa8LoNhHEZkq1d1aOpZS"
            },
            {
                "task_id": 861406261,
                "post_id": "874654654",
                "se_id": 2933,
                "loc_id": 1006886,
                "product_id": "15056200872717257258",
                "result_position": 5,
                "result_datetime": "2018-07-02 12:45:21 +00:00",
                "result_title": "Hartley's Best Strawberry Jam - Min Basket Order £25.",
                "result_seller_name": "Morrisons.com",
                "result_base_price": 1.58,
                "result_total_price": 6.58,
                "result_currency": "GBP",
                "result_shop_stat": 4.9,
                "result_shop_stat_type": "stars",
                "result_shop_stat_count": 1293,
                "result_details": null,
                "result_shop_ad_url": "https://www.google.co.uk/aclk?sa=l&ai=DChcSEwjtw531uIDcAhVcZBkKHSr1AYMYABAJGgJsZg&sig=AOD64_2hBeGgFiBIz76JMYFloo7g6zJrEQ&ctype=5&q=&ved=0ahUKEwjSspv1uIDcAhVBYlAKHTy8CHAQ2ikINQ&adurl=",
                "result_se_check_url": "https://www.google.co.uk/shopping/product/15056200872717257258/online?&hl=en&gl=GB&gws_rd=cr&uule=w+CAIQIFISCXXeIa8LoNhHEZkq1d1aOpZS"
            },
            {
                "task_id": 861406261,
                "post_id": "874654654",
                "se_id": 2933,
                "loc_id": 1006886,
                "product_id": "15056200872717257258",
                "result_position": 6,
                "result_datetime": "2018-07-02 12:45:21 +00:00",
                "result_title": "Hartley's Best Strawberry Jam - Min Basket Order £25.",
                "result_seller_name": "Best 4 Deals UK",
                "result_base_price": 11.38,
                "result_total_price": 15.67,
                "result_currency": "GBP",
                "result_shop_stat": null,
                "result_shop_stat_type": null,
                "result_shop_stat_count": null,
                "result_details": null,
                "result_shop_ad_url": "https://www.google.co.uk/aclk?sa=l&ai=DChcSEwjtw531uIDcAhVcZBkKHSr1AYMYABALGgJsZg&sig=AOD64_0xe4GzkmJtaBofCERE8awbq0Zk3g&ctype=5&q=&ved=0ahUKEwjSspv1uIDcAhVBYlAKHTy8CHAQ2ikIOQ&adurl=",
                "result_se_check_url": "https://www.google.co.uk/shopping/product/15056200872717257258/online?&hl=en&gl=GB&gws_rd=cr&uule=w+CAIQIFISCXXeIa8LoNhHEZkq1d1aOpZS"
            }
        ]
    }
}

Another way you can use Google Shopping API for is monitoring products’ positions on the Google Shopping results pages. At the same time, it can sometimes get tricky because first off, sellers can’t specify keywords they want their product to be shown for. As you already know, Google automatically determines what keywords this or that product will be ranking for based on the seller’s and product’s descriptions.

Secondly, positions of the items on the Google Shopping results page change incredibly fast. Something as simple as page reload can cause a complete regrouping of results.

Nonetheless, checking up with results is very useful when it comes to monitoring competitor’s products ranking alongside yours for your targeted keywords. You can take these keywords from the Search Terms Reports at Google Adwords and then get Google Shopping results for them through our Google Shopping API.

1 Let’s get a results page for the “buy strawberry jam” query.

Request Sample
{
    "data":
        {
            "874654654":
                {
                    "se_id": 2933,
                    "loc_name_canonical":"London,England,United Kingdom",
                    "key": "buy strawberry jam"
                }
        }
}
Response Sample 
{
    "status": "ok",
    "results_time": "0.0629 sec.",
    "results_count": 1,
    "results": {
        "874654654": {
            "post_id": 874654654,
            "post_key": "buy strawberry jam",
            "task_id": 862013374,
            "se_id": 2933,
            "loc_id": 1006886,
            "key_id": 112839464,
            "status": "ok"
        }
    }
}

2 The response contains TOP100 Google Shopping results for a specified keyword. Each result includes product’s name, description, price, rating, number of reviews, plus both Google Shopping and product page URLs.

Request Sample

https://api.dataforseo.com/v2/merchant_google_shopping_tasks_get/862013374
Response Sample 

{
    "status": "ok",
    "results_time": "0.0424 sec.",
    "results_count": 100,
    "results": {
        "organic": [
            {
                "task_id": 862013374,
                "post_id": "874654654",
                "se_id": 2933,
                "loc_id": 1006886,
                "key_id": 112839464,
                "post_key": "buy strawberry jam",
                "result_position": 1,
                "result_datetime": "2018-07-02 14:31:45 +00:00",
                "result_title": "Stockwell And Co Strawberry Jam 454G",
                "result_description": null,
                "result_product_id": null,
                "result_price": 0.4,
                "result_currency": "GBP",
                "result_stars": null,
                "result_reviews_count": null,
                "result_shop_stat": 4.6,
                "result_shop_stat_type": "stars",
                "result_shop_stat_count": null,
                "result_tags": [],
                "result_spell": null,
                "result_url": "https://www.tesco.com/groceries/en-GB/products/299600086",
                "result_shopping_url": null,
                "result_se_check_url": "https://www.google.co.uk/search?q=buy%20strawberry%20jam&num=100&tbm=shop&tbs=vw:l&hl=en&gl=GB&gws_rd=cr&uule=w+CAIQIFISCXXeIa8LoNhHEZkq1d1aOpZS"
            },
            {
                "task_id": 862013374,
                "post_id": "874654654",
                "se_id": 2933,
                "loc_id": 1006886,
                "key_id": 112839464,
                "post_key": "buy strawberry jam",
                "result_position": 2,
                "result_datetime": "2018-07-02 14:31:45 +00:00",
                "result_title": "Tesco Strawberry Jam 454G",
                "result_description": null,
                "result_product_id": null,
                "result_price": 0.75,
                "result_currency": "GBP",
                "result_stars": null,
                "result_reviews_count": null,
                "result_shop_stat": 4.6,
                "result_shop_stat_type": "stars",
                "result_shop_stat_count": null,
                "result_tags": [],
                "result_spell": null,
                "result_url": "https://www.tesco.com/groceries/en-GB/products/254849603",
                "result_shopping_url": null,
                "result_se_check_url": "https://www.google.co.uk/search?q=buy%20strawberry%20jam&num=100&tbm=shop&tbs=vw:l&hl=en&gl=GB&gws_rd=cr&uule=w+CAIQIFISCXXeIa8LoNhHEZkq1d1aOpZS"
            },
            {
                "task_id": 862013374,
                "post_id": "874654654",
                "se_id": 2933,
                "loc_id": 1006886,
                "key_id": 112839464,
                "post_key": "buy strawberry jam",
                "result_position": 3,
                "result_datetime": "2018-07-02 14:31:45 +00:00",
                "result_title": "Hartleys Best Strawberry Jam 340G",
                "result_description": "Hartleys best strawberry jam340g 340gm hartleys jam is full of the finest fruit to make best tasting jam, the perfect topping ...",
                "result_product_id": "15056200872717257258",
                "result_price": 1,
                "result_currency": "GBP",
                "result_stars": 4.5,
                "result_reviews_count": 6,
                "result_shop_stat": null,
                "result_shop_stat_type": null,
                "result_shop_stat_count": null,
                "result_tags": [],
                "result_spell": null,
                "result_url": "https://www.tesco.com/groceries/en-GB/products/253415519",
                "result_shopping_url": "https://www.google.co.uk/shopping/product/15056200872717257258?q=buy+strawberry+jam&num=100&hl=en&gl=GB&gws_rd=cr&uule=w+CAIQIFISCXXeIa8LoNhHEZkq1d1aOpZS&prds=paur:ClkAsKraX56p3A12n-Yqi4OctmLnSIQlBK9dNvGE0T6rinI-gZuJlP4T2LFiHy_3OP4F8z9yjaJ02af93fXw3iBBy-P1AmHluFqYoZxg8ocoe9X6hK7l0jGa_hIZAFPVH71pa8UsDu3CLXX3Uq2QUCC3FwvBxA&sa=X&ved=0ahUKEwjt-_Pm0IDcAhVML48KHRwWAaYQ8wII3Ag",
                "result_se_check_url": "https://www.google.co.uk/search?q=buy%20strawberry%20jam&num=100&tbm=shop&tbs=vw:l&hl=en&gl=GB&gws_rd=cr&uule=w+CAIQIFISCXXeIa8LoNhHEZkq1d1aOpZS"
            },
            {
                "task_id": 862013374,
                "post_id": "874654654",
                "se_id": 2933,
                "loc_id": 1006886,
                "key_id": 112839464,
                "post_key": "buy strawberry jam",
                "result_position": 4,
                "result_datetime": "2018-07-02 14:31:45 +00:00",
                "result_title": "Bonne Maman Strawberry Conserve 370G",
                "result_description": "Bonne Maman Raspberry Conserve 750gm Rich and delicious because they are made with only the highest quality raspberries and ...",
                "result_product_id": "17146491114766766681",
                "result_price": 2,
                "result_currency": "GBP",
                "result_stars": 5,
                "result_reviews_count": 2,
                "result_shop_stat": null,
                "result_shop_stat_type": null,
                "result_shop_stat_count": null,
                "result_tags": [],
                "result_spell": null,
                "result_url": "https://www.tesco.com/groceries/en-GB/products/250782952",
                "result_shopping_url": "https://www.google.co.uk/shopping/product/17146491114766766681?q=buy+strawberry+jam&num=100&hl=en&gl=GB&gws_rd=cr&uule=w+CAIQIFISCXXeIa8LoNhHEZkq1d1aOpZS&prds=paur:ClkAsKraX-bw3NGmg1r1FFaVH3D-iVnhMs0R8nGXoIepdiLf5mRx-M_cNnjILP_5fy62_L2ODWTMf4r32lvhf_d8SqVeFzCLhsvX_PciBi53VA2ucnS2uGVj3RIZAFPVH70V9Xhoo9vlk-z4uy-ZfnLRlxRKHg&sa=X&ved=0ahUKEwjt-_Pm0IDcAhVML48KHRwWAaYQ8wII6Ag",
                "result_se_check_url": "https://www.google.co.uk/search?q=buy%20strawberry%20jam&num=100&tbm=shop&tbs=vw:l&hl=en&gl=GB&gws_rd=cr&uule=w+CAIQIFISCXXeIa8LoNhHEZkq1d1aOpZS"
            },
…

           {
                "task_id": 862013374,
                "post_id": "874654654",
                "se_id": 2933,
                "loc_id": 1006886,
                "key_id": 112839464,
                "post_key": "buy strawberry jam",
                "result_position": 100,
                "result_datetime": "2018-07-02 14:31:45 +00:00",
                "result_title": "Tesco Fresh Cream Strawberry Jam Doughnut 2 Pack",
                "result_description": "2 Doughnuts filled with stabilised cream and strawberry jam. Light and fluffy doughnuts generously filled with fresh cream and ...",
                "result_product_id": null,
                "result_price": 1.4,
                "result_currency": "GBP",
                "result_stars": null,
                "result_reviews_count": null,
                "result_shop_stat": 4.6,
                "result_shop_stat_type": "stars",
                "result_shop_stat_count": null,
                "result_tags": [],
                "result_spell": null,
                "result_url": "https://www.tesco.com/groceries/en-GB/products/254985809",
                "result_shopping_url": null,
                "result_se_check_url": "https://www.google.co.uk/search?q=buy%20strawberry%20jam&num=100&tbm=shop&tbs=vw:l&hl=en&gl=GB&gws_rd=cr&uule=w+CAIQIFISCXXeIa8LoNhHEZkq1d1aOpZS"
            }
        ],
        "paid": [],
        "extra": {}
    }
}

The bottom line

Historically, Google APIs were considered a safe bet. Many of us have relied on them when building our websites and web apps. However, if we take an example of Search API for Google Shopping – it’s no longer the case. Google wants people to use its services for searching and shopping, so the shutdown of services that could possibly make people follow a different path when looking for products online came as no surprise.

Nonetheless, we are ready to provide you with a reliable alternative. Google Shopping provides all the data you need to fill in the extensive product list with all the necessary details, and run price monitoring of shops. You can use any programming language (check our documentation for examples in PHP, Python, Csharp, and Java) to develop an app that would update product info automatically.

Get free credits today!

Post a Comment