Search by Image API: Take Full Advantage of Reverse Image Search

Google introduced the Google Images search engine back in 2001. For the first ten years since its release, it supported a keyword-based search only, providing images relevant to user queries.

However, that approach was often not sufficient. For instance, a user might want to find a very specific image but couldn’t come up with a term for which Google would return what they wanted.

That was one of the reasons why ten years later, in 2011, Google expanded the functionality of Google Images. The company added the “Search by Image” feature, which allowed reverse image search.

In short, reverse image search is the process of finding images using an image as the starting point rather than a written search query.

Today, anyone can initiate a search session by uploading an image to Google Images and discover visually similar pictures as well as pages where the specified image or its modified versions show up.

With DataForSEO Search by Image API, you can capitalize on that functionality and build several solutions around it. Read further to learn how the API works and, more importantly, what you can do with the data it offers.

How DataForSEO Search by Image API works?

Search By Image API returns up to 700 search results for the image specified in a POST request.

To make an API call, you must indicate the image URL, the country from which you would like to emulate a search session, and its language.

Your POST request should be structured as in the following example:

[
    {
        "language_code": "en",
        "location_code": 2840,
        "image_url": "https://upload.wikimedia.org/wikipedia/commons/1/19/Eiffel_Tower_at_Night.jpg"
    }
]

You can get even more specific and set the exact browser screen resolution to imitate — learn more about additional POST request parameters on our documentation page.

When you send a request, the API emulates set parameters and initiates a search session on the Google Images search engine. Then it scrapes the data on the returned results page and provides it in convenient JSON format.

To help you understand the data, let us refer to Google’s “Search by Image” feature. We will show you what SERP elements it returns and how they are displayed in the API response.

Let’s start a search session by uploading a photo of the Eiffel Tower.

Google Images will provide us with four types of SERP elements:

• Possible related search.
• Pages relevant to the possible related search.
• Visually similar images.
• Pages that include matching images.

1 At the top of the page, Google suggests a possible related search query relevant to the uploaded image.

In the API response, it is displayed as “keyword”.

Example:

{
  "keyword": "eiffel tower"
}

2 Under the possible related search element come organic results relevant to the suggested keyword.

Organic results are displayed as “type”: “organic” in the API response.

Example:

{
  "type": "organic",
  "rank_group": 1,
  "rank_absolute": 1,
  "position": "left",
  "xpath": "/html[1]/body[1]/div[7]/div[1]/div[10]/div[1]/div[1]/div[2]/div[2]/div[1]/div[1]/div[1]/div[1]",
  "domain": "www.toureiffel.paris",
  "title": "The OFFICIAL Eiffel Tower website: tickets, news, info...",
  "url": "https://www.toureiffel.paris/en",
  "cache_url": "https://webcache.googleusercontent.com/search?q=cache:GyN0RQzr504J:https://www.toureiffel.paris/en+&cd=1&hl=en&ct=clnk&gl=us",
  "related_search_url": null,
  "breadcrumb": "https://www.toureiffel.paris › ...",
  "is_image": false,
  "is_video": false,
  "is_featured_snippet": false,
  "is_malicious": false,
  "is_web_story": false,
  "description": "Don't miss · The Tower. History, key figures, lights, paintings, explore all the secrets of the world's most iconic monument · Restaurants & stores. On every ...",
  "pre_snippet": null,
  "extended_snippet": null,
  "images": null,
  "amp_version": false,
  "rating": null,
  "price": null,
  "highlighted": [
    "Tower"
  ],
  "links": null,
  "faq": null,
  "extended_people_also_search": null,
  "about_this_result": null,
  "related_result": null,
  "timestamp": null,
  "rectangle": null
}

3 Google may also show a knowledge graph in the upper right corner of the page when available.

Knowledge graph is displayed as “type”: “knowledge_graph”.

Example:

{
  "type": "knowledge_graph",
  "rank_group": 1,
  "rank_absolute": 1,
  "position": "right",
  "xpath": "/html[1]/body[1]/div[7]/div[1]/div[10]/div[2]/div[1]",
  "title": "Eiffel Tower",
  "sub_title": null,
  "description": "Description The Eiffel Tower is a wrought-iron lattice tower on the Champ de Mars in Paris, France. It is named after the engineer Gustave Eiffel, whose company designed and built the tower. Wikipedia",
  "card_id": null,
  "url": null,
  "image_url": null,
  "logo_url": null,
  "cid": null,
  "items": [
    {
      "type": "knowledge_graph_description_item",
      "rank_group": 0,
      "rank_absolute": 0,
      "position": "left",
      "xpath": "/html[1]/body[1]/div[7]/div[1]/div[10]/div[2]/div[1]/div[1]/div[1]/div[1]/div[1]/div[2]/div[1]/div[1]",
      "text": "Description The Eiffel Tower is a wrought-iron lattice tower on the Champ de Mars in Paris, France. It is named after the engineer Gustave Eiffel, whose company designed and built the tower. Wikipedia",
      "links": [
        {
          "type": "link_element",
          "title": "Wikipedia",
          "url": "https://en.wikipedia.org/wiki/Eiffel_Tower",
          "domain": null,
          "snippet": null,
          "xpath": "/html[1]/body[1]/div[7]/div[1]/div[10]/div[2]/div[1]/div[1]/div[1]/div[1]/div[1]/div[2]/div[1]/div[1]/div[1]/div[1]/div[1]/span[2]/a[1]"
        }
      ],
      "rectangle": null
    }
  ]
}

4 In the middle of the results page, the search engine shows images that are visually similar to the uploaded one. As you can see, we used a photo of the Eiffel Tower at night, and Google found other shots of this landmark also captured at night-time.

In the API response, these elements are displayed as “type”: “images_element” — you can find their URLs within the “items” array under the “title”: “Visually similar images” field.

Example:

{
  "type": "images",
  "rank_group": 1,
  "rank_absolute": 3,
  "position": "left",
  "xpath": "/html[1]/body[1]/div[7]/div[1]/div[10]/div[1]/div[1]/div[2]/div[2]/div[1]/div[1]/div[2]/div[1]/div[2]",
  "title": "Visually similar images",
  "url": "https://www.google.com/search?num=100&hl=en&gl=US&tbs=simg:CAESYQmpVZa3iFPAsxpWCxCwjKcIGjoKOAgEEhTbAa8DnBmJJvYXvB_1iI7MDpCXPDRoaQyVX0iaN6fKfoVg3IwHDwzxjqpwZN-CofGIgBTAEDAsQjq7-CBoKCggIARIE1r6bhQw&q=eiffel+tower&tbm=isch&sa=X&ved=2ahUKEwi2177SyIf1AhUW73MBHZZhBOkQjJkEegQIJBAC",
  "items": [
    {
      "type": "images_element",
      "alt": "Image result",
      "url": "https://www.earthtrekkers.com/paris-bucket-list-best-experiences-paris/",
      "image_url": "https://api.dataforseo.com/cdn/i/12290042-2806-0066-0000-ed5165ca4028:0"
    },
    {
      "type": "images_element",
      "alt": "Image result",
      "url": "https://theconstructor.org/case-study/eiffel-tower-construction-features/75182/",
      "image_url": "https://api.dataforseo.com/cdn/i/12290042-2806-0066-0000-ed5165ca4028:1"
    }
  ]
}

5 At the bottom of SERP, Google provides pages that include the uploaded image or its modified version. There may be thousands of pages that store the same picture, and Google will display them all under the “Pages that include matching images” header.

In the API response, these elements are displayed as “type”: “organic” — they come right after the “visually similar images” elements and include URLs of matching images.

Example:

{
  "type": "organic",
  "rank_group": 4,
  "rank_absolute": 6,
  "position": "left",
  "xpath": "/html[1]/body[1]/div[7]/div[1]/div[10]/div[1]/div[1]/div[2]/div[2]/div[1]/div[1]/div[3]/div[3]",
  "domain": "www.globalblue.com",
  "title": "Top 10 facts about the Eiffel Tower - Global Blue",
  "url": "https://www.globalblue.com/destinations/france/paris/top-ten-facts-about-the-eiffel-tower",
  "cache_url": "https://webcache.googleusercontent.com/search?q=cache:qx82I8TxqmkJ:https://www.globalblue.com/destinations/france/paris/top-ten-facts-about-the-eiffel-tower+&cd=32&hl=en&ct=clnk&gl=us",
  "related_search_url": null,
  "breadcrumb": "https://www.globalblue.com › ... › France › Paris",
  "is_image": true,
  "is_video": false,
  "is_featured_snippet": false,
  "is_malicious": false,
  "is_web_story": false,
  "description": "160 × 241 · Apr 8, 2014 — The Eiffel Tower is one of the most recognisable landmarks in the world. Situated on the Left Bank in the heart of Paris, it draws millions ...",
  "pre_snippet": "160 × 241",
  "extended_snippet": null,
  "images": [
    {
      "type": "images_element",
      "alt": null,
      "url": null,
      "image_url": "https://www.globalblue.com/destinations/france/paris/article299668.ece/alternates/PORTRAIT1_160/eiffel_tower_paris_06.jpg"
    }
  ],
  "amp_version": false,
  "rating": null,
  "price": null,
  "highlighted": [
    "Eiffel Tower"
  ],
  "links": null,
  "faq": null,
  "extended_people_also_search": null,
  "about_this_result": null,
  "related_result": null,
  "timestamp": null,
  "rectangle": null
}

Having this data at your disposal, you can use it in various ways. Let us demonstrate what you can do with it — the first use case will be a reverse image search tool.

Reverse image search tool

Software providers offer a great variety of tools that work on the same principle as Google’s Search by Image feature. One of the most popular solutions is arguably TinEye.

TinEye is an image search and recognition company that offers computer vision services to businesses. Its tool allows users to specify the URL of any image and find pages where that image or its changed version appears.

If you’re unfamiliar with such software, you must be wondering how it can help businesses. Let us show you three examples.

#1 Protecting intellectual property

Photographs, illustrations, and other images are protected by copyright as artistic works. It implies that a user needs the copyright owner’s permission to perform certain acts, such as copying the image, using it for commercial purposes, or simply sharing it on the internet.

One of the ways photographers and designers make money is by selling licenses for their works. In other words, they provide permission to use their intellectual property in exchange for remuneration.

However, people often use copyrighted images without bothering to contact owners for permission, which results in financial losses for creators.

With the reverse image search, creators can tackle the issue.

Such tools as TinEye enable photographers and designers to track their works on the web and find website owners who use them illegally. That allows them to contact violators and either ask them to remove the copyrighted images or negotiate deals.

#2 Building a stronger backlink profile

Another way to use a reverse image search tool is to build more backlinks. With a list of pages that have published their image, a creator can contact website owners and ask them to link to the original image source. That way, they will build a stronger backlink profile, which, in turn, will help them rank better.

This method is widely used by SEO professionals: Neil Patel, one of our clients, leverages reverse image search to acquire 26% more backlinks. According to him, original images that use data, facts, infographics, graphs, or charts are more likely to be copied, so a creator should regularly track those via reverse image search software.

#3 Measuring image impact

Besides likes, comments, and shares on social media, an image creator can monitor the presence of their work on the web to see how it resonates with people. If the image is re-uploaded by hundreds of websites, it indicates that users like it and find it interesting. Along with social media signals, this can tell more about the picture’s impact on people.

How to build a reverse image search tool with Search by Image API

Using Search by Image API, you could easily develop a reverse image search tool like TinEye yourself.

As mentioned above, Google displays pages that store the specified image or its modified version under the “Pages that include matching images” header. They are displayed in the API response as “type”: “organic” and come right after the “visually similar images” elements.

Example:

{
  "type": "organic",
  "rank_group": 7,
  "rank_absolute": 9,
  "position": "left",
  "xpath": "/html[1]/body[1]/div[7]/div[1]/div[10]/div[1]/div[1]/div[2]/div[2]/div[1]/div[1]/div[3]/div[6]",
  "domain": "www.scholastic.com",
  "title": "This Week From Bedtime Math: Paint the Tower Red | Scholastic",
  "url": "https://www.scholastic.com/parents/school-success/learning-toolkit-blog/week-bedtime-math-paint-tower-red.html",
  "cache_url": "https://webcache.googleusercontent.com/search?q=cache:HRIsai84H-UJ:https://www.scholastic.com/parents/school-success/learning-toolkit-blog/week-bedtime-math-paint-tower-red.html+&cd=36&hl=en&ct=clnk&gl=us",
  "related_search_url": null,
  "breadcrumb": "https://www.scholastic.com › learning-toolkit-blog › we...",
  "is_image": true,
  "is_video": false,
  "is_featured_snippet": false,
  "is_malicious": false,
  "is_web_story": false,
  "description": "1186 × 1858 · Jun 17, 2014 — Have you ever wished you could paint a famous building a different color? Find out how long it takes to paint the Eiffel Tower in today's ...",
  "pre_snippet": "1186 × 1858",
  "extended_snippet": null,
  "images": [
    {
      "type": "images_element",
      "alt": null,
      "url": null,
      "image_url": "http://www.scholastic.com/content/dam/parents/migrated-assets/blogs/header-images-5/06172014-Eiffel-Tower-bedtime-math.jpg"
    }
  ]
}

All you have to do is extract their URLs and display them in your software so that your customers can use this data for their own purposes.

Consider this, and we, in the meantime, will show you another use case of Search by Image API.

Image recognition with neural networks

Image recognition neural networks are getting more sophisticated, helping companies across industries solve various problems. For example, eBay leverages them to create a seamless shopping experience for its customers. The company has developed two features that give it a significant advantage over competitors:

• Find It On eBay.
• Image Search.

Find It On eBay allows shoppers to share images with eBay via its mobile app. Once the app receives an image, it finds product listings of the item depicted in it.

Images Search enables consumers to take photos of something they want to purchase and use them to search for similar products on the marketplace.

By supporting these features, eBay allows customers to find what they’re interested in without bothering to enter keywords. That way, the company dramatically improves the shopping experience and reduces customer friction, which leads to more sales.

How Search by Image API can help you train your own neural network model

While image recognition neural networks are undoubtedly valuable for businesses, training them is no easy feat.

To make your model recognize a specific object in a picture, you need to provide it with a dataset of images that depict that object. What’s more, you also have to annotate each item from your dataset to let the training model know what the important parts (classes) the image contains. That way, it can later use those notes to identify classes in a new, never-before-seen image.

Now suppose that you have a dataset of 100,000 images. Manually annotating them would take months of hard work, so it’s better to find an automated solution. That’s when DataForSEO’s Search by Image API comes in handy.

As mentioned above, Google suggests a related keyword when a user performs a reverse image search. This keyword often describes what is displayed in the uploaded image — and Google is pretty good at identifying objects.

Consequently, you can use Search by Image API to annotate items from your dataset. Its response will return the keyword for any image you specify in a POST request.

Example:

{
                    "image_url": "https://i.ytimg.com/vi/iPW75ZO4pIA/maxresdefault.jpg",
                    "keyword": "dog playing with baby",
                    "type": "search_by_image",
                    "se_domain": "google.com.ua",
                    "location_code": 2804,
                    "language_code": "en",
                    "check_url": "https://www.google.com.ua/searchbyimage?image_url=https%3A%2F%2Fi.ytimg.com%2Fvi%2FiPW75ZO4pIA%2Fmaxresdefault.jpg&hl=en&gl=UA&gws_rd=cr&ie=UTF-8&oe=UTF-8&uule=w+CAIQIFISCY8OcFTB2dFAERBA9oiEBgEA&num=1",
                    "datetime": "2021-12-28 17:24:42 +00:00",
                    "spell": null,
                    "item_types": [
                        "organic",
                        "images"
                    ]
}

You could write a script that would extract keywords from API responses and use them to annotate your images.

Please note that Google’s image recognition algorithm is not flawless. Sometimes Google fails to identify an object depicted in the uploaded image and thus suggests a wrong related keyword.

That’s why you should always validate your image annotations before using datasets for training your neural network model.

Conclusion

As you learned from this article, Search by Image API can be helpful in many ways. Whether you are planning to build a reverse image search tool or looking for an automated solution to annotate images from your dataset, our API will suit your needs.

So don’t hesitate and create your account today to get a $1 credit for your first API calls!