In this tutorial, we will explore how to harness and format the wealth of data provided by Google for external sources, explicitly focusing on scraping Google Shopping and its product pages. To achieve this, we will use iWeb Data Scraping API designed to scrape Google page results.
We aim to extract Google shopping results such as price, description, reviews, title, price comparisons with other online stores, and additional product suggestions for the "DeWalt DCD771C2" product.
Please note that this API offers two distinct APIs targeting shopping and products: (1) the Google Shopping Results API and (2) the Google Product Results API. We will solely focus on the latter to extract product data from Google Shopping.
Both APIs extract shopping/product data from Google but serve different purposes. The Google Shopping Results API scrapes search results from querying Google Shopping. Depending on whether Google recognizes those constraints, it allows you to apply filters based on price, seller, and specific parameters and features unique to each product category. For instance, when searching for the DeWalt drill mentioned earlier, unique parameters returned could include battery features, chuck size of drills, weight, power type, cordless or corded, etc. These parameters will vary depending on the product type you are searching for.
With the Google Product Results API, you can scrape data directly from the specific Google product page associated with each product box in the Google Shopping search results. Each product box in the search results will link to its respective product page, and this API enables you to extract information from those individual pages.
A Google product page is a distinct component of Google's Shopping platform, dedicated to each product and characterized by several unique identifying properties. Conceptually, you can view a product page as a dedicated landing page for a particular product. These pages encompass essential details such as the product's title, price, rating, reviews, description, specifications, features, price comparisons from other online stores, related products from the same manufacturer, and similar offerings from competing manufacturers. A unique ID number identifies each product listed on these pages.
Google product pages are accessible in two ways: directly navigating to the dedicated page for a specific product or following a link from the search results on Google Shopping when searching for that particular product.
To begin with, scraping product data from Google Shopping, go to Google's Shopping page and enter the search query DeWalt DCD771C2. The first result displayed on the page will be our focus. Clicking on the product's title will expand the result box, revealing more details about the item. Two crucial links are at the bottom of this product box: "Related Items" and "Reviews." Clicking on either of these links will lead you to the unique stand-alone page dedicated to that specific product, as shown in the image above.
Our documentation provided a screenshot highlighting the various scrape-able sections of the product pages. Additionally, we offer a comprehensive breakdown of the entire page, presenting a detailed list of all available extracted parameters. The data are available from the Google Shopping scraper returns in JSON format for easy understanding. To access this information, please refer to the linked resource, which offers a clear overview of the product page structure and the corresponding
data in JSON format
Let's utilize our playground to simulate a search and retrieve the scraped data. The playground is a user-friendly dashboard, granting convenient access to our APIs and their respective parameters. To proceed, ensure that you set the search type at the top left corner of the playground to "Google Product" for this specific task. Once you execute the search, the playground generates a link containing the scraped data in HTML and JSON formats. If you haven't used our playground before, it offers a seamless and efficient method to interact with our APIs and obtain the desired information.
Next, discuss the search field of "Product ID" (product_id). This field is crucial for identifying the specific queried product and only accepts the unique Product ID assigned by Google. You can find this Product ID in the respective product's page URL. Look for the number immediately following "product/" in the URL. For example, in this case, the Product ID is "2478210754218635618."
After setting all the parameters, click on the "Search" button. The API will respond with two visuals displayed on the page - the results in HTML and JSON formats. To access the links providing these results, click the "Export To Code" button in the top-right corner. A drop-down box will appear, offering links to the HTML and JSON URLs and the code version of the query parameters in eight different languages/environments. This feature provides convenient access to the data obtained from your query in various formats and programming languages.
You can access the scraped data results by clicking on either the HTML or JSON URLs provided below:
JSON —
HTML —
Congratulations! You have successfully utilized Google Product API to scrape and obtain valuable data. The potential applications and possibilities are virtually limitless with this powerful tool.
For further details, contact iWeb Data Scraping now! You can also reach us for all your web scraping service and mobile app data scraping needs