Google AI Overview Scraper

Google AI Overview Scraper - A tool to scrape Google's AI Overview (formerly SGE) results with a simple API. Get AI-generated summaries, text blocks, and reference citations from Google's AI-powered search features.

We provide the results in a structured JSON format, eliminating the need for parsing, coding, proxies, or any other web scraping headaches for developers.

This engine is used only when Google requires a separate request to fetch AI Overview results. See AI Overview Results for results embedded in the main search results.

How to scrape Google AI Overview?

The Google AI Overview can be obtained from the Google Search API. In many cases, the AI Overview content is returned directly. If a page_token is returned instead, a second API call is needed to fetch the full content:

https://serpapi.com/search.json?engine=google&q=what+is+machine+learning&api_key=YOUR_API_KEY

Register for free at SerpApi to get your API Key
The ai_overview object may contain the content directly, or a page_token for fetching expanded content.
If page_token is present, use the Google AI Overview API to fetch the full content (token expires within 1 minute).

Code examples

Here are some code examples based on your favorite programming languages.

cURL Integration

# First, search Google to get AI Overview
curl --get https://serpapi.com/search \
 -d engine="google" \
 -d q="what is machine learning" \
 -d api_key="secret_api_key"

# If page_token is returned, fetch expanded content
curl --get https://serpapi.com/search \
 -d engine="google_ai_overview" \
 -d page_token="YOUR_PAGE_TOKEN" \
 -d api_key="secret_api_key"

Python Integration

Step 1: Create a new main.py file.

Step 2: Install requests package with:

pip install requests

Step 3: Add this code to your file:

import requests
SERPAPI_API_KEY = "YOUR_SERPAPI_API_KEY"

# Step 1: Search Google to get AI Overview
search_params = {
    "api_key": SERPAPI_API_KEY,
    "engine": "google",
    "q": "what is machine learning"
}

search = requests.get("https://serpapi.com/search", params=search_params)
search_response = search.json()

# Step 2: Check if AI Overview is available
if "ai_overview" in search_response:
    ai_overview = search_response["ai_overview"]

    # Check if page_token exists (need second call for expanded content)
    if "page_token" in ai_overview:
        page_token = ai_overview["page_token"]

        ai_params = {
            "api_key": SERPAPI_API_KEY,
            "engine": "google_ai_overview",
            "page_token": page_token
        }

        ai_search = requests.get("https://serpapi.com/search", params=ai_params)
        ai_response = ai_search.json()
        print(ai_response["ai_overview"])
    else:
        # AI Overview content is already available directly
        print(ai_overview)
else:
    print("No AI Overview available for this query")

JavaScript Integration

Step 1: Install the SerpApi JavaScript package:

npm install serpapi

Step 2: Create a new index.js file.

Step 3: Add this to your file:

const { getJson } = require("serpapi");

// Step 1: Search Google to get AI Overview
getJson({
  api_key: API_KEY,
  engine: "google",
  q: "what is machine learning"
}, (searchJson) => {
  // Step 2: Check if AI Overview is available
  if (searchJson.ai_overview) {
    // Check if page_token exists (need second call for expanded content)
    if (searchJson.ai_overview.page_token) {
      getJson({
        api_key: API_KEY,
        engine: "google_ai_overview",
        page_token: searchJson.ai_overview.page_token
      }, (aiJson) => {
        console.log(aiJson["ai_overview"]);
      });
    } else {
      // AI Overview content is already available directly
      console.log(searchJson.ai_overview);
    }
  } else {
    console.log("No AI Overview available for this query");
  }
});

Other Programming Languages

While you can use our APIs using a simple GET request with any programming language, you can also see our ready-to-use libraries here: SerpApi Integrations.