AI visibility data tells you how often ChatGPT and Google AI Overviews mention your brand, which of your pages they cite, and which competitors show up in the same answers. In most AI visibility platforms, that data stays inside the vendor’s dashboard. That works until someone asks whether the pages AI cites are also the ones bringing organic traffic, or wants AI mentions in the same Looker or Power BI report as the rest of your marketing data. At that point, the data needs to live in your own warehouse.
This guide shows how to set up a scheduled AI visibility data export into BigQuery, Snowflake, or Redshift, join it with SEO and traffic metrics, and feed it to Looker, Power BI, or Tableau.
One thing to know up front: there is no one-click connector. The export runs through our LLM Mentions API: your scheduler calls it, and your script loads the response into your tables. It takes a short script instead of a toggle, but you decide which targets to pull, how often, and how the tables are modeled. The full script is below.
Can you export AI visibility metrics for your own analysis?
Yes. Many AI visibility platforms show metrics only in their own interface, but the LLM Mentions API returns everything as structured JSON, so every metric can be stored, modeled, and joined like any other dataset in your AI SEO analytics stack. The data comes in two shapes:
- Mention-level records. The Search Mentions endpoint returns individual AI responses that match your target, including quoted links. Useful for content audits, heavier to store.
- Ready-made aggregates. The other endpoints return pre-counted mentions and AI search volume, grouped by target, platform, location, language, page, domain, or brand. Most pipelines need only these.
Here’s how the aggregate endpoints map to the questions a report usually answers:
- How visible are we overall? Target Metrics returns mentions and AI search volume by platform, location, and language. Multi-Target Metrics compares 2 to 10 keyed target sets in one call, such as your brand against four competitors.
- Which pages and domains do AI answers cite? Top Mentioned Pages and Top Mentioned Domains.
- Which brands appear next to ours? Top Mentioned Brands and Top Mentioned Brand Categories.
- How is visibility changing? Historical returns month-by-month mentions and AI search volume, with data from August 2025. Timeseries Delta returns the net change, and Timeseries New and Lost splits it into gained and lost mentions, grouped by day, week, month, or year. Our guide to historical mentions data covers all three.
- What’s easiest to load? Lite versions of Target Metrics and the four Top Mentioned endpoints return the same data in a flattened format at the same price. Each item maps to one table row, so the pipeline below uses them.
A few coverage facts shape the pipeline design:
- Two platforms: Google AI Overview (
"google") and ChatGPT ("chat_gpt"). Omitplatformto get both. - ChatGPT data is US-only: location code 2840 (United States) and English. Other markets get Google AI Overview data only.
- Live method only: if we’re speaking strictly about delivery, the LLM Mentions API returns results in the response itself, with no task queue to poll. A call can take up to 120 seconds.
- Up to 10 entities per target: each request takes one
targetarray of up to 10 domain or keyword entities, each with anincludeorexcludefilter. - 60-day update cycle: the database updates continuously, and a full refresh cycle takes 60 days, which affects how often to pull.
You can build and test the whole pipeline for free in our Sandbox. It returns dummy data in the same structure as live responses; the only change is the hostname, sandbox.dataforseo.com instead of api.dataforseo.com.
How to pull AI search visibility data into your own data warehouse
The pattern is the same whichever warehouse you use. A scheduler (cron, Airflow, Cloud Scheduler with Cloud Run, or Amazon EventBridge with Lambda) triggers a script. The script makes one Live request per target and endpoint, turns each response item into a row with a snapshot date, and loads the rows into date-partitioned tables. Looker, Power BI, or Tableau then read those tables, or views built on top of them.
Step 1: Define your targets
Decide what you track before writing any code. A target can be a domain (example.com) or a keyword such as your brand name, and each tracked brand gets its own key in your config. For domain targets, "search_scope": ["sources"] counts only answers that cite the domain as a source, which is usually what “AI visibility” means in a report. Our guide to LLM citation data explains the difference between sources and other search results in more detail.
Then fix the market. To compare Google AI Overview and ChatGPT side by side, use location code 2840 and language code en. To pull KPIs for several brands in one request, you can use Multi-Target Metrics; the script below loops over targets one at a time instead, which keeps each response small and easy to flatten.
Step 2: Choose a cadence
Weekly snapshots are a reasonable default. With a 60-day refresh cycle, daily pulls of the same target are likely to return near-identical numbers, multiplying cost without adding much signal.
For trend lines, you don’t need months of accumulated snapshots. The Historical endpoint already returns month-by-month data, so a one-time backfill call per target and platform gives you a trend chart on day one. Start with low-frequency pulls and increase them only if your reports need it.
Step 3: Model the tables
Three tables cover most reporting needs:
llm_mentions_snapshot: one row per snapshot date, target, platform, location, and language, withmentionsandai_search_volume. Source: Target Metrics Lite.llm_top_pages: the same keys pluspage, one row per cited page. Source: Top Mentioned Pages Lite.llm_mentions_monthly: one row per month, target, and platform. Source: Historical.
Partition the snapshot tables by snapshot_date. Queries then scan only the dates they need, and each run can overwrite just its own day, so a re-run never duplicates rows.
A note on formats: appending .ai to an endpoint URL returns our AI-optimized response at no extra cost. It trims payloads by keeping only id, status_code, and status_message as service fields and dropping empty, null, or false values. It also changes the response shape and sets the default limit to 10, so if you switch, read items from the top level, set limit explicitly, and treat missing keys as nulls. The script below uses the standard format.
Step 4: Load the data into BigQuery
The script below runs one AI visibility data export. It calls Target Metrics Lite and Top Mentioned Pages Lite for each target, flattens the items into rows, and replaces today’s partition in two BigQuery tables. API credentials come from environment variables, and BigQuery uses Application Default Credentials. Schedule it to run weekly.
"""Weekly AI visibility export: DataForSEO LLM Mentions API -> BigQuery."""
import datetime as dt
import os
import time
from concurrent.futures import ThreadPoolExecutor
import requests
from google.cloud import bigquery
# While building, swap the host for sandbox.dataforseo.com: free, dummy data, same structure
BASE = "https://api.dataforseo.com/v3/ai_optimization/llm_mentions"
AUTH = (os.environ["DFS_LOGIN"], os.environ["DFS_PASSWORD"])
DATASET = "my-project.ai_visibility"
# One key per tracked brand; each target array takes up to 10 domain/keyword entities
TARGETS = {
"our_brand": [{"domain": "example.com", "search_scope": ["sources"]}],
"competitor_a": [{"domain": "competitor-a.com", "search_scope": ["sources"]}],
"competitor_b": [{"domain": "competitor-b.com", "search_scope": ["sources"]}],
}
# ChatGPT data is US/English only, so this market covers both platforms
MARKET = {"location_code": 2840, "language_code": "en"}
S = bigquery.SchemaField
KEYS = [S("snapshot_date", "DATE"), S("target_key", "STRING"), S("platform", "STRING"),
S("location_code", "INT64"), S("language_code", "STRING")]
METRICS = [S("mentions", "INT64"), S("ai_search_volume", "INT64")]
def call(endpoint, payload, attempts=3):
"""POST one Live task and return its items."""
for attempt in range(attempts):
try:
resp = requests.post(f"{BASE}/{endpoint}/live", auth=AUTH,
json=[payload], timeout=180) # Live calls can take up to 120 s
resp.raise_for_status()
body = resp.json()
task = (body.get("tasks") or [body])[0] # fall back to top-level status if no task came back
if task["status_code"] == 20000:
return (task.get("result") or [{}])[0].get("items") or []
if task["status_code"] < 50000: # 4xxxx = request problem; retrying won't fix it
raise ValueError(f"{endpoint}: {task['status_code']} {task['status_message']}")
except requests.RequestException:
if attempt == attempts - 1:
raise
if attempt < attempts - 1:
time.sleep(5 * 2 ** attempt)
raise RuntimeError(f"{endpoint}: no successful response after {attempts} attempts")
def row(key, item, day, **extra):
return {"snapshot_date": day, "target_key": key, "platform": item["platform"],
"location_code": int(item["location"]), "language_code": item["language"],
"mentions": item["metrics"]["mentions"],
"ai_search_volume": item["metrics"]["ai_search_volume"], **extra}
def snapshot_rows(key, target, day):
# No "platform" in the request = one item per platform (google, chat_gpt)
items = call("target_metrics_lite", {"target": target, **MARKET})
return [row(key, i, day) for i in items]
def top_page_rows(key, target, day):
rows = []
for platform in ("google", "chat_gpt"): # separate calls so one platform can't crowd out the other
items = call("top_mentioned_pages_lite",
{"target": target, "platform": platform, "limit": 50, **MARKET})
rows += [row(key, i, day, page=i["page"]) for i in items]
return rows
def load(client, table, schema, rows, day):
"""Replace one daily partition, so re-running the job never duplicates rows."""
table_id = f"{DATASET}.{table}"
tbl = bigquery.Table(table_id, schema=schema)
tbl.time_partitioning = bigquery.TimePartitioning(field="snapshot_date")
client.create_table(tbl, exists_ok=True)
if rows:
client.load_table_from_json(
rows, f"{table_id}${day.replace('-', '')}",
job_config=bigquery.LoadJobConfig(schema=schema, write_disposition="WRITE_TRUNCATE"),
).result()
def main():
day = dt.date.today().isoformat()
with ThreadPoolExecutor(max_workers=10) as pool: # well under the 30-simultaneous-request cap
snaps = pool.map(lambda kv: snapshot_rows(*kv, day), TARGETS.items())
pages = pool.map(lambda kv: top_page_rows(*kv, day), TARGETS.items())
snapshot = [r for rows in snaps for r in rows]
top_pages = [r for rows in pages for r in rows]
client = bigquery.Client()
load(client, "llm_mentions_snapshot", KEYS + METRICS, snapshot, day)
load(client, "llm_top_pages", KEYS + [S("page", "STRING")] + METRICS, top_pages, day)
if __name__ == "__main__":
main()
Endpoints used: Target Metrics Lite and Top Mentioned Pages Lite. Install dependencies with pip install requests google-cloud-bigquery.
For the one-time trend backfill, the function below reuses the same helpers to call Historical for each target and platform. Historical items carry no platform field, so if you omit platform in the request, you can’t tell Google AI Overview and ChatGPT numbers apart in the response. The function requests each platform separately and records it on the rows itself. Each run replaces the whole table, so the table always mirrors what the endpoint returns.
from export_ai_visibility import DATASET, METRICS, MARKET, S, TARGETS, bigquery, call
def backfill_monthly(client):
rows = []
for key, target in TARGETS.items():
# Historical items carry no platform field, so request each platform separately
for platform in ("google", "chat_gpt"):
for i in call("historical", {"target": target, "platform": platform, **MARKET}):
rows.append({"month": f"{i['year']}-{i['month']:02d}-01", "target_key": key,
"platform": platform, "mentions": i["metrics"]["mentions"],
"ai_search_volume": i["metrics"]["ai_search_volume"]})
schema = [S("month", "DATE"), S("target_key", "STRING"), S("platform", "STRING")] + METRICS
client.load_table_from_json(
rows, f"{DATASET}.llm_mentions_monthly",
job_config=bigquery.LoadJobConfig(schema=schema, write_disposition="WRITE_TRUNCATE"),
).result()
Endpoint used: Historical. Save the first script as export_ai_visibility.py so the import resolves, then run backfill_monthly(bigquery.Client()) once.
Adapting the loader for Snowflake and Redshift
Only the load() function changes. The LLM Mentions API export side, meaning the API calls and the row format, stays the same.
- Snowflake: put the rows into a DataFrame and write it with
write_pandas()fromsnowflake-connector-python[pandas]. For safe re-runs, delete that day’s rows first orMERGEfrom a staging table. - Redshift: write the rows to S3 as newline-delimited JSON and load them with
COPY ... FORMAT AS JSON 'auto', wrapping a delete of that day’s rows and theCOPYin one transaction. AWS SDK for pandas (wr.redshift.copy()) handles both the upload and theCOPY. - Views: the SQL later in this guide uses BigQuery syntax. In Snowflake and Redshift, swap
IF()forCASE WHENandDATE_TRUNC(date, MONTH)forDATE_TRUNC('month', date).
Operations and cost notes
- Timeouts: set your HTTP timeout above 120 seconds, the upper bound of Live execution time.
- Rate limits: up to 30 simultaneous requests and 10,000 calls per minute. The script’s 10 worker threads stay well inside both.
- Cost: based on our current LLM Mentions pricing, each call costs $0.1 per task plus $0.001 per returned row, with no separate export fee. For the three targets in the script, a weekly run comes to about $1.21 or less (three Target Metrics Lite calls returning a couple of rows each, plus six Top Mentioned Pages Lite calls capped at 50 rows), or roughly $5 per month.
- Never re-pull history you already have: backfill monthly history once, then let the weekly job add new snapshots.
How to connect AI visibility data with SEO and traffic metrics
Once AI visibility sits in your warehouse, joining it with other data comes down to shared keys:
- Page URL, normalized (no protocol,
www, query string, or trailing slash) sohttps://www.example.com/pricing/?ref=xmatchesexample.com/pricing. - Domain and keyword for brand- and topic-level joins.
- Location and language codes for market-level joins.
- Date, usually truncated to the month, since snapshots are weekly and history is monthly.
On the SEO side, we’ve got you covered within the same API account, with the same credentials, location and language codes, and loader pattern. For example, the Ranked Keywords endpoint of our DataForSEO Labs API with "item_types": ["ai_overview_reference"] returns the keywords for which a domain is cited in Google AI Overviews, as described in our guide on getting keywords a website ranks for in Google AI Overview. Search volumes come from our Keyword Data API. With one vendor behind both datasets, you skip the glue code that maps one tool’s location IDs to another’s.
On the traffic side, GA4’s native BigQuery Export writes raw events into daily events_YYYYMMDD tables. For Snowflake or Redshift, land GA4 data through your ETL tool and keep the same join logic.
Three unified views tend to earn a spot in reports:
- AI mentions vs. organic sessions per page: are the pages AI cites also the ones bringing search traffic?
- Brand mentions in AI answers vs. branded search volume, by month.
- Pages cited by AI vs. pages that rank and convert: where AI visibility and revenue don’t overlap.
The view below builds the first one in BigQuery. It takes the last snapshot of each month from llm_top_pages, keeps only your own pages, turns platforms into columns, and joins the result with organic sessions from the GA4 export.
CREATE OR REPLACE VIEW `my-project.ai_visibility.v_page_ai_vs_organic` AS
WITH latest AS ( -- last snapshot of each month
SELECT DATE_TRUNC(snapshot_date, MONTH) AS month, MAX(snapshot_date) AS snapshot_date
FROM `my-project.ai_visibility.llm_top_pages`
GROUP BY month
),
ai_pages AS (
SELECT
l.month,
-- strip protocol, www, query string and trailing slash so URLs match GA4's page_location
REGEXP_REPLACE(REGEXP_REPLACE(LOWER(p.page), r'^https?://(www\.)?|[?#].*$', ''), r'/$', '') AS page_key,
SUM(IF(p.platform = 'google', p.mentions, 0)) AS ai_overview_mentions,
SUM(IF(p.platform = 'chat_gpt', p.mentions, 0)) AS chatgpt_mentions
FROM `my-project.ai_visibility.llm_top_pages` AS p
JOIN latest AS l USING (snapshot_date)
WHERE p.target_key = 'our_brand'
AND REGEXP_CONTAINS(p.page, r'^https?://(www\.)?example\.com/') -- your own pages only
GROUP BY l.month, page_key
),
organic AS (
SELECT
DATE_TRUNC(PARSE_DATE('%Y%m%d', event_date), MONTH) AS month,
REGEXP_REPLACE(REGEXP_REPLACE(LOWER(
(SELECT value.string_value FROM UNNEST(event_params) WHERE key = 'page_location')
), r'^https?://(www\.)?|[?#].*$', ''), r'/$', '') AS page_key,
COUNT(DISTINCT CONCAT(user_pseudo_id, '.', CAST(
(SELECT value.int_value FROM UNNEST(event_params) WHERE key = 'ga_session_id') AS STRING
))) AS organic_sessions
FROM `my-project.analytics_123456789.events_*`
-- last 13 months of daily tables; the range also skips events_intraday_* tables
WHERE _TABLE_SUFFIX BETWEEN FORMAT_DATE('%Y%m%d', DATE_SUB(CURRENT_DATE(), INTERVAL 13 MONTH))
AND FORMAT_DATE('%Y%m%d', CURRENT_DATE())
AND event_name = 'page_view'
-- the export has no channel-group column; organic search arrives with medium 'organic'
AND LOWER(IFNULL(session_traffic_source_last_click.manual_campaign.medium, '')) IN ('organic', 'organic search') GROUP BY month, page_key
)
SELECT
month,
page_key,
a.ai_overview_mentions,
a.chatgpt_mentions,
IFNULL(o.organic_sessions, 0) AS organic_sessions
FROM ai_pages AS a
LEFT JOIN organic AS o USING (month, page_key);
Replace my-project, analytics_123456789, our_brand, and example.com with your own values. The BigQuery export has no ready-made channel-group column, so the view derives organic sessions from GA4’s session-scoped last-click traffic source fields — here, the session’s medium. If your property uses custom channel definitions, sanity-check the medium values in your own export. Both datasets must be in the same BigQuery location for the join to run.
Two cautions when reading the result. First, AI metrics describe the state of AI answers at snapshot time, while sessions count visits within a month, so compare direction and rank (which pages gain AI mentions and whether their organic sessions follow) rather than ratios between the two. Second, the script stores the top 50 cited pages per platform; raise limit (up to 1,000) if you need the long tail.
Feeding Looker, Power BI, and Tableau
Once the tables exist, no custom integration is needed. Looker, Power BI, and Tableau all offer native connectors for BigQuery, Snowflake, and Redshift, and Looker Studio connects to BigQuery natively. Point the BI tool at the dataset with a read-only service account and build on the views. A practical starting dashboard includes:
- KPI cards for mentions and AI search volume, plus new vs. lost mentions if you add Timeseries New and Lost to the pipeline.
- Monthly trend lines from
llm_mentions_monthly, split by platform. - A brand comparison chart: your target key against competitor keys.
- A top-cited-pages table with AI mentions next to organic sessions.
Keep calculations in warehouse views rather than in each BI tool. A share-of-voice formula defined once in a view shows the same number in Looker, Power BI, and Tableau; defined three times, it can drift.
Can you push AI visibility data to analytics platforms or tag managers?
Not natively. There’s no built-in push to GA4, Google Tag Manager, or other analytics platforms: the data is pulled through the API on your schedule. Some of our APIs support postback and pingback URLs for tasks set with the Standard method, but the LLM Mentions API works only with the Live method, so there’s no webhook-style delivery for this data either.
But let’s back up for a second. Whichever AI visibility platform the data comes from, analytics tools are the wrong destination for it:
- Mentions are aggregates, not events. A weekly count of 979 mentions isn’t a user action. Sent as a GA4 event, it becomes a hit with no real visit behind it and can skew session and engagement metrics.
- High-cardinality dimensions get bucketed. Page URLs, brands, and platforms add up to many unique values, and GA4 can group the excess into an “(other)” row.
- Analytics interfaces can’t do the joins. Relating AI mentions to rankings and traffic takes SQL.
You could technically send the numbers as custom events through GA4’s Measurement Protocol or a server-side tagging container, but we don’t recommend it. Keeping AI visibility in the warehouse and showing it in the BI layer next to GA4 data gives you the same side-by-side view without polluting session metrics. The same holds for any GEO data export: the warehouse is the hub, and the BI tool is the window.
Quick-start checklist and further reading
- List your targets (your domain and brand terms, plus competitors) and pick the market: US and English if you need ChatGPT data.
- Build and test the script against the Sandbox, then switch the host to
api.dataforseo.com. - Run the Historical backfill once to seed monthly trends.
- Schedule the weekly snapshot job and load into date-partitioned tables.
- Create the views, join them with GA4 and SEO data, and connect your BI tool to the views.
To go deeper on the data itself:
- How to get LLM citation data with LLM Mentions API
- Track Brand Mentions in AI Answers
- 5 LLM Mentions API use cases, from custom reports to competitor analysis
- DataForSEO data for generative engine optimization (GEO)
FAQ
How can I pull AI search visibility data into our own data warehouse?
Through scheduled API calls. There’s no direct BigQuery or Snowflake connector: a script like the one in this guide, running on cron, Airflow, or a cloud function, calls the LLM Mentions API and loads the rows into your tables.
Can I export metrics data for my own analysis from AI visibility platforms?
With the LLM Mentions API, yes. Every metric comes as structured JSON that you can store and model in your own warehouse, spreadsheet, or BI tool.
How often should I export AI visibility data?
Weekly works for most reports. With a 60-day refresh cycle, daily pulls rarely add signal, and the Historical endpoint covers monthly trends.
Does an LLM Mentions API export cost extra?
No. You pay standard per-call pricing, currently $0.1 per task plus $0.001 per returned row, and the AI-optimized response format costs nothing extra.
Can I push data to analytics platforms or tag managers from AI visibility platforms?
Not natively, and we don’t recommend it. Mentions are aggregates rather than user events, so they belong in your warehouse and BI layer next to GA4 data, not inside GA4 itself.
Can I export AI visibility data to CSV or Google Sheets?
Yes. The rows the script builds can be written to a CSV file or a spreadsheet instead of a warehouse table.
Get your AI visibility data next to everything else
AI visibility numbers are most useful next to the rest of your marketing data, where they can be joined, trended, and reported instead of screenshotted from a dashboard. Create an account, run the script against the Sandbox, and switch to live data once the tables look right. If any part of the setup gets stuck, our support manager will do their best to help, so feel free to reach out.