What We Shared With Search Engine Journal About Measuring AI Search

What We Shared With Search Engine Journal About Measuring AI Search

By Stas Levitan · · 6 min read

This week I joined Loren Baker and Search Engine Journal for a webinar about something I think the AI search industry still gets wrong: we are measuring too much probability and not enough performance.

Most GEO dashboards focus on mentions, citations, share of voice and sentiment. I use these metrics too, but mainly for benchmarking. They tell me where a brand may stand relative to competitors. They do not necessarily tell me what happened on the website or where I should spend the next dollar.

That was the main idea behind our Search Engine Journal session on AI Search & SEO KPIs.

We need signals closer to what actually happened

There is no Search Console for ChatGPT, Gemini, Claude or Perplexity. We cannot see every time a brand was considered, every source used in an answer, or every recommendation.

But we are not completely blind.

At LightSite we sit directly on websites, so we can observe four things much closer to the source:

  • AI bot activity — which AI systems actually accessed the site.
  • Content consumed — which pages and resources they used, revisited or ignored.
  • Human AI referrals — which people actually arrived from AI assistants.
  • The relationship between machine attention and human demand.

Search Engine Journal summarized the same idea ahead of the webinar: AI crawler activity, pages consumed by bots and AI referral traffic can help marketers understand whether machine attention is turning into real visitors.

That last relationship is particularly important to me.

From AI crawls to human demand

Earlier this year we started using AI CTR inside LightSite as a core performance metric. The idea is simple: compare verified AI bot activity with human visits arriving from AI assistants.

AI CTR = AI-referred human visits ÷ verified AI bot activity

We were not the first people on the internet to compare crawls and referrals. Similar crawl-to-referral approaches have appeared elsewhere. What is different about LightSite is that this relationship is built directly into a much deeper measurement layer.

We do not only see that an AI bot requested a URL. Because LightSite deploys its own machine-readable infrastructure alongside the website, we can also measure which structured resources bots use, which pages they prefer or ignore, how much they extract, which Skills agents invoke, and how those journeys change after we modify the site.

Then we compare that machine behavior with human AI traffic page by page.

That is what makes the metric useful to us. It is not another score. It is one part of a measurement loop. The full definition and formula live in our original guide: AI CTR: How To Measure AI Search Performance.

Why page-level measurement changes the decision

During the SEJ webinar I shared a pattern we keep seeing across our data.

A page can receive a lot of machine attention and almost no human traffic. Another page can receive relatively little crawler activity but produce high-intent visits from AI.

Those two pages should not receive the same marketing investment.

We saw this very clearly with TitanDXP. Their assumption was that they needed more editorial content. But when we looked at what AI was actually consuming and where humans from AI were landing, support and documentation were doing most of the work.

The team invested where the demand already existed. The result was a 48% increase in AI visitors and a 228% increase in on-site conversions.

For me, this is the reason AI search measurement matters. Not because we need more dashboards, but because better signals change what we do next.

This is part of a bigger shift

I think AI search will gradually move from simulated measurement toward first-party performance signals.

Prompt tracking will remain useful. Mentions and share of voice will remain useful. But marketers should increasingly combine them with things that actually happened:

  • Did AI access the site?
  • What did it consume?
  • Did it come back?
  • Did a human arrive afterward?
  • Which pages turned machine attention into demand?

This is also why LightSite is architected differently from most AI visibility platforms. We are not only watching AI answers from the outside. We operate an infrastructure and measurement layer directly on the website, which gives us much deeper visibility into how AI systems interact with it. If you are comparing vendors on that basis, we published the full breakdown in Best Technical GEO Platforms for AI Search in 2026.

That data lets us close the loop: measure → identify the gap → change the website → measure again.

That was the main message I wanted people to take away from the Search Engine Journal webinar.

Where to go next