Research

How I collect and check my data

This page describes the practices I used on the studies in the hub. Steps not written here are not part of the method.

Last checked .

How do I pull Search Console data?

I pull it through the Search Console API, one property at a time, and I save the raw result before I analyse anything.

The dimension depends on the question. The query studies use query. The update tracker uses date. The benchmarks use page by date. All of it is web search. The API holds about 16 months, caps the rows it returns each day and revises recent days. I stop at the last complete day.

The benchmarks study lists these limits in full.

What is the difference between whole-property and page-level impressions?

Whole-property impressions count a site once when it appears on a results page. Page-level impressions count it once for each page that appears, so the page rows add up to more.

This caused a restatement on 2 November 2026. My first cut of the May 2026 core update summed page-level impressions. For one large site, the page rows gave 100,189 impressions for 9 to 22 June 2026. Its whole-property total was 33,513. That site helped produce a pooled change of +0.3%. On whole-property data the pooled change is -18.6%.

Two things changed in that restatement. I also moved from 28-day to 14-day windows, and the panel went from 46 sites to 44. The old 28-day windows on whole-property data, with 47 sites, give a median of +28.1% and a pooled change of -15.6%. The update tracker now uses whole-property totals for every row. The full before-and-after is under May 2026 core update, restated.

How do I match AI referrals in GA4?

I match the session source text against five assistant names. One script does it, on one stored raw pull per property, and it ignores how GA4 labels the channel.

Source text matched for each AI assistant
AssistantSource contains, any case
ChatGPTchatgpt or openai
Microsoft Copilotcopilot (the source string in my data is copilot.com)
Perplexityperplexity
Claudeclaude
Geminigemini
  • Of 6,165 matched sessions in edition 2, 5,770 already carried GA4's own AI Assistant label. The other 395 only matched my rule.
  • Organic sessions are rows where the default channel group is exactly Organic Search.
  • Dates use each property's own GA4 reporting time zone.
  • A plain bing source counts as Bing organic, so a Copilot click that arrives as bing is not visible.
  • Google AI Overviews and AI Mode cannot be separated from Google organic sessions. That traffic is hidden, not zero.
  • Unclicked citations never reach GA4. The method counts sessions, not citations or people.

A public Python script recomputes the headline table from the anonymised JSON. It is in the data catalogue.

What is the sample?

The benchmarks, query length and update tracker studies use sites from one panel of 100 that I own. No client site is in those three.

I built that sample from 183 Search Console property rows, and I recorded a reason for every row. I excluded:

  • client sites, leads and family sites
  • properties where I could not confirm ownership
  • duplicate properties
  • two accounts I do not publish from
  • sunnypatel.co.uk, in the new-page timing study, because its migration distorts that measure

The GA4 study uses its own frame. It covers 108 properties in accounts where I am the sole or majority owner. Before publication an ownership review removed 7 more properties, 2 clients and 5 I could not confirm. I made that cut on ownership only, and not on any property's numbers. Each study page states its own frame.

The sites are mostly UK affiliate and programmatic sites. A few large ones dominate pooled totals, so I show site medians next to pooled figures. Dead, deindexed and sold sites are missing, so the sample leans towards survivors. It describes my sites and does not describe the web.

How do I check my own numbers?

I write a second script, separately from the first, and compare every headline number with the first script's output.

  • The benchmarks used separate pulls, and all 5 headline numbers matched exactly.
  • The query length study used an independent rewrite, and every headline number matched.
  • The update tracker uses two scripts. All nine rows match on n, counts, median, pooled change and both sums.

One check caught a rounding error. A pooled zero-click share first read 89.2% and was 89.1% after a double-rounding fix (104,089 of 116,760 pages).

Both scripts use the same API. The check covers my arithmetic and does not cover Google's own sampling. Some studies have no second script. The data catalogue says which.

What counts as too small to report?

Each study sets its own floor, and none of them reports a figure that sits below it.

  • In the update tracker, a group with fewer than 30 eligible sites gets counts only, with no median and no pooled change.
  • A sector in the AI referral study needs at least 3 active properties. A sector with 2 is left out.
  • A fee bucket in the pricing study needs 8 providers before it gets a median.
  • Query length medians per site need 100 or more query rows, or 30 or more clicks for click share.
  • Spreads are shown as the 25th to 75th percentile, and every figure keeps its denominator.

You can see the tracker rule at work in the update tracker, where the early rows show counts only.

How do I handle anonymised queries?

Google withholds rare and anonymised queries, so my query studies cover visible queries only.

The hidden queries skew long, so every query length figure describes the visible ones. The same gap helped end the CTR study. Query-level rows covered a median of about 19% of each property's clicks, which is too little for a defensible curve.

None of my public data files lists search queries. They hold pooled counts and shares.

How do corrections work?

I correct the page itself, with a dated note, and I leave the old figure visible next to the new one.

Each correction sits on the page it changes. These are the ones so far.

Corrections made to research pages
DatePageWhat changed
Google update trackerI restated the May 2026 core update row after switching from page-level to whole-property impressions.
AI traffic across 20 sitesI corrected a line that said one site's AI referrals held steady after its Google decline. Later data showed a fall.
Google CTR by positionI withdrew the portfolio CTR figures and kept them online, marked, with the reasons.

The wider rules are in the editorial policy.

What do these studies not show?

They show what happened on my own sites. They do not show why.

  • Before-and-after windows do not isolate a cause, and I do not remove seasonality.
  • Young properties that are still growing can push the up counts higher.
  • The convenience sample has survivorship bias and a narrow niche mix.
  • No study here is a forecast for another site.

Want this kind of check on your own site? The free 20-minute diagnosis is at /contact/.