Ivan BuildsResearch
Original research · ChatGPT · AI Overviews

After GPT-6, 50% of ChatGPT's Answers About Hiring an SEO Agency Warn That a Ranking or Claim May Not Be Independent, Up From 13% (Same 300 Searches, Before and After)

TL;DRIf your agency publishes a "best agencies" list with itself on it, my bet is that the list still helps on ChatGPT but now arrives with a warning: in October ChatGPT recommended the agency in 27% of the runs where it cited the agency's own list, against 9% when it did not, and 43% of its answers warned that a ranking may not be independent.

I asked ChatGPT the same 300 SEO agency-hiring questions five times before GPT-6 became its default and five times after, then compared what it warned about, what it cited and which agencies it recommended.

IS
By Ivan Smiljkovic
Published 15 October 2026 · 1,500 answers from ChatGPT in September, on 13 September 2026, and 1,500 on 14 October 2026 · ChatGPT, with Google AI Overviews as a check · 10 min read
Start with the short versionFindingsMethodCite
50%
of ChatGPT's answers in October warned that a ranking or an agency's own claims may not be independent, against 13% for ChatGPT in September
27%
of the runs in which ChatGPT cited an agency's own list in October also recommended that agency, against 9% when the list was not cited
16%
of ChatGPT's citation links in October went to agency-written lists, against 29% for ChatGPT in September
8%
of the question and agency pairs ChatGPT recommended in October came back in all five runs

How I checked itA reader that never saw my extractor's output listed the agencies in 180 answers from both months: 98% of the agencies my rules pulled out were ones the answer offered, and the rules caught 93% of them. A second reader labelled 150 warning sentences blind and agreed on 99% of them. Every figure was computed in Python and again in SQL.

On this page
  1. The short version
  2. Earlier studies
  3. How I did it
  4. 0150% of ChatGPT's answers in October warned about a ranking or an agency's own claims, against 13% for ChatGPT in September
  5. 02When ChatGPT cited an agency's own list, it recommended that agency in 27% of the runs in October, against 9% when it did not cite the list
  6. 03Agency-written lists fell from 29% of ChatGPT's citation links to 16%, and agencies' own sites rose from 42% to 60%
  7. 04Victorious fell from 24% of ChatGPT's agency recommendations to 10%, and SmartSites rose from 17% to 25%
  8. 058% of the question and agency pairs ChatGPT recommended in October came back in all five runs
  9. What I would test
  10. Limits
  11. Data
  12. Cite this study
  13. About the author

Since GPT-6 became ChatGPT's default on 7 October 2026, David Konitzny at Peec AI has measured it warning about the lists it cites: warnings that a ranking had not been independently verified rose from 0.56% to 3.46% of nearly 3 million chats that cite a listicle or comparison page, and warnings about vendors' own claims from 1.01% to 2.27%. I wanted to see what that looks like in one market where the lists are written by the people being ranked. So I took the 300 SEO agency-hiring questions from my September study, asked ChatGPT them again after the switch, five times each, and compared the two months answer by answer.

The short version

50% of ChatGPT's answers in October warned that a ranking or an agency's own claims may not be independent, against 13% for ChatGPT in September, yet an agency whose own list ChatGPT cited was still recommended about three times as often as when its list was not cited, in both months.

Two snapshots show what changed in ChatGPT's answers, not why, so the points below are bets to test, not rules. Under each one are the numbers behind it and the finding they come from. The full version, with the strongest objection to each bet, is in What I would test.

  1. BetA self-ranking list still helps, but expect ChatGPT to say who wrote it.Evidence in Finding 1 and Finding 2:
    • 43% of ChatGPT's answers in October warned that a ranking may not be independent, against 13% for ChatGPT in September.
    • When ChatGPT cited an agency's own list, it recommended that agency in 27% of the runs in October and 28% for ChatGPT in September; when the list was not cited, 9% and 10%.
  2. BetPut your own claims where someone else can confirm them.Evidence in Finding 1:
    • 15% of ChatGPT's answers in October warned that results or case studies came from the agency itself, against 1% for ChatGPT in September.
  3. BetFor city searches, your own site now matters more than the directories.Evidence in Finding 3:
    • For city searches, directories took 33% of ChatGPT's citation links in October, against 83% for ChatGPT in September; agencies' own sites took 53%, against 14%.
  4. BetCheck your own share of ChatGPT's recommendations after a model switch; it can move a lot.Evidence in Finding 4:
    • Victorious was recommended in 10% of the answers that recommended agencies in October, against 24% in September; SmartSites rose from 17% to 25%.

What earlier studies found

  • David Konitzny at Peec AI measured the warnings across ChatGPT's traffic: warnings that a ranking had not been independently verified rose from 0.56% to 3.46% of nearly 3 million chats that cite a listicle or comparison page, and warnings about vendors' own claims from 1.01% to 2.27% (LinkedIn). A week earlier he found that listicles rose from 6.45% to 9.70% of the URLs ChatGPT cited after GPT-6.
  • Glen Allsopp at Ahrefs found that "best X" blog lists were 43.8% of the page types ChatGPT cited, across 750 prompts (Ahrefs).
  • Lily Ray found on Google's AI Overviews that when a business software brand's own "best" list was cited, the brand was left out of the recommendation 69% of the time (224 of 323 cited lists) (Lily Ray's newsletter).
  • Mateusz Makosiewicz at Ahrefs published self-promotional pages as an experiment and found that when AI cited a page promoting Ahrefs' own conference, it still skipped the conference 43% of the time (Ahrefs).
  • Louise Linehan at Ahrefs compared AI citations with Google's results for the same prompts and found that on average only 12% of the links ChatGPT, Gemini and Copilot cited were in Google's top 10 for the same prompt, across 15,000 prompts (Ahrefs).
  • Minuttia and Convex Works looked at the lists AI assistants cited for software questions and found that 34% to 47% of the cited lists were self-promotional, meaning a software vendor's list of its own category with its own product on it (Minuttia and Convex Works).

What's new here: the same 300 SEO agency-hiring questions asked before and after GPT-6, so each change is measured on the same questions; how often the answers in one market warn about rankings and claims; whether an agency whose own list is cited is still recommended; which agencies gained and lost recommendations; and Google's AI Overviews on the same questions as a check.

How I did it

  • The questions The 300 questions of my September study: best-agency searches, who-should-I-hire questions and service searches, across industries, specialties, buyer needs and US cities.
  • ChatGPT Each question five times on 13 September 2026 and again on 14 October 2026, about an hour apart, through the same collection path. In the raw responses, every one of October's 1,500 ChatGPT responses reports the model gpt-6; of September's, 34 report gpt-5-6 and the rest name no model.
  • Google AI Overviews The same questions as a check, five runs in September and three in October.
  • Warnings Every sentence that a broad word pattern flagged was labelled by a reader as a warning about a ranking, about an agency's own claims, or neither, with the month hidden. Every ChatGPT answer the pattern flagged nothing in was read in full, and the sentences found there were labelled the same way.
  • Recommendations A rule-based extractor pulled out every agency an answer offered as an option. GPT-6 lays its answers out differently, with picture cards and escaped numbering, so I taught the extractor those layouts; all 21,444 of September's recommendation rows came out exactly as before.
  • Sources Every cited page was labelled by who published it. Every site that published a list counted as agency-written was checked by a reader, in both months.

Finding 0150% of ChatGPT's answers in October warned about a ranking or an agency's own claims, against 13% for ChatGPT in September

  1. ChatGPT, Septembern = 1,50013%
  2. ChatGPT, Octobern = 1,50050%
  3. Google AI Overviews, Septembern = 1,2060%
  4. Google AI Overviews, Octobern = 7830%
Share of answers that warned that a ranking or an agency's own claims may not be independent.
  • Warnings about rankings rose from 13% of the answers from ChatGPT in September to 43% in October: sentences such as "several of these lists are published by agencies that rank themselves first".
  • Warnings about an agency's own claims rose from 1% for ChatGPT in September to 15% in October: "these results are self-reported by the agency".
  • Google AI Overviews in September warned in 1 of 1,206 answers, and in October in 0 of 783.
  • Industry searches drew the most warnings: 80% of ChatGPT's answers to them in October, against 19% for ChatGPT in September. City searches drew the fewest: 20% in October, against 11% for ChatGPT in September.
  1. Industry searches, Septembern = 45019%
  2. Industry searches, Octobern = 45080%
  3. Specialty searches, Septembern = 3008%
  4. Specialty searches, Octobern = 30054%
  5. Buyer-need searches, Septembern = 30012%
  6. Buyer-need searches, Octobern = 30049%
  7. City searches, Septembern = 45011%
  8. City searches, Octobern = 45020%
Share of ChatGPT's answers that warned, by kind of search.

Finding 02When ChatGPT cited an agency's own list, it recommended that agency in 27% of the runs in October, against 9% when it did not cite the list

  1. September, own list citedn = 1,38828%
  2. September, own list not citedn = 2,29110%
  3. October, own list citedn = 1,27127%
  4. October, own list not citedn = 2,2349%
How often an agency that publishes a list was recommended, by whether ChatGPT cited its list in that run.
  • The gap was much the same for ChatGPT in September: 28% of the runs when the list was cited, against 10%.
  • Counted per citation, ChatGPT recommended the author of a cited agency-written list 32% of the time in October, against 36% for ChatGPT in September.
  • 85% of the agency-written lists ChatGPT cited in October put their own agency on the list, against 87% for ChatGPT in September.

Finding 03Agency-written lists fell from 29% of ChatGPT's citation links to 16%, and agencies' own sites rose from 42% to 60%

  1. Agency-written lists, Septembern = 6,44029%
  2. Agency-written lists, Octobern = 10,52916%
  3. The recommended agency's own site, Septembern = 6,44042%
  4. The recommended agency's own site, Octobern = 10,52960%
  5. Directories, Septembern = 6,44024%
  6. Directories, Octobern = 10,52916%
Who published the pages ChatGPT cited when it recommended agencies.
  • ChatGPT cited more links in October: 10,529 citation links in the answers that recommended agencies, against 6,440 for ChatGPT in September.
  • For city searches, directories took 33% of ChatGPT's citation links in October, against 83% for ChatGPT in September, and agencies' own sites 53%, against 14%.
  • Clutch was still cited in 59% of the answers that recommended agencies, against 58% for ChatGPT in September.
  • Google's AI Overviews moved much less: agency-written lists were 35% of the citation links for Google AI Overviews in September and 30% in October.
  1. City searches, Septembern = 89683%
  2. City searches, Octobern = 2,42733%
  3. Agencies' sites for city searches, Septembern = 89614%
  4. Agencies' sites for city searches, Octobern = 2,42753%
Who ChatGPT cited for city searches.

Finding 04Victorious fell from 24% of ChatGPT's agency recommendations to 10%, and SmartSites rose from 17% to 25%

The agencies whose share of ChatGPT's answers that recommended agencies moved most, among those recommended in at least twenty answers in one of the months:

Agency September October
Victorious 24% 10%
WebFX 26% 19%
Siege Media 15% 10%
NP Digital 5% 1%
SmartSites 17% 25%
Funnel Boost Media 6% 11%
Coalition Technologies 4% 7%
  • The agencies ChatGPT recommended most often in October were SmartSites, Searchbloom, WebFX; for ChatGPT in September, WebFX, Victorious, Searchbloom.
  • On Google's AI Overviews the biggest fall was Searchbloom, from 21% to 12%.

Finding 058% of the question and agency pairs ChatGPT recommended in October came back in all five runs

  1. In one run only54%
  2. In two runs18%
  3. In three runs11%
  4. In four runs8%
  5. In all five runs8%
In how many of the runs each question and agency pair ChatGPT recommended in October came back. Base: 3,865 pairs.
  • 54% of ChatGPT's October pairs came back in only one run. A single check of one question shows a different shortlist from the next.

What I would test

Two snapshots show what changed in ChatGPT's answers between two days, not what caused it, so each point below is something to test rather than a rule.

If you publish a "best agencies" list with yourself on it, test how ChatGPT describes it. In October ChatGPT recommended an agency in 27% of the runs where it cited the agency's own list, against 9% when it did not, and 43% of its answers warned that a ranking may not be independent. The test is to ask ChatGPT the searches your list targets and read what it says about your list.

Test whether results confirmed by someone else get cited without a warning. 15% of ChatGPT's answers in October warned that results or case studies came from the agency itself. The test is a client's own write-up or a third-party review of a result, then the same searches again.

For city searches, test your own pages. Directories took 33% of ChatGPT's citation links for city searches in October, against 83% for ChatGPT in September. The test is a page that says plainly which city you serve and what you do there, checked in the city searches over the following weeks.

What this study does not show

Two days are two snapshots. September's answers came from an earlier model than October's, and the change between them may include more than the model: the questions were the same, but the pages and listings ChatGPT found were a month newer.

The warning count depends on what counts as a warning. A sentence counted when it said or plainly implied that a ranking or claim may be biased, paid, self-published or not independent; softer caveats, such as "this is not a definitive ranking", did not count.

Citations and recommendations are not clicks or clients. This measures what ChatGPT said, not what anyone did after reading it.

Google's AI Overviews had five runs in September and three in October, so Google's figures are a rougher check than ChatGPT's.

Data

The questions, every raw answer, the labels and the code that computes each figure are kept with the study; every figure on this page has a numerator and a denominator there.

Study by Ivan Smiljkovic. Data collected on 13 September 2026 and 14 October 2026. Every figure has a numerator and a denominator in the study dataset; every raw answer is preserved.

Cite this study

Ivan Smiljkovic (2026). After GPT-6, 50% of ChatGPT's Answers About Hiring an SEO Agency Warn That a Ranking or Claim May Not Be Independent, Up From 13% (Same 300 Searches, Before and After). Ivan Builds. https://ivanbuilds.com/chatgpt-gpt6-agency-recommendations.html

Every section has its own link: hover a heading and use the # to copy it.

About the author

IS
Ivan Smiljkovic
Independent research on search and AI answers

I study how Google and AI assistants choose what to cite, using public data I collect myself, and I publish the method and limits with every study.

More research

More research

Original research
83% of Google AI Overviews for How-To Questions Cite a YouTube Video (Study of 1,777 AI Overviews)
I searched Google with 600 real how-to questions, three times each, and traced every YouTube video its AI Overviews cited, down to the second they linked to. Then I compared the cited moment with the rest of the same video, the cited videos with the videos YouTube itself ranks for the same question, and the line at the cited second with the lines around it.
Original research
66% of the Pages ChatGPT Cites When Recommending SEO Agencies Are Written by SEO Agencies (Study of 3,000 AI Searches)
I asked ChatGPT and Google the same 300 SEO agency-hiring questions five times, then traced every page they cited and every agency they named. Most of the evidence is written by SEO agencies.
Original research
82% of the Pages ChatGPT Cites When Recommending a Local Business Are Directories and Review Sites (Study of 2,112 AI Searches)
I asked ChatGPT and Google for local roofers, plumbers, HVAC contractors, pest control companies, electricians, dentists, auto repair shops and landscapers in 22 US cities, three runs an hour apart, and traced every page they cited and every business they recommended. On ChatGPT, the cited pages are mostly directories and review sites, and the businesses' own websites barely appear. Google AI Overviews lean the other way.
Original research
64% of the Passages Google's AI Overviews Quote Start at a Heading or Right Below One, Against 31% of Other Passages on the Same Page (Study of 3,224 Quotes)
I searched Google with 900 questions, read the web pages its AI Overviews cited, and found where on each page the source card's words sit. Then I compared each quoted passage with up to twelve other passages of the same length on the same page.
Original research
Google Didn't Show 18% of the Pages ChatGPT Cites, Against 6% of the Pages Its Own AI Overviews Cite (Study of 2,232 Pages)
I asked ChatGPT and Google the same 300 questions three times, then searched Google for every page ChatGPT cited, up to four times each, with the pages Google's own AI Overviews cited as the control.
Original research
How Many AI Overview Links Stay With Google? From 3% to 86%, Depending on the Search (Study of 3,288 AI Overviews)
I ran the same 1,252 Google searches in September and October and sorted every link in their AI Overviews by where it goes: a website, YouTube, or one of Google's own pages. How much stays with Google depends on the kind of search.
Original research
58% of the YouTube Citations in Google's AI Overviews Go to Independent Creators; in Legal Searches, 68% Go to Businesses' Own Channels (Study of 2,929 YouTube Citations)
I ran 588 real US searches in 20 industries through Google three times and sorted every YouTube video its AI Overviews cited by who is behind the channel: a brand, an independent creator, a media company or an institution.