Eight terms this subject is argued in, including three that are the same thing under different
names. Every definition says what the term excludes as well as what it covers.
Agent Experience (AX)
How well a website serves software that reads it on a person’s behalf — whether an agent can reach the page, understand it, extract facts from it, act on it, and know the rules the site has set.
The term is deliberately parallel to user experience and developer experience, and means the same kind of thing for a third audience. A site with good AX is one an assistant can read, summarise correctly, quote accurately and act on without guessing.
AXRAY measures it with 70 checks across five pillars and returns a score out of 100. The rubric is published in full, because a score you cannot argue with is not worth having.
Also called: AEO, Answer Engine Optimization, AI SEO, LLM SEO, AI visibility — all the same subject.
Making a site legible and quotable to systems that answer questions rather than list links. All of the names in that list mean the same subject; none of them is a standard.
The useful part of GEO that is genuinely not SEO comes down to one thing: an assistant reads your page without a browser, so anything assembled in JavaScript never arrives, and the page is then cut into passages of a few hundred tokens of which the model sees one.
The rest of it — clear headings, structured data, server-side rendering, being worth citing — is ordinary technical SEO and content work correctly applied. That is not a criticism, but it does mean you have probably already done most of it and should not pay for it twice.
What nobody can sell is the output side. There is no ranking, no submission form, and no way to make an assistant cite you.
A Markdown file at the root of a domain that tells an AI assistant what the site is and which pages matter — a curated twenty links rather than a sitemap’s four hundred.
Proposed by Jeremy Howard of Answer.AI in September 2024. It is a convention rather than a standard: there is no committee behind it and nothing you can be non-compliant with.
No major assistant has publicly committed to reading it, and anyone claiming that publishing one gets you into ChatGPT is selling something. What is true is narrower: it costs an afternoon, several agent frameworks and documentation tools do fetch it, and writing one forces you to decide in a sentence what your site is for.
Of 292 well-known public sites measured on 2026-09-06, 64 publish one.
A crawler that indexes a site so an assistant can cite and link it in an answer. Distinct from a training crawler, and the distinction is the one most site owners get wrong.
"Block the AI bots" is nearly always meant as "do not train a model on my writing". The same robots.txt line routinely also blocks the crawler that would have put your link in front of a customer, and those are completely different decisions with completely different costs.
AXRAY resolves robots.txt against 54 named crawlers individually rather than reporting one verdict for all of them, because one verdict is what hides this mistake.
A crawler that collects text to train a model on. It sends nobody to your site, now or later, which makes it the only category where blocking has no traffic cost.
GPTBot, ClaudeBot, Google-Extended, CCBot and Bytespider are the commonly named ones. Refusing them is a legitimate position and this site takes no view on whether you should.
What blocking achieves is worth being precise about: robots.txt is honoured by operators who choose to honour it, and does nothing about your text reaching a model through a third-party copy, a screenshot, or somebody pasting it into a chat window. It is a stated position, not a technical guarantee, and nobody should sell it to you as one.
The few hundred tokens of your page that a retrieval system actually hands to a model. Everything the model says about you is written from that fragment alone.
This is why a paragraph beginning "This means you can..." is unusable: whatever *this* was is in a different passage and the model never saw it. A paragraph that names its own subject survives being cut out; one that relies on the heading above it does not.
Across 261 sites cut up the way a retrieval system cuts them, the median page produced 11 passages, and on 23 of them most of those passages were navigation rather than prose.
Assembling the page in the visitor’s browser with JavaScript. Most AI crawlers do not run JavaScript, so what arrives is the shell and none of the content.
The failure is silent in both directions. The page looks perfect to you, and it looks fine to Googlebot, which does render. Nothing logs an error, and the traffic you never received leaves no trace.
The fix is to render the primary content on the server — server components, static generation, an SSR mode, or a prerender step for the routes that matter. It does not have to be the whole application, only the part you want quoted.
A challenge or interstitial page served by bot protection in place of your real page, because an unfamiliar user agent was classified as a threat.
Nobody decides this. A managed firewall or anti-bot product makes the call, the assistant receives the challenge instead of your article, and as far as your CDN is concerned the request was handled correctly.
Of 292 well-known public sites measured on 2026-09-06, 45 returned a bot wall rather than their page and a further 72 returned something gated or partial.