One plain HTTP request, then 70 questions about what came back.

1. We fetch it the way an agent does

AXRAY sends a single GET with an honest user-agent, a twelve-second timeout and a hard byte cap, following redirects manually so the chain is reportable. Then it stops. If your content needs a browser to appear, it does not appear here — which is exactly what happens inside a crawler.

In parallel we fetch the origin-level artefacts an agent looks for before it trusts a site: /robots.txt, /sitemap.xml, /llms.txt, /.well-known/security.txt and /.well-known/mcp.json.

2. We parse it without a DOM

The HTML is parsed by a small, forgiving tokenizer written for this purpose. No browser engine, no third-party parser, no JavaScript. Real pages are malformed and a scanner that throws on bad markup is a scanner nobody trusts, so it is tolerant by design.

3. We ask 70 questions

Each check is a pure function of the same frozen set of facts, which is why the same URL scores the same in the CLI, in CI and on this site. Every check returns a verdict, the evidence behind it, a sentence explaining why an agent cares, and where possible a snippet you can paste.

4. We weight, gate and rank

Checks that do not apply to your page leave the denominator rather than costing you points: a page with no forms is never punished for having no form labels. Pillar scores are combined by weight into the overall number.

Then the gates apply. Some failures are severe enough that a good average would be a lie — a bot wall, a total robots block, a page that ships no content. In those cases the score is capped and the report says so in a sentence, because an unexplained ceiling reads as a bug.

Finally every finding is converted into the points it is costing you on the overall scale, and the fix list is sorted by that number. It is a work queue, not a list of complaints.

5. We show you what the agent came away with

Every report carries the page rebuilt as Markdown from the same parse the score came from: the headings, the text, the links and the image alt text that survived a fetch with no browser in it. Nothing is added and nothing is inferred. On a page that renders in the visitor’s browser it is usually three lines long, and that single fact explains the score faster than any list of failed checks.

Above it are two numbers: roughly how many tokens the whole page costs an agent to read, and how many of those are the page’s own content rather than the navigation, header and footer wrapped around it. The second number is the useful one. A context window is a fixed budget, and a site that spends two thirds of every page on a mega-menu is paying that on every page an assistant ever reads.

6. We cut it up the way a retrieval system does

No assistant reads a whole page. A retrieval system splits it into passages of a few hundred tokens, stores each separately, and hands back one of them next to passages from other sites — and the answer is written from that passage alone. So we run the same cut: split at the headings, carry the heading path into each passage the way a real chunker carries it, cap each at about three hundred tokens, and then ask of every passage whether somebody who saw only it would know what it was about.

Two things come out of that. The share of your content that stands on its own, and the share of what would be retrieved that is navigation rather than content — menus and link lists go into the index too, get retrieved too, and answer nothing. No language model is involved: every verdict comes from the structure of the page, which is what makes it reproducible and what makes it something you can argue with.

Alongside it we read your structured data against your visible text and report where the two disagree. A price in the JSON-LD that is not one of the prices on the page. A product marked in stock on a page that says sold out. None of that shows in a browser, none of it breaks a test, and all of it gets quoted with confidence, because an assistant has no reason to doubt the half of the page it can simply believe.

How the token estimate is made, and how wrong it can be

It is an estimate and it is labelled as one everywhere it appears. AXRAY ships no dependencies and no data files, and the vocabularies behind cl100k_base and o200k_base are megabytes each, so there is no tokenizer in here to run. Instead: Latin script, digits and punctuation merge into pieces averaging about four characters and are charged a quarter of a token each; Han, Kana and Hangul do not merge that way and are charged close to a token per character, which is the difference between being roughly right and being wrong by a factor of three on a Japanese page; line breaks are charged because they are real tokens.

On English prose that lands within about ten per cent of a real tokenizer. It is accurate enough to size a page against a context window and to compare the content half of a page with the chrome half. It is not accurate enough to bill anyone with, and nothing here does.

Questions people actually ask

Is this just SEO with a different name?

No, and the difference is the failure mode. A search engine with a bad title shows a bad title. An assistant with a bad title invents one, attributes it to you, and shows that to somebody who will never see your site. Roughly a third of these checks have no SEO equivalent at all: whether a named AI crawler is allowed through, whether your prices are machine-readable, whether an agent can operate your forms, whether you have published anything an agent can call.

Why not render the page with a browser first?

Because that would flatter you. Most crawling agents fetch plain HTML with no JavaScript and a hard timeout, and reproducing that budget is the only honest way to report what they see. If your content needs a browser to appear, it does not appear here — which is exactly what happens inside the crawler.

What does a scan cost me, as the site being scanned?

A handful of requests: the page, your robots.txt, sitemap, llms.txt and a few well-known paths. One host is fetched at most 30 times an hour across every AXRAY user combined, repeat requests inside 10 minutes are served from a stored result, and crawls obey your robots.txt and your crawl delay. If you would rather we did not fetch you at all, two lines in robots.txt stops us.

Does scanning a site publish anything about it?

No. A report is reachable only through its own unguessable link, and nothing enters the public AX Index unless the domain's owner has proved they control it and asked to be listed.

Can I not just add the files it asks for and score well?

Not any more, and we publish how that is stopped. Twenty checks that an empty artefact used to satisfy now look at what is in it: an llms.txt has to say something and link somewhere, a Product has to carry offers, a security.txt has to name a contact. Where something is present but hollow the report labels it thin evidence and it earns at most half its weight.

What we deliberately do not do

• We do not render. A rendered score would flatter you and mislead you.
• We do not hammer your origin. A default scan is a handful of requests; crawls run two at a time with a delay.
• We do not guess at intent. If a check cannot be evaluated it is marked not applicable, not failed.
• We do not hide the rubric. The full specification is public and generated from the scoring code.

Does AXRAY pass its own test?

It has to. This site is server-rendered, semantically landmarked, described in JSON-LD, and publishes llms.txt, robots.txt, sitemap.xml and an agent manifest. Run npx axray-cli axray.online and check.