robots.txt can only say whether a crawler may fetch you. It has never been able to say what may be done with the bytes afterwards. The IETF vocabulary is the first machine-readable way to draw that line, and stating your position now costs one header and settles a question you would otherwise argue about later.
Agent Contract — Have you told agents the rules: what they may use, where the docs are, who to contact?
Weight within the pillar
5 points
Share of the overall score
about 1.3 of 100
Kind of change
Configuration or content — no engineering needed
Two numbers rather than one, because a check's weight is inside its pillar and the pillar is a share
of the whole. The largest check in the smallest pillar moves your score less than a middling one in
the largest, and knowing which you are looking at is the difference between fixing the right thing
and fixing the loudest thing.
Why an agent cares
robots.txt can only say whether a crawler may fetch you. It has never been able to say what may be done with the bytes afterwards. The IETF vocabulary is the first machine-readable way to draw that line, and stating your position now costs one header and settles a question you would otherwise argue about later.
Worth saying plainly: this is not a search-engine ranking factor and passing it will not move you up
a results page. It measures something else — whether software with no browser, no JavaScript
and a few hundred tokens of budget can tell what your page is and do something with it. Those are the
conditions an AI assistant reads you under, and they are much harsher than a browser's.
How to fix it
Send a Content-Usage header, or add the same directive to robots.txt, saying whether you permit training and search use.
Draft standard: draft-ietf-aipref-vocab and draft-ietf-aipref-attach. It expresses a preference rather than enforcing one, and it sits alongside your existing robots.txt rules rather than replacing them.
# As an HTTP response header on your pages:
Content-Usage: train-ai=n
# Or, for the whole site, in /robots.txt:
User-agent: *
Content-Usage: train-ai=n, search=y
Allow: /
That remediation is the one the scanner itself offers when this check fails, read from the same code
that does the scoring rather than written alongside it. If it were typed out separately it could
drift from the check that judges it, which is the failure that makes published advice worthless.
How often the web fails this
Measured on 2026-09-06 against AX specification v1.5.1, across a
sample of 292 public websites.
Result
Sites
Share of those assessed
Failed
0
0%
Warned
292
100%
Passed
0
0%
Did not apply
0
excluded from the shares
The last row is kept out of the percentages deliberately. A check that did not apply to
0 of 292 sites is not a check most of the web passes, and folding those in is the
commonest way this sort of table is used to say something untrue.
At 0% this is something most sites already get right, which makes failing it more costly than the point value suggests — you are behind the field rather than level with it.
Testing it on your own site
npx axray-cli your-site.com
All 70 checks against one URL, this one among them, with what was found rather than only whether it
passed. Add --min 70 in a pipeline and the command exits non-zero below that score, so a
change that breaks what agents can read fails the build instead of shipping.
Questions about this check
What does the "Declares AI usage preferences the standard way" check mean?
robots.txt can only say whether a crawler may fetch you. It has never been able to say what may be done with the bytes afterwards. The IETF vocabulary is the first machine-readable way to draw that line, and stating your position now costs one header and settles a question you would otherwise argue about later. It sits in the Agent Contract pillar of the Agent Experience rubric and is worth 5 points within that pillar, which works out at about 1.3 points of the overall score out of 100.
How do I fix content-usage?
Send a Content-Usage header, or add the same directive to robots.txt, saying whether you permit training and search use. Draft standard: draft-ietf-aipref-vocab and draft-ietf-aipref-attach. It expresses a preference rather than enforcing one, and it sits alongside your existing robots.txt rules rather than replacing them. It is a configuration or content change rather than an engineering one, so it can usually ship the same day.
Is it unusual to fail content-usage?
No. Of 292 public sites where this check applied, 0 failed it — 0%. The measurement was taken on 2026-09-06 against AX specification v1.5.1, and no site is named.
How do I test content-usage on my own site?
Run npx axray-cli your-site.com from a terminal. It runs all 70 checks against one URL and prints this one among them, with what it found rather than only whether it passed. It is free, needs no account and installs nothing.
The rest of the Agent Contract pillar
Have you told agents the rules: what they may use, where the docs are, who to contact?