AXRAY
Sign inCreate account

Reachability · cache-validators

Re-fetching is cheap for a crawler

Crawlers revisit. With a validator they send one conditional request and receive an empty 304; without one they download the whole page every time, which is the traffic sites then complain about.

The full AX specification, all 70 checks

What it is worth

Check idcache-validators
PillarReachability — Can an agent fetch your content at all, without a browser and without being blocked?
Weight within the pillar4 points
Share of the overall scoreabout 0.8 of 100
Kind of changeConfiguration or content — no engineering needed

Two numbers rather than one, because a check's weight is inside its pillar and the pillar is a share of the whole. The largest check in the smallest pillar moves your score less than a middling one in the largest, and knowing which you are looking at is the difference between fixing the right thing and fixing the loudest thing.

Why an agent cares

Crawlers revisit. With a validator they send one conditional request and receive an empty 304; without one they download the whole page every time, which is the traffic sites then complain about.

Worth saying plainly: this is not a search-engine ranking factor and passing it will not move you up a results page. It measures something else — whether software with no browser, no JavaScript and a few hundred tokens of budget can tell what your page is and do something with it. Those are the conditions an AI assistant reads you under, and they are much harsher than a browser's.

How to fix it

Send an ETag or a Last-Modified header on HTML responses and answer If-None-Match with a 304.

Most static hosts and CDNs do this already; it is usually an origin that generates HTML per request that does not.

ETag: "a1b2c3d4"
Last-Modified: Wed, 03 Sep 2026 09:12:00 GMT
Cache-Control: public, max-age=300, stale-while-revalidate=86400

That remediation is the one the scanner itself offers when this check fails, read from the same code that does the scoring rather than written alongside it. If it were typed out separately it could drift from the check that judges it, which is the failure that makes published advice worthless.

How often the web fails this

Measured on 2026-09-06 against AX specification v1.5.1, across a sample of 292 public websites.

ResultSitesShare of those assessed
Failed15051%
Warned124%
Passed13045%
Did not apply0excluded from the shares

The last row is kept out of the percentages deliberately. A check that did not apply to 0 of 292 sites is not a check most of the web passes, and folding those in is the commonest way this sort of table is used to say something untrue.

At 51% the web is genuinely split on this one, so it is neither table stakes nor a differentiator on its own.

Testing it on your own site

npx axray-cli your-site.com

All 70 checks against one URL, this one among them, with what was found rather than only whether it passed. Add --min 70 in a pipeline and the command exits non-zero below that score, so a change that breaks what agents can read fails the build instead of shipping.

Questions about this check

What does the "Re-fetching is cheap for a crawler" check mean?

Crawlers revisit. With a validator they send one conditional request and receive an empty 304; without one they download the whole page every time, which is the traffic sites then complain about. It sits in the Reachability pillar of the Agent Experience rubric and is worth 4 points within that pillar, which works out at about 0.8 points of the overall score out of 100.

How do I fix cache-validators?

Send an ETag or a Last-Modified header on HTML responses and answer If-None-Match with a 304. Most static hosts and CDNs do this already; it is usually an origin that generates HTML per request that does not. It is a configuration or content change rather than an engineering one, so it can usually ship the same day.

Is it unusual to fail cache-validators?

No. Of 292 public sites where this check applied, 150 failed it — 51%. The measurement was taken on 2026-09-06 against AX specification v1.5.1, and no site is named.

How do I test cache-validators on my own site?

Run npx axray-cli your-site.com from a terminal. It runs all 70 checks against one URL and prints this one among them, with what it found rather than only whether it passed. It is free, needs no account and installs nothing.

The rest of the Reachability pillar

Can an agent fetch your content at all, without a browser and without being blocked?

See how your own page does on this check and the other 69, with the fixes ordered by how many points each one recovers. One scan, no account.