Free tools
No account, no card, nothing to install. Each one is a piece of the scanner on its own page, and each one says plainly what it will not do for you.
Decide once per category — train on me, cite me, fetch when a human asks, shop on my behalf — and get a robots.txt naming all 54 AI crawlers we track, with the cost of every refusal stated beside it.
What it refuses to do: It will not hand you a one-line snippet. "Block the AI bots" almost always also blocks the crawler that sends you visitors, and a tool that made that easy would be doing the damage faster.
Paste a URL and get a first draft of llms.txt, built from that site's own title, description and navigation.
What it refuses to do: It leaves the one-sentence description after each link blank. Only you know what is on those pages, and inventing a sentence would be inventing a fact about a site we fetched once.
No. Neither tool asks for one, neither takes a card, and neither spends a scan from your daily allowance. An account is free as well and raises the scanner's daily limit, but nothing on this page is behind it.
The robots.txt generator never fetches your site at all — it builds the file from a roster that ships with the scanner, so nothing leaves your browser but the categories you ticked. The llms.txt generator fetches the one page you give it, reads its title, description and navigation, and stores nothing. Neither publishes anything anywhere. Nothing about a site appears publicly unless its owner verifies the domain and asks to be listed.
They do different jobs and you want both. robots.txt grants and refuses permission:
it is the file that decides which crawlers may read you at all. llms.txt grants
nothing — it is a curated short map of your site, the twenty links you would send somebody
who asked what you do. The longer
comparison puts sitemap.xml beside them too.
Both tools above are one answer each. The scanner is all 70 of them: paste a URL and get a score out of 100 with an ordered fix list, in about two seconds. There is a command-line version for your pipeline, a JSON API, and an MCP server so you can ask an assistant instead of opening a page.
See what an AI crawler actually gets from your site.