instagram.com

No JSON-LD and no microdata anywhere on the page.

Structured data is the only part of your page an agent can consume without interpretation. Everything else it has to infer, and inference is where it invents facts about you.

Copy<script type="application/ld+json">
{
  "@context": "https://schema.org",
  "@type": "Organization",
  "name": "Your company",
  "url": "https://example.com",
  "description": "One sentence an assistant can quote.",
  "sameAs": ["https://x.com/you", "https://github.com/you"]
}
</script>

No /llms.txt or /llms-full.txt at https://instagram.com.

Without it, an assistant deciding what your site is about crawls whatever it happens to find. llms.txt is the cheapest way to control that first impression, and today most of your competitors do not have one either.

Copy# Example Inc

> One paragraph an assistant can quote verbatim about what you do and who it is for.

## Docs
- [Quickstart](https://example.com/docs/quickstart): Get running in five minutes.
- [API reference](https://example.com/docs/api): Every endpoint with examples.

## Product
- [Pricing](https://example.com/pricing): Plans, limits and exact prices.
- [Changelog](https://example.com/changelog): What shipped, newest first.

## Policies
- [Terms](https://example.com/terms)
- [Contact](https://example.com/contact): support@example.com

No sitemap found at https://www.instagram.com/sitemap/ig_seo_profile_sitemap_non_media_eligible.xml.gz or in robots.txt.

Without a sitemap, discovery depends entirely on crawlable links. Any page not reachable by a plain <a href> is invisible.

CopySitemap: https://example.com/sitemap.xml

5 of 5 blocked by robots.txt: OAI-SearchBot, Claude-SearchBot, PerplexityBot, Amazonbot, DuckAssistBot.

These crawlers are the ones that put your link inside an AI answer. Blocking them removes the referral traffic, not just the training use.

• OAI-SearchBot (OpenAI) - You will not appear as a source in ChatGPT search results.
• Claude-SearchBot (Anthropic) - You will not be cited in Claude web search answers.
• PerplexityBot (Perplexity) - You lose Perplexity citations, one of the highest-converting AI referral sources.
• Amazonbot (Amazon) - Excluded from Alexa and Rufus style shopping answers.
• DuckAssistBot (DuckDuckGo) - Excluded from DuckDuckGo AI assist answers.

CopyUser-agent: OAI-SearchBot
Allow: /

User-agent: Claude-SearchBot
Allow: /

User-agent: PerplexityBot
Allow: /

User-agent: Amazonbot
Allow: /

User-agent: DuckAssistBot
Allow: /

Only 68 words of readable text reached the agent.

There is not enough here for an assistant to build an answer from, so it will use someone else as the source.

The page has no headings at all.

Without an h1 the agent has no anchor for the document topic and falls back to the title tag or the URL slug.

0 of 1 images have no alt attribute at all, 1 more are empty.

To an agent an undescribed image is a hole in the page. On product, chart and diagram pages that hole is usually the point of the page.

Copy<img src="/chart.png" alt="AX scores of the top 100 SaaS sites, median 41 out of 100">

0.2% of the payload is readable text.

The document is almost entirely machinery. An agent truncating this page will likely truncate before reaching any of your content.

3 of 3 blocked by robots.txt: ChatGPT-User, Claude-User, Perplexity-User.

These fetches happen because a human just pasted your link into an assistant. A block here fails in front of a real person who wanted to read you.

• ChatGPT-User (OpenAI) - When a user asks ChatGPT to open your link, it fails in front of them.
• Claude-User (Anthropic) - Claude cannot open your page when a user pastes the link.
• Perplexity-User (Perplexity) - Live user-requested fetches from Perplexity fail.

CopyUser-agent: ChatGPT-User
Allow: /

User-agent: Claude-User
Allow: /

User-agent: Perplexity-User
Allow: /

No /.well-known/mcp.json and no ai-plugin.json.

This is the highest-leverage unclaimed slot on the web right now. The sites that publish a callable surface get used directly instead of guessed at.

Copy{
  "schema_version": "2025-06-18",
  "name": "example",
  "description": "Search the catalogue and check availability.",
  "documentation": "https://example.com/docs",
  "servers": [
    { "type": "http", "url": "https://mcp.example.com" }
  ],
  "contact": "support@example.com"
}

Only 68 words arrived in the raw HTML. Some of the page is probably assembled in the browser.

Partial server rendering means agents see a fragment of your argument and may summarise you from the fragment.

Only 0 headings on the page.

Agents chunk long pages by heading. A page with no outline is chunked arbitrarily and quoted out of context.

No <main> element. Landmarks present: footer.

Without a main landmark an extractor has to guess your content boundary heuristically, and on marketing pages it usually guesses the header.

Copy<main>
  <article>
    <h1>…</h1>
    …
  </article>
</main>

1% semantic (1 semantic vs 121 div).

A page built entirely from divs communicates its structure only through CSS, which agents never load. The visual hierarchy you designed does not reach them.

The title is only 9 characters: "Instagram".

Very short titles force the assistant to guess at context and it will guess from the URL.

No Last-Modified, ETag, dateModified or <time> element anywhere.

Agents deprioritise content they cannot date, and re-fetch undated pages more often, which costs you bandwidth for nothing.

Copy{"@type":"Article","datePublished":"2026-01-15","dateModified":"2026-08-01"}

No Organization structured data on the page.

Entity resolution is how assistants decide two mentions are the same company. Unresolved entities get merged with, or mistaken for, someone else.

Copy{"@context":"https://schema.org","@type":"Organization","name":"Example Inc","url":"https://example.com","logo":"https://example.com/logo.png","sameAs":["https://x.com/example","https://github.com/example","https://www.linkedin.com/company/example"]}

Canonical points to another origin: https://www.instagram.com/

Agents dedupe sources by canonical URL, and cross-origin canonicals hand your citation to somebody else.

No rel="license" link and no license or copyright fields in structured data.

If you care how your content is reused, say so where machines can read it. A footer copyright line is not machine-readable.

Copy<link rel="license" href="https://example.com/terms">

Missing: og:description, og:type.

When a page is JavaScript-heavy or truncated, Open Graph tags are frequently the only clean summary an agent gets.

Copy<meta property="og:title" content="Page title">
<meta property="og:description" content="One sentence.">
<meta property="og:type" content="website">
<meta property="og:url" content="https://example.com/page">
<meta property="og:image" content="https://example.com/og.png">

No /.well-known/security.txt.

It is the standard machine-readable place to say who to contact. Agents and researchers both look there first.

CopyContact: mailto:security@example.com
Expires: 2027-01-01T00:00:00.000Z
Preferred-Languages: en

2 div/span elements styled as buttons against 1 native controls.

Agents identify what they can click from element semantics. Styled divs are invisible as controls, so any flow gated behind one cannot be completed.

docs link: https://developers.facebook.com/docs/instagram

A docs link is a start, but nothing tells an agent where the machine-readable entry point is.

No crawlable GET search form and no SearchAction.

Without a GET-addressable search, an agent looking for one specific page on your site has to fetch dozens of pages to find it, and usually gives up first.

No twitter:* card tags.

Several agents fall back to card metadata when they cannot extract the body reliably.

Copy<meta name="twitter:card" content="summary_large_image">
<meta name="twitter:title" content="Page title">
<meta name="twitter:description" content="One sentence.">

Only one weak contact signal found.

"How do I get in touch with them" is one of the most common agent-mediated questions about a company. Make the answer machine-readable.

Copy{"@type":"Organization","contactPoint":[{"@type":"ContactPoint","contactType":"customer support","email":"support@example.com","url":"https://example.com/contact"}]}

462 KB of HTML, 54% of it inline script.

Agents truncate long documents. Anything past the cut simply is not read.

Only 1 of 2 text fields declare an autocomplete token.

These tokens are the closest thing the web has to a machine-readable form schema, and they cost one attribute each.

Copy<input name="email" type="email" autocomplete="email">

1 field without a name attribute.

Agents build form submissions from name attributes. Missing or randomised names make your form unusable to automation and fragile for everyone.

• <input type="checkbox" name="">

HTTP 200 in 866 ms.

The baseline: an agent with no browser, no cookies and no JavaScript received your page.

The response body was the real page, not a challenge or block screen.

Bot protection that cannot tell a shopping assistant from a scraper silently removes you from AI answers, with no error anyone on your team will ever see.

robots.txt served with 36 user-agent groups.

A valid robots.txt is how you talk to crawlers before they read a single page.

All 2 crawlers in this group may fetch /.

AI Overviews and Copilot answers are grounded on the classic search index. Blocking Googlebot or Bingbot now also removes you from AI surfaces.

Robots directives present and permissive: noarchive, noimageindex

noindex and nosnippet remove you from the surfaces that would otherwise quote and link you.

1 redirect before the final page.

Every hop costs latency and some agents cap redirects at two or three.

• 301 https://instagram.com/ -> https://www.instagram.com/

Full response in 866 ms.

Agents fetch many pages per answer and drop the slow ones. Speed is a ranking input you control.

Content-Type: text/html; charset="utf-8"

A declared charset avoids mojibake when an agent quotes your text back to a user.

94 characters: "Create an account or log in to Instagram - Share what you're into with the people who get you."

This is the summary assistants reuse verbatim more often than any other field.

<html lang="en">

Language is how an assistant decides whether to answer a user in your words or translate them.

Site root.

Assistants show the URL next to the citation, and readable URLs get clicked more than opaque ids.

All 18 links resolve to a real URL.

Real hrefs are the only navigation primitive an agent has.

100% of links have descriptive text.

Link text is how an agent decides which link to follow when it is looking for something specific.

8 internal links give an agent somewhere to go next.

Agents explore a site breadth-first from whatever page they landed on. Dense internal linking is how the rest of your content gets found.

All 3 fields have an associated label or aria-label.

An agent filling a form on a user's behalf reads labels to decide what goes in each box.

robots.txt names specific AI crawlers rather than relying on the wildcard group.

Naming crawlers explicitly is how you separate "do not train on me" from "do not send me traffic". The wildcard group cannot express that distinction.

Live user-agent probing was not run for this scan.

Some sites answer 200 to a generic client and 403 to a named AI crawler. Only a live probe finds that.

Not needed: the page already server-renders its content.

No JSON-LD to validate.

No JSON-LD to evaluate.

No prices detected in the page text.

Top-level page; breadcrumbs are not expected.

No microdata; JSON-LD is the preferred format anyway.

No llms.txt to evaluate.