Search engines used to be the only crawler that mattered. In 2026 a second class of visitor arrives on every website: language-model crawlers that read the page, summarise it, and answer a user's question without ever showing your link. If the LLM cannot parse who you are, what you offer and how to reach you, your site is invisible to the fastest-growing discovery channel of the decade.
This post is a working checklist. It is derived from a small experiment: I fed 40 consultant websites to four frontier models and asked the same three questions of each. The sites that scored well shared eight properties. The sites that failed shared three.
1. Serve real HTML on the first request
Most LLM crawlers do not execute JavaScript. If your hero copy is rendered client-side, the model sees an empty shell. Server-side rendering (SSR) or static generation is the single largest predictor of retrieval quality. Test with `curl -A "GPTBot" https://your.site` and read what actually comes back.
2. Write one clear H1 per page
Language models weight headings heavily. A page with three H1 tags competes with itself. One H1 that names the page, followed by descriptive H2s, produces the cleanest outline. The outline is what the model quotes back to users.
3. Put the answer above the fold — in prose, not marketing
Models reward pages that answer the implicit question in the first paragraph. "I help mid-sized companies redesign their strategy through facilitated workshops" beats "Unlocking potential through transformative experiences" every time. Concrete nouns, concrete verbs.
4. Use semantic HTML for structured content
- <article> for standalone posts and case studies
- <section> for grouped content with its own heading
- <nav> for navigation, <footer> for footers, <main> for the primary content column
- <dl>/<dt>/<dd> for definitions and FAQ-style pairs
- <time datetime="2026-06-02"> for dates so the model reads them unambiguously
5. Add JSON-LD schema at the leaf
schema.org markup is the closest thing to a direct message to the crawler. Organization on the root, Person on the bio page, Article on blog posts, Service on service pages, BreadcrumbList on deep routes. Validate with Google's Rich Results test — it doubles as a JSON-LD linter.
{
"@context": "https://schema.org",
"@type": "Article",
"headline": "How to build an LLM-crawler friendly website",
"author": { "@type": "Person", "name": "Marko Rillo" },
"datePublished": "2026-06-02",
"description": "A working checklist for making your site legible to LLM crawlers."
}6. Keep the robots.txt permissive for the crawlers you want
Blocking GPTBot, ClaudeBot, PerplexityBot and Google-Extended is a choice — but choose deliberately. If your business depends on being cited when a prospect asks "who does strategy facilitation in the Nordics," letting the crawlers in is the whole game.
7. Publish a sitemap.xml and link it from robots.txt
LLM crawlers use sitemaps to discover pages faster than by following links. Include every canonical URL, keep it under 50MB, list one language variant per URL, and reference the sitemap from robots.txt with `Sitemap: https://your.site/sitemap.xml`.
8. Write for the answer, not the click
The old SEO game was to tease enough to earn a click. The new game is to answer completely enough to earn a citation. A page that fully answers a question gets quoted with attribution. A page that withholds gets summarised without attribution — or ignored entirely.
The three failure modes
- Client-side-only rendering. The model sees a loading spinner and moves on.
- Marketing prose. "We empower change agents" tells the model nothing about what you actually do.
- Missing contact detail. If the model cannot find a phone, email or booking link, it cannot recommend you.
How to test in ten minutes
- Open a model with browsing (ChatGPT, Claude, Perplexity, Gemini).
- Ask: "What does the person / company behind [your URL] do, and how would I hire them?"
- Read the answer literally. Anything wrong or missing is a page-level fix on your side.
“If a language model cannot describe your business in one paragraph, neither can your prospects.”
The web is being re-read by a new class of reader. Making a site legible to that reader takes an afternoon and pays back for years. Start with SSR and one clear H1 — everything else is optimisation on top.