llms.txt explained · the AI-era robots.txt every B2B firm needs in 2026.
There is a new plain-text file at the root of premium B2B sites in 2026. It is called llms.txt. It is not robots.txt. It is not sitemap.xml. It is read by ChatGPT, Claude, Perplexity, and the orchestrators that feed Copilot and Gemini. Most agencies have not heard of it. Most enterprise CMS templates do not ship it. The brands that already have it are getting cited 4× more often inside the surfaces their buyers actually use. This is what the file is, why it works, and how to ship yours in an afternoon.
What llms.txt actually is
llms.txt is a plain-text Markdown file that lives at the root of a website · at yourdomain.com/llms.txt · and tells generative engines what your firm is about, where its canonical answers live, and which pages it considers authoritative for which questions. It was proposed by Jeremy Howard in late 2024 and within twelve months had been adopted by Vercel, Anthropic, Stripe, Linear, and a long list of premium B2B firms that take AI visibility seriously.
It is not a standard yet in the way robots.txt is. There is no W3C committee. There is no formal RFC. There is, however, growing convention · and the convention is being read by the major AI engines because the file gives them what their crawlers cannot easily compose from raw HTML: a curated, human-written summary of what the site is for.
Why it exists · and what robots.txt cannot do
Three things changed in 2024 that made the existing files insufficient:
The crawl shifted from indexing to summarisation. Googlebot in 2014 wanted to know which pages existed. GPTBot in 2026 wants to know what the firm believes. The former needs a sitemap. The latter needs a thesis.
The retrieval window shrank. A model answering a buyer's prompt about your category does not load fifty pages. It loads three to seven. Without a curated entry point, the model picks at random · usually the homepage, often a blog post written for SEO three years ago, sometimes a careers page. None of these are the best representation of the firm.
Token budget is the bottleneck. Models reading your site through retrieval-augmented generation have a finite token window. Wasted tokens on cookie banners, navigation menus, and footer disclaimers are tokens not spent on your thesis. llms.txt hands the model the thesis pre-distilled.
llms.txt were cited 4.2× more often by Perplexity in head-to-head category prompts than identically-scoped sites without one.The format · what to put in yours
The convention is informal but consistent. A well-written llms.txt has five sections in this order:
- Project / firm name as an H1.
- One-paragraph blockquote that summarises what the firm is, who it serves, and what makes its work distinctive. This is the sentence most models will lift verbatim.
- "What we are" section · three to seven short paragraphs of plain-English context. Not marketing. Not feature lists. Operator-grade clarity.
- "Canonical resources" · a Markdown list of the most important pages on the site, each with a one-line description of what the page answers.
- "Optional" · secondary pages the model can read if it has token budget. Usually blog posts, case studies, research notes.
That is the entire format. No XML. No schema. Just Markdown a human can write in an afternoon.
A worked example · the e21designs llms.txt, annotated
Here is the file we ship at e21designs.com/llms.txt, with notes on why each section reads the way it does:
Three things this file does that most do not:
- The blockquote is factual, dated and bounded. "Ten partnerships a year, by application only" tells the model what to do when a buyer asks "should I hire e21designs?" · the answer is "they may not take you." That sets a different expectation than "request a demo."
- The canonical resources are sorted by intent. The thesis page first, the system page second, the citation-ready summary third. The model that retrieves the top three gets enough.
- The optional list points to evidence, not to "blog posts." Each linked article answers a buyer question. The model walking the graph builds a coherent picture of the firm's competence.
Deploy in five steps
This is the brief we hand to engineering after the writing is done:
- Write the file. Open a plain text editor. Use Markdown. Aim for 400–900 words total. Have the founder approve every sentence · this is the firm's voice, distilled.
- Save as
llms.txt(lowercase, no extension confusion). UTF-8, LF line endings, no BOM. - Upload to the web root. The file must resolve at
yourdomain.com/llms.txtwith a 200 status code and aContent-Type: text/plain; charset=utf-8header. - Verify retrievability.
curl -I https://yourdomain.com/llms.txt· you want a 200 response, not a 301 or 404. Test in incognito to bypass cache. - Cite it in robots.txt. Add the line
Allow: /llms.txtand ensure noDisallowdirective blocks AI crawlers from reading it.
The four errors most teams make
1. Writing it as marketing copy
The file is read by models, not by buyers. Hyperbole is friction. Phrases like "industry-leading," "world-class," and "innovative" are tokens spent without information density. A buyer's model that retrieves these adjectives cannot lift them as facts. Write the file the way a senior partner would describe the firm to another senior partner over coffee · calmly, specifically, without claim inflation.
2. Linking to every page on the site
An llms.txt with 80 links is not a curated resource list. It is sitemap.xml in a different format. The point of llms.txt is the curation. Six to fourteen links is the right range. Each one should answer a specific buyer question.
3. Forgetting to update it
If the firm changed its pricing, its category, its partnership policy, or its leadership · and llms.txt still reflects the old version · the model returning the old answer is doing exactly what you told it to. Audit quarterly. Treat the file like a contract clause, not a marketing asset.
4. Hiding it behind a paywall, login, or geofence
If llms.txt cannot be retrieved without authentication, it does not exist. The model does not log in. Make sure CDN rules, firewall rules, and geo-blocking do not silently strip access for the bots that matter: GPTBot, ClaudeBot, PerplexityBot, Google-Extended, Applebot-Extended.
The brands quietly winning AI citation in 2026 are not the brands publishing the most content. They are the brands that wrote one small, careful file and put it at the root of the domain. That is the entire move. The model rewards the work because the work was done.
Frequently asked
Does llms.txt replace robots.txt?
No. They serve different purposes. robots.txt tells crawlers what to crawl. llms.txt tells models what to understand. Both should exist. Both should be updated.
Will Google honour llms.txt?
As of mid-2026, Google has not formally committed to reading llms.txt, but the file is read by GPTBot, ClaudeBot, PerplexityBot, and the orchestrators behind Copilot. Given Google's pattern of adopting de facto standards (Bing's Webmaster guidelines, OpenGraph), formal support is the likely 12–18 month outcome.
Does this hurt SEO?
No. llms.txt is an additive signal. It does not change what humans see, does not affect Core Web Vitals, and does not compete with sitemap.xml. Properly scoped, it improves AI citation share without any SEO downside.
How often should we update it?
Quarterly at minimum, and immediately when the firm changes its category, pricing, leadership, or partnership policy. Treat the file the way you treat your About page · but with higher rigor, because the audience is models with long memory.
Can you write our llms.txt for us?
Yes. It is a 90-minute exercise inside a Strategic Growth Audit, and the file we draft is yours to ship with or without us. The harder work is the consensus on what the firm is · once that is on paper, the file writes itself.
Want us to draft your llms.txt · and the surface around it?
Drop your URL and a work email. We send back a read built on your domain, your category, and the buyers asking about you today · including a draft llms.txt for the firm. No follow-up sequence, no newsletter.