llms.txt: What It Is, Why It Matters, and How to Add It to Your Site

AEO

llms.txt is a plain-text file that lives at the root of your website (yoursite.com/llms.txt) and gives AI systems a curated summary of what your site is about and where your most important content lives. Think of it as a table of contents written for language models: robots.txt tells crawlers where they may go, sitemap.xml lists every URL, and llms.txt explains what actually matters and why.

What is llms.txt, exactly?

The llms.txt standard was proposed by Jeremy Howard of Answer.AI in September 2024. It is a Markdown file with a simple structure: an H1 with your site or project name, a short blockquote summary, and then sections of links with one-line descriptions. Because it is Markdown, both humans and language models can read it without any parsing gymnastics.

A minimal example looks like this:

# Rankioz

> Rankioz is an AEO platform that tracks how often ChatGPT, Perplexity, Gemini and Claude mention your brand, and publishes content that earns citations.

## Product
- [Pricing](https://rankioz.com/pricing/): Free plan and Premium at $49.99/mo
- [AEO Toolkit plugin](https://rankioz.com/aeo-toolkit/): Free WordPress plugin for AI crawler control, llms.txt and schema

## Guides
- [What is Answer Engine Optimization?](https://rankioz.com/what-is-aeo/): Plain-language introduction to AEO

There is also an optional companion file, llms-full.txt, which contains the full text of your key pages in one document, for tools that want to ingest everything at once.

Do AI engines actually use llms.txt?

Honest answer: adoption is early and uneven, and you should treat llms.txt as a low-cost bet rather than a guaranteed ranking lever. Anthropic publishes llms.txt for its own documentation, and a growing list of developer-tool companies (Zapier, Stripe, Cloudflare and thousands of documentation sites) serve one. Several AI coding assistants and answer tools fetch it when present. Google has said its AI products rely on conventional crawling and indexing, so llms.txt complements rather than replaces normal SEO.

Here is the practical math: the file takes ten minutes to create, costs nothing to serve, cannot hurt you, and puts a clean, self-authored summary of your site exactly where future AI systems will look for one. Early adopters of robots.txt and schema.org enjoyed years of advantage while competitors caught up. llms.txt has the same risk profile: tiny downside, meaningful upside.

What should go in your llms.txt?

Curate, do not dump. The file exists to answer two questions for a machine with limited attention: what is this site, and which pages should I read to understand it?

  • The one-paragraph summary. Write the blockquote as if answering "what is this company?" in two sentences. This text often gets quoted nearly verbatim, so make it accurate and specific.
  • Your money pages. Pricing, product overview, and the two or three pages that define what you sell.
  • Your best answers. The guides and articles that directly answer the questions customers ask. These are the pages you want an AI to cite.
  • Skip the noise. No tag archives, no pagination, no legal boilerplate unless it genuinely matters for your product.

How to add llms.txt to WordPress

You have three options, from easiest to most manual:

  1. Use a plugin. Our free Rankioz AEO Toolkit generates a valid llms.txt from your site title, tagline and key pages, serves it at /llms.txt with the correct text/plain content type, and lets you edit every line before it goes live. It also handles the two related jobs most sites get wrong: AI crawler access in robots.txt and the schema markup that lets engines attribute your content.
  2. Use your SEO plugin if it supports it. Yoast and some others have added llms.txt features. Check that the output follows the Markdown structure above rather than a plain URL list.
  3. Create it manually. Write the Markdown file, upload it to your site root, and confirm yoursite.com/llms.txt returns it as plain text. On WordPress this usually means an FTP upload or a snippet that registers a rewrite rule, which is exactly the fiddly work the plugin route avoids.

How to add llms.txt to any other site

If you control your web root, this is a five-minute job: write the file, name it llms.txt, and deploy it alongside robots.txt. For static sites (Next.js, Astro, Hugo, plain HTML), drop it in your public directory. Verify three things: the URL returns HTTP 200, the content type is text/plain, and the links in it are absolute URLs.

Common llms.txt mistakes

  • Blocking AI crawlers while serving llms.txt. If your robots.txt blocks GPTBot, ClaudeBot or PerplexityBot, the llms.txt is pointless because the engines cannot fetch your content anyway. Audit crawler access first.
  • Dumping your entire sitemap. A 500-link llms.txt defeats the purpose. Fifteen well-chosen links beat five hundred unsorted ones.
  • Letting it rot. Update the file when you ship new key pages, change pricing, or rename products. A stale summary teaches AI systems the wrong facts about you.
  • Stopping at llms.txt. The file is one of three basics. The other two are AI crawler access and structured data (Organization, Article and FAQ schema). Engines can only cite what they can read and attribute.

How do I know if any of this is working?

Measure it. Ask the AI assistants the questions your customers ask and check whether the answers mention you. Doing this by hand across ChatGPT, Perplexity, Gemini and Claude gets old fast, which is why we built Rankioz: it runs those questions daily, tracks how often your brand appears in the answers, and shows your share of voice against competitors. The free plan includes AI visibility tracking, so you can see your baseline before and after adding llms.txt.

Frequently asked questions

Is llms.txt an official standard?

It is a community proposal, not a formal web standard. Adoption is driven by usefulness, the same way robots.txt and sitemap.xml spread before any standards body blessed them.

Does llms.txt replace robots.txt or sitemap.xml?

No. Keep all three. robots.txt controls access, sitemap.xml enumerates URLs for indexing, and llms.txt explains meaning and priority to language models.

Will llms.txt improve my Google rankings?

Not directly. It targets AI answer engines, not the classic ranking algorithm. Your Google SEO and your AEO share most of their foundations, though: readable HTML, clear structure, and content that actually answers questions.

Where can I see real examples?

Try docs.anthropic.com/llms.txt, or our own rankioz.com/llms.txt once you finish this article. The llmstxt.org site maintains a directory of adopters.