llms.txt

Also called llms full txt, llms.txt file

A proposed file at /llms.txt that describes a site for a language model reading it: a short summary, then a curated list of links with one line of context each, written in markdown.

# Example Docs

> Documentation for the Example API.

## Core
- [Quickstart](https://example.com/quickstart): install and first request
- [Authentication](https://example.com/auth): API keys and scopes

A companion convention, /llms-full.txt, inlines the content itself rather than linking to it.

The mistake

Reading it as an AI version of robots.txt. It is the opposite kind of file. Robots.txt restricts access. llms.txt offers a guided summary, and it has no access control semantics whatsoever. Publishing one grants nothing and blocks nothing.

The second mistake is assuming adoption. No major operator has publicly committed to reading it. GPTBot, ClaudeBot, PerplexityBot and Google-Extended are all documented in terms of robots.txt tokens, and none of their documentation mentions llms.txt. The proposal is real and the tooling around it is real; the consumption is speculative.

That does not make it worthless. For a documentation site, writing a clean index of what exists and what each page covers is useful work regardless of who reads the output, and it is cheap to generate from the same source that builds your sitemap. Just be clear about which column it belongs in. It is a bet on a convention, not a control you can rely on.

If your actual goal is controlling AI use of your content, the levers that exist today are per-operator robots.txt tokens, and an edge rule matched against published IP ranges for anything that ignores them. A 403 is enforcement. A markdown file is a suggestion.