What a valid llms.txt looks like
llms.txt is markdown with a required shape, read from the root of a host. Checking one means looking at the structure, the links and the response itself, and the response is the part a text editor cannot show you.
The shape the format asks for
One H1 at the top carrying the name of the site or project. A short summary under it, written as a blockquote, saying what the site is in a sentence or two. Then H2 sections, each holding a list of links, where every link has a title and a short note about what the reader will find there. The whole thing is meant to be read start to finish by something with a limited context window, which is why brevity is part of the specification rather than a style preference.
So the parts worth checking separately are: whether the file opens with an H1, whether a summary follows it, how many sections there are, how many links they carry, how many of those links are relative, the byte size, and whether the response body was markdown at all.
The three failures worth catching
Relative links are the quiet one. A path such as /docs/start resolves fine in a browser sitting on your domain and resolves to nothing at all for a reader that was handed the file contents without the URL it came from. Write every link absolute, with the scheme and host. Counting the relative ones turns this into a number rather than a vague feeling that something is off.
An HTML response is the loud one. Static file routing is easy to get wrong on a framework that owns every path, and the result is a 200 with a full page in the body. Any reader that parses markdown gets a wall of tags. Read the first bytes of the body rather than trusting the content type header, because the header is often the thing that is wrong.
Size is the slow one. There is no hard cap in the format, but a file with several hundred links stops being a summary, and a reader that truncates it drops whatever sat at the bottom. Keep the index short and put the bulk in llms-full.txt, which exists for exactly that split.
What publishing it does not do
The file grants no access and blocks nothing. Crawl permission is decided in robots.txt and at your server, and an AI crawler that is disallowed there stays disallowed no matter what your llms.txt says. If your goal is to control access rather than to describe your content, start with how robots.txt matching works instead.
The format, its history and the operators who have actually committed to reading it are covered in the llms.txt glossary entry. If your worry is how much of your site a crawler gets through in the first place, that is a different measurement, and crawl budget explains what sets it.
Checking your own file
Fetch /llms.txt and /llms-full.txt from the host and read the raw response rather than the rendered one. The status, the content type and the first bytes of the body are enough to tell a real markdown file from a 404 page wearing a 200, and the third of those is the one to trust.
Which crawlers reach the file at all is decided elsewhere. GPTBot and ClaudeBot each document their own robots token, and a Disallow there applies to llms.txt like any other path.
Common questions
Does anything read llms.txt today?
Some tools and assistants fetch it when a user points them at a site. No major search crawler has committed to reading it as a ranking or access input, so treat it as documentation for machines rather than as a control.
Is llms.txt a replacement for robots.txt?
No. robots.txt decides whether a crawler may fetch a URL and is enforced by the crawler. llms.txt describes what is worth reading and enforces nothing. A site that wants to block an AI crawler has to do it in robots.txt or at the server.
What is llms-full.txt for?
It carries the actual content rather than a map of it, usually the documentation flattened into one markdown file. Size is the thing to watch, because a multi-megabyte file gets truncated by most readers and whatever sat at the bottom is dropped.
Why would my file be served as HTML?
Because a framework catch-all route or a 404 handler answered instead of a static file, so the body starts with markup and a reader that expected markdown gets a web page. Serve it as text/plain or text/markdown from the site root.