What llms.txt can and can’t do for your site

llms.txt is a plain-text index that points an AI reader toward the pages that matter. It can make a site's structure easier to discover when a person or tool asks an agent to read it. It does not make the content public by itself, and publishing the file does not guarantee that a model will fetch or cite your site.

Keep the file useful

Start with the product or project name, one factual sentence, and a few links to stable pages. Group related pages under short headings. Link to canonical URLs that return useful content without a login.

# Example product

> A one-sentence description of the product.

## Docs

- [Getting started](https://example.com/docs/start)
- [API reference](https://example.com/docs/api)
- [Security](https://example.com/security)

Do not copy the whole site into this file. A short index is easier to keep accurate, and a focused index avoids making the reader guess which page is current.

What it does not replace

llms.txt is not the same as:

• robots.txt, which expresses crawler access preferences;
• a sitemap, which lists URLs for search crawlers;
• authentication, which protects private content; or
• server-rendered page content, which makes a page readable without running scripts.

Treat the format as a useful convention, not a search ranking factor. Keep relying on normal links, accurate page metadata, and a clear access policy.

Check the file after publishing

Confirm the file is available and its links work:

curl -fsS https://example.com/llms.txt
npx backwatch example.com --pillar readability

The Backwatch llms.txt checker checks whether the file exists, follows the expected structure, and links to reachable pages. It cannot tell whether every AI product will use the file.

For platform-specific setup, see the guides for Next.js, Cloudflare, and nginx.