# Make a Next.js site readable to AI agents

A practical Next.js setup for server-rendered pages, llms.txt, Markdown alternatives, and routes agents can fetch without a browser.

Updated 2026-11-09.

AI agents often fetch public pages with a plain HTTP request. A Next.js page can look complete in a browser and still be empty to a reader that does not run JavaScript. Keep the useful content in the server response, publish a small index, and make the clean text version easy to find.

## 1. Render the content on the server

Use Server Components for public copy and docs data. A client component can add interaction, but the page title, explanation, examples, and links should already be present in the HTML response.

Check the first response without executing scripts:

```sh
curl -fsS https://your-site.com/docs | head -c 1200
```

If the response contains only an app shell, move the public content into a Server Component or statically render the route. Do not add private data to a public render just to make the page easier to crawl.

## 2. Publish a short llms.txt

Place `llms.txt` at the site root. Say what the project is, then link to the few pages that explain the product and its API. Keep the file maintained as routes change.

```md
# Example product

> A short, factual description of what the product does.

## Docs

- [Getting started](https://your-site.com/docs/start)
- [API reference](https://your-site.com/docs/api)
```

`llms.txt` is a useful index convention, not a replacement for robots.txt or a sitemap, and its presence does not guarantee that a model will cite the site. See Backwatch's [llms.txt checker](/tools/llms-txt-checker).

## 3. Offer Markdown as a second representation

For each important page, provide either a `.md` route or a response negotiated with `Accept: text/markdown`. If one URL returns HTML or Markdown depending on that header, return `Vary: Accept` so caches do not mix the two representations.

Next.js can advertise the Markdown URL in metadata:

```ts
import type { Metadata } from "next";

export const metadata: Metadata = {
  alternates: {
    types: { "text/markdown": "/docs/start.md" },
  },
};
```

Confirm both representations:

```sh
curl -sSI https://your-site.com/docs/start.md
curl -sSI -H 'Accept: text/markdown' https://your-site.com/docs/start
```

The Markdown response should use a Markdown content type. The negotiated HTML route should vary on `Accept`. Backwatch's [Markdown tester](/tools/markdown-negotiation-tester) checks the common paths and headers.

## 4. Make missing routes honest

Return an actual 404 for a missing page, not a 200 response containing “not found.” Keep robots.txt, the sitemap, and any public `/.well-known` files reachable through the app router, reverse proxy, and CDN.

Run a focused check after deployment:

```sh
npx backwatch your-site.com --pillar readability
```

Backwatch checks the public responses it can observe from outside. A passing scan does not prove that every AI product indexes or uses the content.