Serve Markdown to agents without confusing your caches

Many public pages are written for people but fetched by software. Returning a Markdown version can give an agent the same words without navigation, scripts, or layout markup. The important part is serving the right representation consistently.

Choose a route shape

There are two common patterns:

1. Separate URL: serve a page at /docs/start.md.
2. Content negotiation: return Markdown from /docs/start when the request asks for Accept: text/markdown, and HTML otherwise.

The separate URL is easy to inspect and cache. Negotiation keeps one canonical path, but the cache must know the response depends on the Accept header.

Set the right headers

Send a Markdown media type with the Markdown body. When the same URL can return HTML or Markdown, include:

Vary: Accept
Content-Type: text/markdown; charset=utf-8

Vary: Accept tells shared caches to keep the two representations separate. Without it, one visitor may receive a response cached for a different kind of request.

HTML pages can advertise a separate Markdown alternative:

<link rel="alternate" type="text/markdown" href="/docs/start.md">

Do not return Markdown for every request based only on a User-Agent string. User agents can be copied, and content negotiation is a direct request for the representation.

Test both paths

curl -sSI https://example.com/docs/start.md
curl -sSI -H 'Accept: text/markdown' https://example.com/docs/start
curl -sS -H 'Accept: text/markdown' https://example.com/docs/start | head

Check status, content type, and the body. If one URL negotiates HTML and Markdown, check for Vary: Accept on both responses. Also test a nonexistent page; an HTML fallback that returns 200 for every route can make a missing document look real.

Backwatch's Markdown negotiation tester checks these public behaviors. For deployment examples, read the Next.js, Cloudflare, or nginx guide.