LLMs are crawling your Next.js site. They're parsing 26 KB of HTML, scripts, and hydration data to extract 101 bytes of actual content.
RSC payloads. Script tags. Font preloads. All to render a title, a price, and a description.
The fix is already built into HTTP: content negotiation. Check the Accept header, respond accordingly. JSON for apps, XML for legacy systems, Markdown for LLMs. Backend developers do this all the time - but Next.js has no built-in way to handle it.
I built next-md-negotiate to close that gap.
It lets you define Markdown versions of your pages alongside your existing routes. Requests with `Accept: text/markdown` get transparently rewritten to your Markdown definitions. Browsers still get your normal HTML. Same URL. No duplicate routing.
Browser -> GET /products/42 Accept: text/html -> normal Next.js page
LLM agent -> GET /products/42 Accept: text/markdown -> clean Markdown
257x smaller payload. Fewer tokens in, better context out.
This also has a massive impact on crawl budget. Markdown responses are tiny compared to full HTML, so AI crawlers can index far more pages in the same amount of time. For e-commerce sites with thousands of product pages, this is critical - it can mean the difference between an AI agent seeing a fraction of your catalog vs nearly all of it.
Works with both App Router and Pages Router. Type-safe route params. Plugs into existing middleware or uses Next.js rewrites.
LLMs are crawling your Next.js site. They're parsing 26 KB of HTML, scripts, and hydration data to extract 101 bytes of actual content.
RSC payloads. Script tags. Font preloads. All to render a title, a price, and a description.
The fix is already built into HTTP: content negotiation. Check the Accept header, respond accordingly. JSON for apps, XML for legacy systems, Markdown for LLMs. Backend developers do this all the time - but Next.js has no built-in way to handle it.
I built next-md-negotiate to close that gap.
It lets you define Markdown versions of your pages alongside your existing routes. Requests with `Accept: text/markdown` get transparently rewritten to your Markdown definitions. Browsers still get your normal HTML. Same URL. No duplicate routing.
Browser -> GET /products/42 Accept: text/html -> normal Next.js page
LLM agent -> GET /products/42 Accept: text/markdown -> clean Markdown
257x smaller payload. Fewer tokens in, better context out.
This also has a massive impact on crawl budget. Markdown responses are tiny compared to full HTML, so AI crawlers can index far more pages in the same amount of time. For e-commerce sites with thousands of product pages, this is critical - it can mean the difference between an AI agent seeing a fraction of your catalog vs nearly all of it.
Works with both App Router and Pages Router. Type-safe route params. Plugs into existing middleware or uses Next.js rewrites.
Two commands to scaffold everything.Repo: https://github.com/kasin-it/next-md-negotiate