llms.txt Example: A Real Next.js Implementation

The shortest useful llms.txt example is a Markdown file at your site root with an H1, a one-line summary in a blockquote, and a few H2 sections of annotated links. That is essentially the whole specification. The harder questions are what belongs in it, how to serve it from a Next.js App Router project without it drifting out of date, and whether anything reads it — and on that last one the honest answer is that nobody has shown that a major assistant consumes yours. This article covers all three, with the file and the code we actually ship.

What llms.txt actually is

The convention was proposed by Jeremy Howard in September 2024 and is maintained at llmstxt.org, where the specification is now at v2. It asks for a file at /llms.txt in Markdown, structured as:

• an H1 with the project or site name (the only required section)
• a blockquote with a short summary
• zero or more free Markdown sections
• zero or more H2-delimited file lists, where each list item is a link followed optionally by a colon and a note

The stated goal is to give an agent a curated, low-noise map of a site, rather than making it infer structure from navigation HTML. The spec also recommends offering clean Markdown versions of important pages, either by appending .md to the URL or by replacing the extension.

That is the entire proposal. It is a content-discovery convention, not a standard, and no body ratifies it.

llms.txt vs robots.txt vs sitemap.xml

These get conflated constantly, usually by people selling a plugin that writes one of them.

The important distinction is between the second and third columns. sitemap.xml is exhaustive and machine-shaped; it lists your privacy policy and your pagination pages alongside everything else. llms.txt is editorial. It is the twenty pages you would hand someone who asked what this company does, each with a sentence explaining why it is there. Those are different artifacts, and the second one does not replace the first.

If someone tells you llms.txt is "robots.txt for AI", they have the wrong mental model. robots.txt controls access. llms.txt grants nothing and blocks nothing.

A real llms.txt example

Here are the first two sections of the file this site serves at /llms.txt, verbatim:

# Devfinix

> Full-service digital agency delivering custom web development, mobile app development, and ROI-driven performance marketing. Serving clients worldwide.

Devfinix builds custom software, high-performance websites, mobile apps, and runs SEO/AEO, Google Ads, and Meta Ads campaigns. Founded in 2022 by Burhan Tahir (Founder & CEO).

## Company

- [About](https://devfinix.com/about): Company story, team, and timeline.
- [Contact](https://devfinix.com/contact): Get a free project proposal.
- [Portfolio](https://devfinix.com/portfolio): Web, app, and marketing work.
- [Case Studies](https://devfinix.com/case-studies): In-depth write-ups of custom builds.
- [Sitemap](https://devfinix.com/sitemap-page): Full list of pages on the site.

## Services

- [Web Design & Development](https://devfinix.com/services/web-design): Custom, conversion-optimised websites on Next.js and React.
- [Shopify Development](https://devfinix.com/services/shopify): Custom Shopify stores and conversion-rate optimisation.
- [WordPress Development](https://devfinix.com/services/wordpress): Custom WordPress sites built for speed and security.
- [App Development](https://devfinix.com/services/app-development): Native iOS/Android and cross-platform mobile apps.
- [Software Development](https://devfinix.com/services/software-development): End-to-end web app and enterprise software engineering.
- [Custom Software](https://devfinix.com/services/custom-software): Bespoke software built around specific business workflows.
- [SEO & AEO](https://devfinix.com/services/seo): Search and answer engine optimisation for organic growth.
- [Google Ads](https://devfinix.com/services/google-ads): Managed Google Search, Display, Shopping, and YouTube ad campaigns.
- [Meta Ads](https://devfinix.com/services/meta-ads): Facebook and Instagram ad campaign management.

The file continues with case studies, blog categories, free tools such as the invoice generator and the significant figures calculator, and legal pages. Four things about it are deliberate:

Absolute URLs. The spec's examples use them, and a file that might be fetched out of context should not rely on the fetcher resolving relative paths correctly.

One sentence per link, written for the destination. "Custom Shopify stores and conversion-rate optimisation" is a description. "Learn more about our Shopify services" is not.

Sections that match how a person would ask. Company, Services, Case Studies, Blog, Free Tools, Legal. Not the navigation order, and not the sitemap order.

Nothing in it that is not also on the site. Every claim in that file is a claim the corresponding page makes in HTML. A curated file that says things the pages do not is a liability, not an asset.

Serving it from the Next.js App Router

Two options, and the trade-off between them is the whole engineering question.

Option A: a static file in the public directory

Put the file at public/llms.txt and Next.js serves it at /llms.txt with no configuration. That is what this site does today, for the honest reason that it took ten minutes.

The cost is drift. Add a service page, rename a route, publish an article, and the file goes quietly stale. Nothing in the build knows it exists.

Option B: a route handler generated from your route data

If your routes already live in a data file — and on this site they do, because src/lib/site-routes.ts is the single source of truth that feeds both sitemap.xml and the human sitemap page — you can generate the file instead. A route handler at src/app/llms.txt/route.ts does it:

import { getAllSummaries } from "@/lib/blog"
import { CATEGORIES } from "@/lib/blog/categories"
import { SITE_ROUTES, type SiteRoute } from "@/lib/site-routes"

const BASE = "https://devfinix.com"

const section = (heading: string, routes: SiteRoute[]) =>
  routes.length
    ? [`## ${heading}`, "", ...routes.map((r) => `- [${r.label}](${BASE}${r.path}): ${r.description}`), ""]
    : []

export const dynamic = "force-static"

export function GET() {
  const byCategory = (category: SiteRoute["category"]) =>
    SITE_ROUTES.filter((r) => r.category === category && r.path !== "/")

  const posts = getAllSummaries().map(
    (p) => `- [${p.title}](${BASE}/blog/${p.slug}): ${p.excerpt}`
  )

  const body = [
    "# Devfinix",
    "",
    "> Full-service digital agency delivering custom web development, mobile app development, and ROI-driven performance marketing. Serving clients worldwide.",
    "",
    ...section("Company", byCategory("Main")),
    ...section("Services", byCategory("Services")),
    ...section("Case Studies", byCategory("Case Studies")),
    "## Blog",
    "",
    ...CATEGORIES.map((c) => `- [${c.name}](${BASE}/blog/category/${c.slug}): ${c.description}`),
    "",
    ...posts,
    "",
    ...section("Free Tools", byCategory("Tools")),
    ...section("Legal", byCategory("Legal")),
  ].join("\n")

  return new Response(body, {
    headers: { "Content-Type": "text/plain; charset=utf-8" },
  })
}

export const dynamic = "force-static" makes this render once at build time, so it costs nothing per request and behaves exactly like the static file — while being regenerated from real data on every deploy. The blog import touches the filesystem, which is fine in a route handler and not fine in anything a client component imports; that constraint is why site-routes.ts deliberately holds no filesystem code and articles are read separately.

There is one thing the generated version loses: editorial judgement. SITE_ROUTES contains everything, including pages that do not belong in a curated list. Filtering by category, as above, is the cheap fix. Adding an llms boolean to the route type is the honest one if the list ever gets long.

Keeping it honest either way

Whichever option you pick, add the assertion to your test suite rather than to your memory. The check is trivial: every link in llms.txt must be a path that exists, and every route flagged as important must appear in llms.txt. We already crawl every URL in sitemap.xml and fail the build on any non-200, so extending that crawl to the links in llms.txt is a few lines. A file full of 404s is worse than no file, and it is exactly the failure a hand-maintained one produces after six months. If your CMS has stopped rendering your pages, this file joins the sitemap, the canonicals and the meta tags on the list of things your front-end now owns, which is one of the underrated costs of going headless.

What is actually known about who reads it

This is where most articles on this term stop being useful, so here is the evidence as it stands, with sources.

Google Search ignores it, on the record. Google's generative AI optimisation guide states: "You don't need to create new machine readable files, AI text files, markup, or Markdown to appear in Google Search (including its generative AI capabilities), as Google Search itself doesn't use them." It adds that maintaining one for other systems "will neither harm nor help your site's visibility or rankings in Google Search." That is as unambiguous as vendor guidance gets.

The labs publish llms.txt files; that is not evidence they read yours. llmstxt.org notes that OpenAI, Anthropic and Google publish llms.txt for their own developer documentation, and OpenAI's own bot documentation links to its /llms.txt as a documentation index. Publishing a file so that agents can navigate your docs is a completely different act from consuming other people's. These get conflated constantly, usually in the same paragraph.

No assistant vendor documents consuming it. Neither OpenAI's crawler documentation, nor Anthropic's, nor Perplexity's describes fetching a site's llms.txt as part of retrieval. Absence of documentation is not proof of absence — but if a system relied on the file, the vendor would have every reason to say so, because it would improve the data they get.

Presence is starting to be measured, which is not the same as being used. Audit tools have begun reporting whether a site serves the file. Passing that check tells you the file exists. It tells you nothing about whether anything read it.

So the defensible position is: llms.txt is a proposed convention with uncertain adoption, explicitly ignored by the largest search engine, published by several labs for their own docs, and not documented as an input by any assistant. Anyone claiming it lifts AI citations owes you a measurement, and nobody has published a credible one. We ship the file because it takes an hour and costs nothing, not because we can show you it works — and we would rather say that than sell you the other version. We do the same on client work where it is cheap: GeoTagImg publishes one alongside a 680-URL multilingual sitemap and hreflang alternates for fifteen languages.

The part that definitely matters: can a crawler render your page

While the industry argues about a Markdown file, a much larger problem goes unexamined on most sites: whether the crawler can see the content at all.

AI crawlers are, broadly, HTTP fetchers. Where a search engine has spent a decade building a rendering pipeline, an assistant's fetcher generally takes the HTML it is given. If your page is a shell that fills in after hydration, what gets extracted is the shell. Google itself, which does render, still warns in the same guide that JavaScript frameworks make this "generally more complex".

The test costs nothing:

# What a non-rendering fetcher sees
curl -s https://example.com/services/seo | wc -c
curl -s https://example.com/services/seo | grep -c 'a distinctive sentence from your page copy'

If the byte count is a few kilobytes of script tags and the grep returns 0, no file at your site root is going to help. In the App Router the usual causes are a page marked as a client component when only a leaf needed to be, content fetched in an effect instead of on the server, and FAQ or tab content that is mounted only when open. All three are ordinary front-end development faults rather than SEO ones, and they belong to the same family of defect that makes a Largest Contentful Paint impossible to fix from the outside. The last of those also quietly breaks structured data, which is covered in detail in the piece on schema markup for AI search.

Fix rendering first. It is the difference between your content existing and not existing for a large class of fetchers, and unlike llms.txt it is not speculative.

Robots rules for GPTBot, ClaudeBot and PerplexityBot

The second thing that definitely matters is that you have not blocked these agents by accident, or allowed them by accident, without deciding which you wanted.

The documented user agents, from each vendor's own pages:

Two details in that table are worth more than the rest of it. Training crawlers and answer crawlers are separate agents at every vendor, so you can decline to be training data while remaining citable. And the user-initiated agents are documented as not necessarily honouring robots.txt when a person supplies a specific URL — check each vendor's current page rather than trusting a table in a blog post, including this one.

On this site the robots rules are generated per request rather than written by hand, because subdomains pointed at the same app must never be indexed:

export const INDEXABLE_HOSTS = ["devfinix.com", "www.devfinix.com"]

export function robotsFor(host: string | null | undefined): MetadataRoute.Robots {
  if (!isIndexableHost(host)) {
    // A subdomain pointed at this app: keep every crawler out
    return { rules: { userAgent: "*", disallow: "/" } }
  }
  return {
    // API routes (e.g. /api/contact) aren't content — keep crawlers out of them
    rules: { userAgent: "*", allow: "/", disallow: "/api/" },
    sitemap: "https://devfinix.com/sitemap.xml",
    host: "https://devfinix.com",
  }
}

// src/app/robots.ts
export default function robots(): MetadataRoute.Robots {
  return robotsFor(headers().get("host"))
}

A wildcard allow covers the AI agents without naming them, which is the right default for a site that wants to be found. If you want to be explicit — usually because someone will ask you to prove it — the App Router takes an array of rules:

export default function robots(): MetadataRoute.Robots {
  return {
    rules: [
      { userAgent: "*", allow: "/", disallow: "/api/" },
      { userAgent: ["GPTBot", "ClaudeBot", "PerplexityBot"], allow: "/", disallow: "/api/" },
    ],
    sitemap: "https://devfinix.com/sitemap.xml",
  }
}

If instead you want to decline training while staying citable, block the training agents and leave the search and user agents alone. That is a commercial decision, not a technical one, and it is worth making explicitly. A marketing site that blocks every AI crawler has removed itself from the tools a growing number of buyers use to shortlist suppliers, usually because someone pasted a snippet from a publisher's blog where the incentives are completely different.

So should you ship one

Ship it if the hour is spare, generate it from your route data so it cannot rot, and hold it loosely. It is a cheap bet on a convention that may or may not get adopted, and the sceptical read — a file almost nothing is documented as reading — is currently the better-evidenced one.

What is not a cheap bet, and what we would do before touching llms.txt on any site: confirm the content is in the server-rendered HTML, confirm robots.txt does not block the agents you want, and make sure the pages worth citing are the ones that exist. That work is the same work that has always made a site legible to machines, which is most of the reason it keeps paying off. It is also the bulk of what we do on an SEO and AEO engagement, and the structured data half of it is covered in the companion piece on schema markup for AI search.