Llms.txt was proposed in September 2024 by Jeremy Howard of Answer.AI. The idea is simple: web pages are full of navigation, scripts and layout that make them hard for large language models to read, so a site can offer a clean, curated guide at yourstore.com/llms.txt. The proposed format is markdown: an H1 with the site name, a short summary in a blockquote, then sections with lists of links and one-line notes. Some sites also publish an llms-full.txt with the full text of key pages.
It is a proposal, not an adopted standard. Unlike robots.txt, it does not allow or block anything. At the time of writing, the major AI providers have not confirmed that they rely on llms.txt when building answers, and Google representatives have said Search does not need it.
Why it matters for eCommerce
Founders hear that llms.txt is the way to appear in ChatGPT or AI Overviews. It is cheap to publish and does no harm, but it is not a shortcut. What drives AI visibility today is the same groundwork: crawlable pages, clear product and policy information, structured data and mentions on sites that AI tools already trust.
How it works in practice
If you publish one, keep it short and useful. List the pages an assistant would need to answer buyer questions: shipping and returns policies, size guides, top collections, buying guides and FAQs. Check how your platform serves a file at the site root, and review the file whenever those pages change.
Common mistakes
- Treating llms.txt as a ranking factor for search or AI answers.
- Trying to block AI crawlers with it, which is the job of robots.txt.
- Dumping hundreds of product URLs into the file instead of a curated guide.
- Publishing it once and letting links go stale.
- Spending time on it before fixing crawlability and product data.