llms.txt

The llms.txt sits at a website's root URL (for example https://example.ch/llms.txt) and is meant to help large language modelsA Large Language Model (LLM) is an AI model that learns from large amounts of text and generates formulated answers from it. LLMs form the core of generative answer systems such as ChatGPT, Perplexity or Gemini.Open glossary entry → (LLMs) grasp relevant content faster. The idea was proposed in 2024 by Jeremy Howard and Answer.AI.

Unlike robots.txtThe robots.txt is a publicly accessible text file in a site's root directory. It tells crawlers which URLs they may fetch and therefore controls crawling.Open glossary entry →, llms.txtllms.txt is a voluntary Markdown file in a website's root directory that gives language models a compact overview of important content. It is a proposal or de facto format, not a binding official web standard.Open glossary entry → does not control access but explains which content is important:

• robots.txt decides whether a bot may crawlCrawling is the process by which a search engine bot fetches a URL and downloads its content. It is the first step before a page can be indexed.Open glossary entry →,
• llms.txt provides guidance on which pages are central.

A good llms.txt is curated, not a second sitemapAn XML sitemap is a machine-readable file listing a website's important URLs. It helps search engines discover content more efficiently but does not replace internal linking or guarantee indexing.Open glossary entry →. It lists central pages with a short description, only public and current content, and no confidential information. It should be aligned with robots.txt, the sitemap and canonicals.

Important: llms.txt is not a ranking or citation standard and no guarantee of mentions. It is an optional optimization opportunity, especially useful for SaaS websites, documentation and guide portals.