Kazanan
What Is llms.txt? Uses, Limits and Creation Guide
llms.txt is a Markdown guide intended to help AI agents find important information and documents on a website. Publishing it does not prove that every provider reads it or mentions your brand. Decide based on a concrete use case and maintenance capacity.
llms.txt is a proposal, with no search visibility guarantee
In a technical SEO audit, do not prioritize llms.txt ahead of crawlability, indexing and accurate content. Jeremy Howard published the proposal on September 3, 2024; its v2 update appeared on August 10, 2026. It guides agents toward useful documents when seeking information. It is not a compulsory protocol automatically implemented by every model.
Google’s official guidance says Search does not use these files and they neither help nor harm Search visibility or rankings. Google’s generative AI search features do not require special files either. Another system’s documentation workflow may differ; that is a separate question from Google ranking.
A provider publishing a file for its own developer documentation does not establish that it automatically consumes every customer website’s file when choosing answers. “Publication completed” and “the target tool used this content” are separate outcomes. An agency proposal should specify the deliverable and the evidence used to assess it.
It serves a different purpose from robots.txt and sitemaps
The crawling and indexing distinction in the robots.txt guide still applies. llms.txt does not authorize access, implement noindex or protect private files. Listing a document does not replace access controls. Similar filenames do not make these tools interchangeable.
Model training, automatic search crawling and user-initiated document retrieval are different flows. OpenAI’s official crawler documentation describes GPTBot and OAI-SearchBot as separate preferences. Publishing llms.txt does not change those robots preferences, grant training permission or establish a data-use policy enforced across every provider.
| Tool | Purpose | What it does not establish |
|---|---|---|
| Robots.txt | Regulate URL-path crawling by cooperating bots. | Access security or certain removal from an index. |
| XML sitemap | Tell search engines about appropriate URLs. | Guaranteed indexing of every listed URL. |
| llms.txt | Provide a curated entry point to information and documents. | Guaranteed use, citations or rankings. |
| Authentication | Restrict unauthorized access. | Protection cannot be provided by an explanatory text file. |
Choose a use case worth maintaining
A content marketing plan should identify who the file will help. Versioned API documentation for a software library has different needs from a small business site with a few service pages. Replace a hypothetical visibility gain with a testable task, such as finding the installation instructions for the correct product version.
Company information, product guides, return policies, support documents and educational material can be candidates. Manually summarizing stock, prices, promotions or changing requirements can create another source of inconsistency. Start with limited scope and keep the original page as the information source. The file should support a usable website, not replace it.
Document access
Content ownership
Work priorities
Write a short introduction and clear document links
Within your content update process, verify information on the original page before adding it. Exclude unpublished customer data, private contracts, personal information and internal paths. Do not select documents solely by traffic: a rarely visited integration limitation may be essential to completing a task correctly.
The proposal uses an H1 project name, a blockquote summary, H2 document groups and annotated Markdown links. For example: # Example Project The address is illustrative; do not list a .md URL that has not actually been published.
> Installation and usage documentation.
## Documents
- [Start](https://example.com/docs/start.md): Initial setup.
Describe each document’s scope and, where relevant, language and version. Preserve tables, code samples, exceptions and links when extracting page content. Removing an important limitation in pursuit of cleaner text improves appearance while damaging accuracy. Use the actual URL for each language instead of deriving an English address from another language’s slug.
Choose publication methods that fit the platform
Your CMS management approach determines whether the file is static, plugin-generated or built during deployment. FTP and hosting panels work for some sites; public_html and identical filesystem permissions are not universal. Check the actual server ownership and access arrangement instead of applying a blanket “set permissions to 644” instruction.
The v2 proposal allows path-specific files such as /docs/llms.txt as well as a root file. It recommends alternate and describedby link relations for discovering Markdown documents and the relevant guide. Verify that the intended client supports the method; root publication alone does not establish automatic use.
Assign one generator
Check WordPress behavior
Read the live response
Separate file validation from provider adoption
A technical audit should check accessibility, format and content accuracy separately. Inspect UTF-8 characters, headings, Markdown links, HTTP responses and broken URLs. A passing validator does not prove that every agent uses the file. Record which proposal version the tool actually validates.
The Lighthouse agentic browsing documentation describes the file as optional and treats a missing file returning 404 as N/A. This audit is not a Google Search ranking assessment. Do not report a passing technical check as evidence of visibility or sales.
Giving a chatbot the file URL and asking it to find a particular document is a controlled task test. Record the expected answer, source, date, tool and prompt; review missing exceptions or incorrect versions. It does not show that the tool will discover the file independently during ordinary questions or produce the same answer every time.
Define maintenance and measurement before launch
Your content measurement plan should separate file requests, task accuracy, external citations and website conversions. A /llms.txt request in server logs demonstrates access; it does not prove use in an answer. A user-agent string alone also does not conclusively verify the requester’s identity.
A bot request is different from a visitor session. If main pages were updated during the same period, do not attribute subsequent traffic changes to this file. Keep baseline observations, publication dates and concurrent changes. Tool and model changes can also affect comparisons between repeated task tests.
Review the file after version releases, URL moves, document removal and policy changes. Inspect automatically selected pages rather than assuming generation guarantees relevance. Reducing scope that has no verified use and consumes maintenance time is reasonable. The goal is sustainable access to accurate information, rather than increasing the number of published files.
Check llms.txt after moving a document to a new address
When a document moves, apply your site migration checks to llms.txt as well. Record the old address, the new canonical address and the link the file should contain before release.
Fetch the file from the intended host after publication. The listing should point directly to the current public document. If the old link remains in use, check that it reaches the intended destination and that the final response contains the actual document.
If an old file still appears, check the generator and the cache separately. Record the date, the live file and the final destination. This test establishes the state of the file and its links.
Frequently asked questions
Anything else? Write to us.
Does llms.txt improve Google rankings?
Google says Search does not use these files. Publishing one neither helps nor harms Google Search visibility and rankings. Usage by other systems must be assessed separately.
Must the file be at the website root?
The current v2 proposal allows root files and scoped paths such as /docs/. Check the intended client and platform’s discovery method instead of following older claims that every subdirectory is invalid.
Can the file be written in Turkish?
Yes. Use clear descriptions and real document links. Specify language and version scope for multilingual content, and do not invent English Markdown URLs.
Does WordPress require a file manager plugin?
No. Static publication, an appropriate CMS generation process or a documented plugin feature can work. Choose based on permissions, existing files and maintenance ownership; an additional file manager is not compulsory.
Does a correct chatbot summary prove successful deployment?
It is evidence from a controlled content-reading test for that prompt and time. It does not prove automatic discovery, support by all providers or guaranteed citations in ordinary answers.
Can llms.txt replace robots.txt?
No. Content guidance and crawler rules have different purposes. Neither replaces authentication or access permissions for private information.
Define a useful scope and maintenance plan
Share your document structure, intended agent task and content owner so we can assess generation, validation and measurement against a concrete need.
