facebook / facebook/docusaurus

New Docusaurus plugin: `docusaurus-plugin-llms-txt`

Open
#10,899 21 comments 36 reactions 0 assignees View on GitHub
feature
Dominant language
TypeScript
Stars
66.2k
Forks
10k
Avg merge
1d 3h
Merged PRs (30d)
52

Description

### Have you read the Contributing Guidelines on issues?

- [X] I have read the [Contributing Guidelines on issues](https://github.com/facebook/docusaurus/blob/main/CONTRIBUTING.md#issues).

### Description

Hello!

I'd like to propose the creation of a new docusaurus plugin for the creation of a `llms.txt` file.

As background, an `llms.txt` file is a way of co-locating needed information such that an LLM processing web pages is able to do so effectively and quickly. More information can be found [on the llms.txt proposal website](https://llmstxt.org/)

The goal of this feature is to create a plugin that will generate this file for your docs.

### Has this been requested on Canny?

No

### Motivation

Having an `llms.txt` is a business need for our company, but I'm certain it would be helpful for others. I have already created a proof-of-concept plugin that writes the file, but I wanted to bring this to the community as a whole to see how the design could be improved and/or match others' use cases.

Additionally, many docs sites that I use and admire, such as [dub.co](https://dub.co/docs/introduction) and [Cloudflare](https://developers.cloudflare.com/) have already implemented `llms.txt` files. The former via Mintlify, the latter via a script.

Adding a new, optional plugin would give Docusaurus users the ability to opt-in to this new standard and allow their documentation to be accessed easily by LLM agents.

### API design

The design is very similar to the [sitemap plugin](https://docusaurus.io/docs/api/plugins/@docusaurus/plugin-sitemap).

**Configuration**
- `ignorePatterns`: ability to ignore specific paths from being included in the `llms.txt`.
- `filename`: ability to change the name and output path of the `llms.txt` file.

I also considered configuration modifying the output of the file, but the `llms.txt` proposal is fairly set on the file structure so I declined to add that as a configuration option.

### Have you tried building it?

Yes! I made a simple plugin that works for our use case. I would like to move it into this repo in order to make it available to other folks and to get feedback on the design. I feel it could be made much simpler and cleaner 😄

### Self-service

- [X] I'd be willing to contribute this feature to Docusaurus myself.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.