AnswerDotAI / AnswerDotAI/llms-txt
Add support for sitemap / multiple llm.txt per domain
- Dominant language
- Jupyter Notebook
- Stars
- 2.6k
- Forks
- 163
- Avg merge
- 1m
- Merged PRs (30d)
- 1
Description
As far as I've noticed the current spec only allows one single file per domain. This makes it harder for tools that have multiple versions and control these via paths.
```
https://example.org # latest version
https://example.org/v1 # version 1
https://example.org/v2 # version 2
```
If the spec only allows a single llm.txt I only see two solutions right now:
1. Only create a llm.txt for the latest version
2. Merge all versions into one big llm.txt
Option 1 doesn't deliver great user experience if you're working with older versions, Option 2 increases tooling on the providers side and could lead into issues when the llm.txt grows so large that it doesn't fix into the context of the model. Additionally I am not sure how good the models could differentiate between the versions.
My suggestion is that there could be some kind of sitemap which tells the llm where it can find specific information.
I've yet have to come up with a great idea for a schema to suggest but wanted to get your feedback on the idea anyway.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reviewing the current llms.txt specification and its single-file-per-domain rule described in the issue. Compare the versioned URL examples and define a concrete sitemap or multiple-file schema, including how tools discover and select versions. Done means the proposal is specific enough to evaluate for inclusion in the specification.
Written by the indexing model from the issue text.
Assessment
- Domain
- documentation
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100