AnswerDotAI / AnswerDotAI/llms-txt
[Question]: llms.full.txt generated is too large
- Dominant language
- Jupyter Notebook
- Stars
- 2.6k
- Forks
- 163
- Avg merge
- 1m
- Merged PRs (30d)
- 1
Description
I am generating llms.full.txt for [lynx](https://lynxjs.org/) or some large document sites within my company, and I am encountering that llms.full.txt generated is too large
If a document has more than 10000 md files, the generated llms.full.txt is 200MB +
So will there be some issues, such as exceeding the context of the llm? or deployment costs are too high?
Contributor guide
No contributing guide indexed for this repository
Research direction
The issue names no files, tests, or entry points. Start by tracing how llms.full.txt is generated for sites with more than 10,000 Markdown files, then establish the project's expected size and context behavior. Done requires an agreed, concrete requirement for handling very large generated files.
Written by the indexing model from the issue text.
Assessment
- Domain
- ai
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100