docling-project / docling-project/docling-core
Maximum length for the markdown exporter
Open
enhancement
- Dominant language
- HTML
- Stars
- 282
- Forks
- 214
- Avg merge
- 2d 15h
- Merged PRs (30d)
- 21
Description
When stuffing content to LLMs you want to get the biggest chunk that fits in the maximum context window.
A trivial implementation could simply do `content_md = doc.export_to_markdown()[:MAX_LEN]`. However, this could break sentences, paragraphs, tables, etc.
If we introduce a `max_size` in the `export_to_markdown()` function, we could control it by adding a document item only if it doesn't go above the limit.
Contributor guide
Assessment
This issue has not been assessed yet.