docling-project / docling-project/docling-mcp
local_document_cache is never evicted: RSS grows ~0.65 MB/page for the life of the server
- Dominant language
- Python
- Stars
- 742
- Forks
- 138
- Avg merge
- 11h 30m
- Merged PRs (30d)
- 5
Description
`local_document_cache` and `local_stack_cache` are never evicted from, so a long-running server grows monotonically.
Verified against `v3.2.0` (installed from PyPI, remote conversion mode).
## Where it breaks
`docling_mcp/shared.py:20-21` declares both caches as plain module-level dicts:
```python
local_document_cache: dict[str, DoclingDocument] = {}
local_stack_cache: dict[str, list[NodeItem]] = {}
```
Every conversion inserts into `local_document_cache` (`tools/conversion.py`, `tools/converters/{local,remote}.py`), and the generation/manipulation tools insert into both. Nothing anywhere in the package removes an entry: there is no `.pop()`, no `.clear()`, no `del`, no size or TTL bound, and no tool that lets a client drop a document it is finished with.
The only memory-facing helper is `cleanup_memory()` (`tools/conversion.py:32`), called after each conversion at lines 89 and 141:
```python
def cleanup_memory() -> None:
"""Force garbage collection to free up memory."""
logger.info("Performed memory cleanup")
gc.collect()
```
`gc.collect()` cannot help here — the documents are still strongly referenced by the dict, so they are not garbage. The log line reads as though memory was reclaimed when nothing was.
## Impact
Measured with `DOCLING_MCP_KEEP_IMAGES=false`, remote mode (no models in-process), one container, RSS via `docker stats`:
| state | RSS |
|---|---|
| idle after startup | ~133 MiB |
| + a 16-page PDF | +5 MiB |
| + a 75-page PDF | +49 MiB |
≈ **0.65 MB per converted page, never released**. `KEEP_IMAGES=true` would be considerably worse.
For a short-lived client session this is fine. For a server left running as a shared long-lived process — which is the deployment the streamable-http transport invites — RSS only ever goes up, and the operator's only remedy is restarting the process, which also throws away every still-wanted document key. With no container memory limit set, a busy day can crowd whatever else shares the host.
## Repro
1. Start the server: `docling-mcp-server --transport streamable-http --host 0.0.0.0 --port 8053 conversion`
2. Note RSS.
3. Call `convert_document_into_docling_document` on a few dozen-page PDFs, using a distinct source each time so the content-keyed cache does not short-circuit.
4. RSS climbs by roughly 0.65 MB/page and never comes back down — including after `cleanup_memory()` runs, and with no client sessions open.
## Possible fixes
Roughly in order of effort:
1. Bound the caches — an LRU (`functools.lru_cache`-style or `OrderedDict` + `move_to_end`) with a configurable `DOCLING_MCP_CACHE_MAX_DOCUMENTS` / max-bytes, evicting oldest first.
2. Add an explicit eviction tool (e.g. `drop_document_from_local_cache(document_key)`), so a client that knows it is done can say so. Cheap, and useful regardless of 1.
3. Make `cleanup_memory()` honest — either have it evict, or drop the claim that it frees memory.
Happy to open a PR for (1)+(2) if that shape sounds right to the maintainers.
Contributor guide
Research direction
Start with the cache declarations in docling_mcp/shared.py:20-21, then trace writes from tools/conversion.py and tools/converters/{local,remote}.py and inspect cleanup_memory() in tools/conversion.py. Reproduce the RSS growth using the streamable-http command and distinct PDFs; done means an agreed cache-eviction or explicit-drop behavior is implemented and the misleading memory-cleanup claim is addressed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100